Usually reported as a percentage agreement or a statistic such as Cohen’s kappa. Low agreement means the scorecard is ambiguous, the reviewers are untrained, or both, and it means the scores that come out of it cannot support the decisions being made from them.