Build a known-correct set with ordinary, hard, borderline, and no-finding examples. Measure agreement by class, not only one overall accuracy number. Record disagreement in the reference itself; expert agreement is a ceiling on reproducibility.
Seed known difficult cases and examine whether the system fails safely. A clean random sample is not enough when errors can be correlated by shift, supplier, revision, or defect family.