half our scratch examples might be reflections

For that pair, yes. There's a sample number in the uncropped originals. Some older pictures have no identity anywhere, so I'm putting those aside for now rather than inventing one.

Keep the unidentified pictures as examples of the imaging problem if useful, but they cannot support a traceable acceptance reference in their present form.

That distinction matters here too; a changing highlight demonstrates presentation sensitivity, but does not prove the faceplate has no genuine defect elsewhere.

Can you reproduce the changing line with one identified sample before collecting hundreds more images?

A controlled small comparison would help define what the larger collection must record, including illumination, camera settings and sample presentation.

We've reproduced it at the bench with the same faceplate and a small orientation change. Quality also found a real fine scratch that is visible in only some views. Both effects need handling.

That's a useful awkward sample. Keep it; the easy images won't tell you whether the setup handles the ambiguity you're actually worried about.

It's identified and retained now. I'm adding the physical inspection result beside the paired images, with the setup details. The uncertain labels are coming out of the training folder.

Coming out of training, yes, but keep a review set too. Sorry, I mean retain the uncertainty somewhere it can be revisited rather than deleting the troublesome pictures.

Use an explicit unresolved judgement for those examples; forcing them into the accepted or rejected class would make the dataset appear more certain than the inspection team is.

Who approves changes to that reference set after the cell is handed over?

Assign that responsibility alongside the imaging setup ownership, so a lighting adjustment does not leave old reference images silently representing a different inspection condition.

Quality will own judgements; the vision lead will own acquisition settings. We have a small traceable review set now, not an approved production classifier. The old screenshots are retained outside training with their limitations.

That gives you a defensible starting point; the next question is whether the chosen imaging conditions reliably reveal the defects your product criteria require you to find.