@AaronBaker0436 I've found that some labels were assigned from the images themselves. The samples are still available, but those labels have no corresponding physical inspection results, making the reference circular.

@DineshChan1082 Mark those labels uncertain and have the samples inspected properly. Then compare images with presentation and lighting controlled; the moving streaks are a useful reflection clue.

@AaronBaker0436 That clue needs a limit: presentation can also change how a real scratch appears. Movement in the image isn't enough to reclassify the sample as acceptable.

Correct. I meant a reason to investigate the imaging, not a new defect label. Physical acceptance stays with the agreed inspection method.

So uncertain isn't the same as good? Where do those images go?

Retain their sample identities and undecided status. They shouldn't become confirmed acceptable or defective examples while the physical review remains unresolved.

I've checked the image folders and found that exposure and presentation both changed between them. Their improved appearance can't be attributed to either change individually.

@DineshChan1082 I had a separate inspection setup where the 'better lighting' folder used easier samples too. Its name implied a lighting conclusion that the comparison didn't support.

Which details did you retain with each image? I can record sample identity and settings, but I'd like a practical record people will actually maintain.

@DineshChan1082 On my setup we kept sample ID, exposure, lighting arrangement and presentation condition. Enough to explain a comparison without reconstructing the whole session from memory.

Use those details to ask your vision specialist for controlled comparisons on representative surfaces. Keep the sample set fixed when assessing an acquisition change.

@AaronBaker0436 Would a learned model just cope with the lighting differences?

It still needs trustworthy labels and representative evaluation. A different model doesn't establish whether your reference scratches are real or your acquisition comparison is fair.

@AaronBaker0436 Keep physical sample identity in mind when forming evaluation sets. Different images of a sample used for tuning don't provide an independent sample-level evaluation.

@CarlaBrooks0836 I'll group images by physical sample before dividing the tuning and evaluation sets. The current filenames wouldn't reliably expose the same sample appearing in both.

I can explain the reference uncertainty better, although the example images still don't support a performance judgement. Physical classification and image-condition comparisons remain outstanding.

That follows from the reference gap. Explaining it more clearly helps, but the images still can't carry a performance claim.