@HanaAllen0274 I've found that some labels were assigned from the images themselves. The samples are still available, but those labels have no corresponding physical inspection results, making the reference circular.

Keep the image-only labels uncertain pending physical inspection. A controlled comparison of lighting and presentation can then investigate whether the moving streaks indicate reflection effects.

A scratch's visibility can change with presentation too. Don't let 'moving streak' quietly become the rule for relabelling something acceptable.

Correct. I meant a reason to investigate the imaging, not a new defect label. Physical acceptance stays with the agreed inspection method.

Should uncertain images be treated as acceptable, or kept somewhere separate while their status is established?

Keep them explicitly undecided, with the sample link. Don't use them as confirmed examples of either class until the physical review settles them.

@HanaAllen0274 I've checked the image folders and found that exposure and presentation both changed between them. Their improved appearance can't be attributed to either change individually.

On my inspection setup, a folder called 'better lighting' also contained easier samples. The folder name was doing a lot of unearned work.

Which details did you retain with each image? I can record sample identity and settings, but I'd like a practical record people will actually maintain.

On my setup we kept sample ID, exposure, lighting arrangement and presentation condition. Enough to explain a comparison without reconstructing the whole session from memory.

Use those details to ask your vision specialist for controlled comparisons on representative surfaces. Keep the sample set fixed when assessing an acquisition change.

Could using a learned inspection model remove the need to resolve the lighting variation?

@FarahChan1113 It still needs trustworthy labels and representative evaluation. A different model doesn't establish whether your reference scratches are real or your acquisition comparison is fair.

@HanaAllen0274 And don't tune and test on different pictures of the same physical sample. That isn't the independence the result would suggest.

@CarlaAllen0314 I'll group images by physical sample before dividing the tuning and evaluation sets. The current filenames wouldn't reliably expose the same sample appearing in both.

I can explain the reference uncertainty better, although the example images still don't support a performance judgement. Physical classification and image-condition comparisons remain outstanding.

That follows from the reference gap. Explaining it more clearly helps, but the images still can't carry a performance claim.