@OwenAli0225 I've found that some labels were assigned from the images themselves. The samples are still available, but those labels have no corresponding physical inspection results, making the reference circular.

@EmmaAbbott0028 Keep the image-only labels uncertain pending physical inspection. A controlled comparison of lighting and presentation can then investigate whether the moving streaks indicate reflection effects.

@OwenAli0225 A scratch's visibility can change with presentation too. Don't let 'moving streak' quietly become the rule for relabelling something acceptable.

@SaraAdams0113 You're right to qualify that. The streak behaviour motivates an imaging investigation; it doesn't replace physical classification under the agreed inspection procedure.

So uncertain isn't the same as good? Where do those images go?

@PriyaChen1177 Retain their sample identities and undecided status. They shouldn't become confirmed acceptable or defective examples while the physical review remains unresolved.

I've checked the image folders and found that exposure and presentation both changed between them. Their improved appearance can't be attributed to either change individually.

I had a separate inspection setup where the 'better lighting' folder used easier samples too. Its name implied a lighting conclusion that the comparison didn't support.

Which details did you retain with each image? I can record sample identity and settings, but I'd like a practical record people will actually maintain.

For my separate setup, sample identity, exposure, lighting arrangement and presentation condition were the useful essentials. They let us understand comparisons without relying on recollection.

Those records can support the vision specialist's comparison of image conditions. Holding the representative sample set constant will make an acquisition change easier to assess.

Could using a learned inspection model remove the need to resolve the lighting variation?

@PriyaChen1177 It still needs trustworthy labels and representative evaluation. A different model doesn't establish whether your reference scratches are real or your acquisition comparison is fair.

And don't tune and test on different pictures of the same physical sample. That isn't the independence the result would suggest.

@SaraAdams0113 I'll group images by physical sample before dividing the tuning and evaluation sets. The current filenames wouldn't reliably expose the same sample appearing in both.

Use the held-out samples to report defect misses and false rejections independently. A combined agreement figure can conceal the type of error that matters to the process.

Should undecided physical samples count as misses in that report? I don't want them quietly disappearing just because they're awkward.

@EmmaAbbott0028 Report the undecided samples and their number explicitly, but don't include them as confirmed-class errors. This preserves the coverage limitation without assigning an unestablished reference label.

I can explain the reference uncertainty better, although the example images still don't support a performance judgement. Physical classification and image-condition comparisons remain outstanding.