Some labels came from the pictures alone. We still have the samples, but those labels aren't tied to physical inspection results. That's irritatingly circular

@CarlaBarnes0575 Keep the image-only labels uncertain pending physical inspection. A controlled comparison of lighting and presentation can then investigate whether the moving streaks indicate reflection effects.

A scratch's visibility can change with presentation too. Don't let 'moving streak' quietly become the rule for relabelling something acceptable.

Correct. I meant a reason to investigate the imaging, not a new defect label. Physical acceptance stays with the agreed inspection method.

So uncertain isn't the same as good? Where do those images go?

Retain their sample identities and undecided status. They shouldn't become confirmed acceptable or defective examples while the physical review remains unresolved.

@FarahAli0243 Our two comparison folders changed exposure and presentation together. I can't tell which change made the images look better. That comparison needs a caveat too

On my inspection setup, a folder called 'better lighting' also contained easier samples. The folder name was doing a lot of unearned work.

What did you keep beside each image? I can manage sample ID and settings, but I don't want a form nobody fills in

On my setup we kept sample ID, exposure, lighting arrangement and presentation condition. Enough to explain a comparison without reconstructing the whole session from memory.

Those records can support the vision specialist's comparison of image conditions. Holding the representative sample set constant will make an acquisition change easier to assess.

Would a learned model just cope with the lighting differences?

It still needs trustworthy labels and representative evaluation. A different model doesn't establish whether your reference scratches are real or your acquisition comparison is fair.

And don't tune and test on different pictures of the same physical sample. That isn't the independence the result would suggest.

I'll keep all images of one sample together when we split tuning and evaluation sets. Our file names alone wouldn't have caught that overlap

The reference problem is clearer, but I still can't judge inspection performance from our example images. The physical labels and image comparisons need more work

The remaining physical-label and comparison work explains the open judgement. Clearer uncertainty doesn't yet establish inspection performance.