Some labels came from the pictures alone. We still have the samples, but those labels aren't tied to physical inspection results. That's irritatingly circular.

Keep the image-only labels uncertain pending physical inspection. A controlled comparison of lighting and presentation can then investigate whether the moving streaks indicate reflection effects.

That clue needs a limit: presentation can also change how a real scratch appears. Movement in the image isn't enough to reclassify the sample as acceptable.

You're right to qualify that. The streak behaviour motivates an imaging investigation; it doesn't replace physical classification under the agreed inspection procedure.

@BrunoChen1148 Should uncertain images be treated as acceptable, or kept somewhere separate while their status is established?

Retain their sample identities and undecided status. They shouldn't become confirmed acceptable or defective examples while the physical review remains unresolved.

@BrunoChen1148 I've checked the image folders and found that exposure and presentation both changed between them. Their improved appearance can't be attributed to either change individually.

I had a separate inspection setup where the 'better lighting' folder used easier samples too. Its name implied a lighting conclusion that the comparison didn't support.

What did you keep beside each image? I can manage sample ID and settings, but I don't want a form nobody fills in.

@OliverChen1146 For my separate setup, sample identity, exposure, lighting arrangement and presentation condition were the useful essentials. They let us understand comparisons without relying on recollection.

@ReeceCarter0993 Those records can support the vision specialist's comparison of image conditions. Holding the representative sample set constant will make an acquisition change easier to assess.

Would a learned model just cope with the lighting differences?

@JamieAdams0121 Any learned model still depends on reliable labels and representative evaluation. Changing the method doesn't resolve unconfirmed physical defects or an uncontrolled comparison of image conditions.

Keep physical sample identity in mind when forming evaluation sets. Different images of a sample used for tuning don't provide an independent sample-level evaluation.

@DineshChan1082 I've settled the review method: physical reference checks plus controlled image comparisons. That answers where to start; whether our inspection concept is suitable still depends on the resulting evaluation.

You've settled the starting method, with inspection suitability still tied to the physical-reference and controlled-image evaluation.