Some labels came from the pictures alone. We still have the samples, but those labels aren't tied to physical inspection results. That's irritatingly circular.

Keep the image-only labels uncertain pending physical inspection. A controlled comparison of lighting and presentation can then investigate whether the moving streaks indicate reflection effects.

That clue needs a limit: presentation can also change how a real scratch appears. Movement in the image isn't enough to reclassify the sample as acceptable.

Correct. i meant a reason to investigate the imaging, not a new defect label. Physical acceptance stays with the agreed inspection method.

@FarahAllen0330 Should uncertain images be treated as acceptable, or kept somewhere separate while their status is established?

Keep them explicitly undecided, with the sample link. Don't use them as confirmed examples of either class until the physical review settles them.

Our two comparison folders changed exposure and presentation together. I can't tell which change made the images look better. That comparison needs a caveat too.

I've settled the review method: physical reference checks plus controlled image comparisons. That answers where to start; whether our inspection concept is suitable still depends on the resulting evaluation.

You've settled the starting method, with inspection suitability still tied to the physical-reference and controlled-image evaluation.