Our planned planning camera inspection of reflective parts in a precision metal-parts inspection area has Fairino FR5 presenting a polished metal cover to a fixed camera. The example images contain bright streaks that shift with small presentation differences, yet those streaks are being labelled as scratches. Before commissioning, I'd like to establish reliable references and image conditions for threshold tuning.
@RosaAbbott0031 Check whether the suspected scratches were confirmed on physical samples under the agreed appearance procedure. Link each image to its sample identity and inspection result first.
@JaneBarnes0597 Some labels came from the pictures alone. We still have the samples, but those labels aren't tied to physical inspection results. That's irritatingly circular.
Keep the image-only labels uncertain pending physical inspection. A controlled comparison of lighting and presentation can then investigate whether the moving streaks indicate reflection effects.
@JonasAli0198 Correct. I meant a reason to investigate the imaging, not a new defect label. Physical acceptance stays with the agreed inspection method.
Retain their sample identities and undecided status. They shouldn't become confirmed acceptable or defective examples while the physical review remains unresolved.
@JaneBarnes0597 Our two comparison folders changed exposure and presentation together. I can't tell which change made the images look better. That comparison needs a caveat too.
I had a separate inspection setup where the 'better lighting' folder used easier samples too. Its name implied a lighting conclusion that the comparison didn't support.
For my separate setup, sample identity, exposure, lighting arrangement and presentation condition were the useful essentials. They let us understand comparisons without relying on recollection.
Those records can support the vision specialist's comparison of image conditions. Holding the representative sample set constant will make an acquisition change easier to assess.
Any learned model still depends on reliable labels and representative evaluation. Changing the method doesn't resolve unconfirmed physical defects or an uncontrolled comparison of image conditions.
Keep physical sample identity in mind when forming evaluation sets. Different images of a sample used for tuning don't provide an independent sample-level evaluation.
@JonasAli0198 I'll keep all images of one sample together when we split tuning and evaluation sets. Our file names alone wouldn't have caught that overlap.
@RosaAbbott0031 Use the held-out samples to report defect misses and false rejections independently. A combined agreement figure can conceal the type of error that matters to the process.
@JaneBarnes0597 The reference problem is clearer, but I still can't judge inspection performance from our example images. The physical labels and image comparisons need more work.