I was asked to rank two handling proposals, one using an FR10, from their demonstration videos. The first used a clean machined tray. The second used our returnable tray, including the awkward corner pockets.
Both suppliers selected a run in which the spacers were placed successfully. Neither supplied setup time or a record of operator interventions. The internal comparison sheet nevertheless has a winner highlighted in green, based on a small difference in the visible cycle.
I am not comfortable signing that sheet. What common trial would let us compare the proposed systems without pretending those videos held everything else constant?
We had a similar comparison where the faster cell needed a person sorting parts off-camera. Once that work was included, the decision looked rather different
Our lab compared two dispensers using little piles of hand-selected parts. Both looked wonderful. Then the technician who actually prepared the work asked where the rest of the bag had gone. We'd removed nearly every awkward sample before the trial. Nobody meant to cheat, we just wanted the demo to go smoothly.
I'd let the operators choose a representative incoming set and write down any preparation. Also ask for the unedited run and intervention notes. The rejected and interrupted bits can tell you more about owning the equipment than another minute of successful transfers.
Careful with that. Representative awkward samples, yes. Every damaged tray you can find, no. Otherwise you've replaced a flattering demo with a punishment test that doesn't describe the workshop either.
We have a proposed incoming set selected with production, including the corner-pocket variation we routinely see; both suppliers will receive the same input description and acceptance criteria.