PR
Replying to GabrielAbbott0018
Part of the task. I'd record whether it asks for clarification and whether the clarified request changes the outcome. Otherwise you're only scoring the neat successful examples.
0points
Still thinking about the March Gemini Robotics demos. The behaviour I'd most like in a small parts-sorting experiment is refusing an ambiguous instruction. Two similar trays, vague request, no confident guess. Would you test that as part of the task, or treat it as a separate interface problem? I'm talking about a proposed experiment, not something I've verified the model does.
https://deepmind.google/blog/gemini-robotics-brings-ai-into-the-physical-world/