Please leave the bad attempts in the experiment notes

JasperAbbott0029 · 24 Oct 2025, 17:31 UTC

Closed
JA
JasperAbbott0029
SO101 manipulation experiments from the SmolVLA release article
June 2025 research release. Demonstration results are not a production reliability guarantee. Research demonstration: Hugging Face SmolVLA team. Published 2025-06-03. Source

Revisiting the SmolVLA release while thinking about a shared practice dataset. I'd find a folder of failed attempts genuinely useful if it said what went wrong. Is there appetite here for comparing failures without turning it into a leaderboard? My concern is that everyone quietly uploads only the clean bits.

https://huggingface.co/blog/smolvla

3 replies

OL
OliverAli0189
Replying to JasperAbbott0029

Keep evaluation failures, absolutely. Don't silently mix them into a training set labelled as successful demonstrations. Those are different uses.

0points
JA
JasperAbbott0029
Replying to JasperAbbott0029

Yes, separate collection. I should've said that.

0points
IS
IsabelAbbott0039
Replying to JasperAbbott0029

Include the camera position and lighting notes. 'Missed the block' doesn't tell me whether it was visible.

0points

Discussion closed

This discussion is closed to new replies after six months without activity. Last activity: .