Please leave the bad attempts in the experiment notes

JasperAbbott0029 · 24 Oct 2025, 17:31 UTC

Closed
JA
JasperAbbott0029
Revisiting the SmolVLA release while thinking about a shared practice dataset. I'd find a folder of failed attempts genuinely useful if it said what went wrong. Is there appetite here for comparing failures without turning it into a leaderboard? My concern is that everyone quietly uploads only the clean bits. https://huggingface.co/blog/smolvla
SO101 manipulation experiments from the SmolVLA release article
SmolVLA release and experiments

June 2025 research release. Demonstration results are not a production reliability guarantee.

Research demonstration: Hugging Face SmolVLA team · · Source record

3 replies

OL
OliverAli0189
Replying to JasperAbbott0029

Keep evaluation failures, absolutely. Don't silently mix them into a training set labelled as successful demonstrations. Those are different uses.

0 points
JA
JasperAbbott0029
Replying to JasperAbbott0029

Yes, separate collection. I should've said that.

0 points
IS
IsabelAbbott0039
Replying to JasperAbbott0029

Include the camera position and lighting notes. 'Missed the block' doesn't tell me whether it was visible.

0 points

Discussion closed

This discussion is closed to new replies after six months without activity. Last activity: .