The infrastructure partner to the world's most ambitious robotics builders.

We ran every trial and scored every episode within 24 hours. If you’d like your model benchmarked on real robots, reach out!
2
137
The costs are also much lower! Up to 41x cheaper per successful run.
1
1
7
474
Evaluations were run on an early preview of Dream-1 with the same control loop interface provided to the LLMs. Better performance is expected with lower latency and the recommended control loop.
1
4
152
Dream-1 outperformed Opus and Fable. While strong on dexterity and manipulation, it’s weaker on reasoning (e.g. on tasks like "Follow the red sticky note"), performing just behind Sol and Astra overall.
1
7
456
Congrats to @dreamlabs_ai on their launch of Dream-1! We evaluated an early preview of the model on our LLM focused benchmark. Despite this, Dream-1 performed well at significantly faster execution speeds!
2
8
64
4,824
OpenAI’s GPT-6 Astra and GPT-6.1 Sol scored higher on our evaluations than their Anthropic counterparts, Fable 5.1 and Opus 5.5. In fact, Sol scored even higher than Astra did.
16
11
224
37,613
If you missed our commentary on Astra and Fable, you can check out more below.
We heard the hype around LLMs and robotics, so we put Astra and Fable to the test. 16 tasks testing 13 capabilities. Our finding: both did surprisingly well on reasoning and planning and struggled with dexterity. But Astra did better overall. Here’s what happened:
1
5
673
We’ll be releasing more information about our evaluations of LLMs on robotics tasks in the coming days. If you’d like to join us in building infrastructure for general purpose robots, we are hiring! xdof.ai/careers
7
598
We heard the hype around LLMs and robotics, so we put Astra and Fable to the test. 16 tasks testing 13 capabilities. Our finding: both did surprisingly well on reasoning and planning and struggled with dexterity. But Astra did better overall. Here’s what happened:
10
13
109
13,511
Fable is worse here. While Astra understood that “serve cereal on the tray” involves pouring cereal, Fable often thought it was a simple pick-and-place.
1
8
328
We’ll be releasing more information about our evaluations of LLMs on robotics tasks in the coming days. If you’d like to join us in building infrastructure for general purpose robots, we are hiring! xdof.ai/careers
1
8
319