rl & robots @UW @NVIDIA, prev @berkeley_ai
Seattle, WA
- Tweets 107
- Following 108
- Followers 814
- Likes 636
Pinned Tweet
This is the most impressive result I’ve seen in my PhD.
One policy, trained from scratch, solves all 20 NIST assembly tasks. It can even complete 10 tasks in a row, with a single binary reward only after all 10 are done.
@octi_zhang combined diverse resets from OmniReset, task sampling from SGS, and multi-task training to unlock finer dexterity and longer horizons. No extra GPUs needed: one run on the same hardware replaces 20 specialist runs.
Videos and details: alpha-nist.github.io
Proud to be part of this incredible work! More in his thread 👇
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
18
59
887
42,868
Patrick Yin retweeted
Every robot already carries contact sensors: its own joints. When a robot touches something, its joints drift slightly off their commands, and the encoders pick it up. Legged robots and humanoids already leverage this signal to walk blind - but is it enough for blind precise assembly?
We find that it is! Our policies can insert pegs, mesh gears, and thread nuts using only the robot's joint positions - no camera, no tactile, and no object poses. And the peg policy works zero-shot in the real world! 🧵
w/ my amazing co-first author @younghyo_park, and advised by @kushalk_ and @pulkitology.
18
54
418
34,510
Patrick Yin retweeted
Can a robot assemble tight-clearance parts using only its proprioception? No camera, no tactile sensor, no F/T sensor, no joint torque sensing, no priors over the object geometries or poses: just joint position readings.
It actually can! Blind policies trained in simulation insert pegs, mesh gears, and thread nuts, and the peg policy works zero-shot in the real world. Here's what we found 🧵
Joint work with @BronarsToni, my wonderful co-first author, who's on the job market right now! Advised by @kushalk_ and @pulkitology.
24
56
503
41,625
Patrick Yin retweeted
Exploration is hard for RL when training high-DoF dex hands ✋
X-Reset makes generalist object-centric RL easier by resetting to human hand-object states 🙋♀️
We mitigate the exploration problem, and train a single policy to manipulate a variety of objects to any goal pose 🧵
3
17
124
14,682
Patrick Yin retweeted
Three AlphaNIST findings I'm particularly excited about:
1. Task diversity can make difficult tasks easier to learn.
2. Dexterity isn't just about more fingers.
3. Long-horizon coordination emerges without explicit planning.
All with binary task reward!
See @octi_zhang 's thread
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
5
17
967
Patrick Yin retweeted
Amazing to see this level of dexterity emerging from a binary success / fail reward. Great work @octi_zhang !
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
2
3
26
1,811
this is super cool, multi-task RL for dexterous assembly!
I'm surprised that multi-task training shows positive transfer between tasks with only 20 tasks, perhaps it's because the skills required are similar? Either way, makes the case for sim rl stronger 😁
(4/6) We started this project with one goal: solve all 20 NIST assembly tasks with RL. Our earlier work on Success-Guided Sampling (SGS) had already solved one of them, so we reused its recipe: a binary task reward with a simple control cost, a broad reset distribution from OmniReset, and the SGS task sampler. Training a specialist policy for each task got us 17 of the 20. The last three, the 4 mm peg, 4 mm rod, and M4 nut, wouldn’t budge. Their policies couldn’t even pick the parts off the table: a slight miss with the gripper pushed these tiny parts away instead of pinching them.
The obvious fix was to design easier resets or denser rewards for those three until they learned. Instead, we wondered: could skills learned on the other 17 tasks help with the hard grasps the last three needed? So we tried what seemed like the harder path: one policy trained on all 20 tasks at once, with each episode starting from any of the 20 parts.
Then the magic happened. The generalist learned to pick up and assemble all three tiny parts, alongside the other 17. Multi-task training, the seemingly harder path, made the hardest tasks easier to learn.
2
20
950
Patrick Yin retweeted
@octi_zhang and team with some pretty cool new results :)
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
6
58
3,108
Patrick Yin retweeted
Simulation will be a bitter lesson for human data folks 😁
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
9
20
496
38,081
Patrick Yin retweeted
@octi_zhang is a legend. the multitask policy learning super impressive
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
3
14
396
Patrick Yin retweeted
At last… torturous, terrifyingly tiny tasks have been solved by RL.
The M4 nut, 4mm peg, and 4mm rod were too small for any single-task policy to learn. Training on all 20 NIST tasks at once enabled success on all of them. And the same policy can do 10 tasks back-to-back, with reward only once at the very end.
Videos and details: alpha-nist.github.io
More in @octi_zhang’s thread
(3/6) We also never told it what order to work in. Over long horizons, AlphaNIST began organizing its own work: bringing parts within reach, moving obstructions aside, and returning to pieces it had left unfinished. It received a single binary reward only when the entire multi-step assembly was complete, so the task order and strategy emerged purely from RL.
3
13
1,094
Patrick Yin retweeted
What does RL learn from 10 assembly tasks in a row, with sparse reward only at the very end of all 10 tasks? Traditionally, absolutely nothing!
That’s 1,125 actions over 45 seconds. Even if a random policy somehow nails each task 1% of the time, chaining all 10 happens with probability 10⁻²⁰. Worse odds than winning Powerball twice in a row.
Excited to share AlphaNIST, a single policy which achieves 98.2% success rate across all tasks, and is able to solve 10 assemblies in a row under sparse reward! Check out our website: alpha-nist.github.io.
More details in @octi_zhang ‘s thread, and some extra comments of my own below! 🧵
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
7
14
158
8,685
This is the most impressive result I’ve seen in my PhD.
One policy, trained from scratch, solves all 20 NIST assembly tasks. It can even complete 10 tasks in a row, with a single binary reward only after all 10 are done.
@octi_zhang combined diverse resets from OmniReset, task sampling from SGS, and multi-task training to unlock finer dexterity and longer horizons. No extra GPUs needed: one run on the same hardware replaces 20 specialist runs.
Videos and details: alpha-nist.github.io
Proud to be part of this incredible work! More in his thread 👇
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
18
59
887
42,868
Beyond the collaborators @octi_zhang thanks in his thread, special thanks to Karl Van Wyk, who invented the @NIST board, and to Joe Falco and Kenneth Kimble for their work on it since. None of this would exist without them.
4
746
Patrick Yin retweeted
Failures are inevitable for robots that are deployed in the *big* world. We introduce ReSYNC, a system that leverages large-scale RL sim2real to recover from local failures and concept discovery to explain/avoid new failures. Check it out here: jaraxxus-me.github.io/ReSYNC…
12
28
85
7,797
Patrick Yin retweeted
since people seemed to enjoy our locomotion results, chef 🧑🍳@octi_zhang cooked up a special treat à la SGS on a different embodiment (a wheeled @UnitreeRobotics robot) to celebrate the release! The robot is placed in this big world, and its job is to reach the yellow dots.
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
4
14
105
4,550
Patrick Yin retweeted
The value of scaled RL training in simulation can't be understated. Huge congratulations to @octi_zhang and everyone else who worked on the SGS paper. What comes next is even cooler - stay tuned!
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
3
10
53
8,531
Patrick Yin retweeted
Super impressive work and recovery behaviors from friends at UW! There's a night and day difference between RL trained models vs behavior cloned ones but It's still challenging to scale this up and apply it to different usecases. It needs a couple of ingredients: Good assets that look good and simulate well. The parts are visually simple (no texture, simple material) so that's a win for sim because it makes the rendering realistic. The only tricky object is the reflective board itself which needed to be taped to make sim renderings close enough to real. Also another part is what would happen if the board is not bolted to the table, the recovery behaviors are too wild for a loose board.
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
3
5
42
6,844
Patrick Yin retweeted
so much more to unlock with simulation :)
very cool work! you know this is good when you see others already reproducing it
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
5
5
97
6,730
We kept throwing new tasks at SGS, and the same recipe kept working. New assembly tasks, new terrains, new embodiments. I really believe this is a step toward one RL recipe for every embodiment and task!
(4/5) The same training recipe works for arms and legs! We believe SGS provides one piece of the puzzle towards a unified RL recipe across all embodiments and tasks.
3
2
48
2,172
Patrick Yin retweeted
A little curriculum learning can make a world of a difference for contact-rich robotic RL! @octi_zhang @mateoguaman @patrickhyin @rosario_scaling showed that choosing sampling strategies carefully while using resets to drive RL exploration in simulation can really shine on high-precision problems in our new work SGS!
The most intuitive way I think about the methodology is by ensuring that the learning signal in any batch of reset-driven RL is maximized by adaptively concentrating reset sampling around states that are "moderate" empirical success rate. This ensures the batch is not wasted on states that are too successful or unsuccessful, enabling RL to solve high-precision problems such as contact-rich assembly and dynamic quadruped locomotion.
This results in some of the most fun robot behaviors we've seen yet on a real robot! I'd recommend reading @octi_zhang's thread and our paper for a more thorough description. Hope y'all have fun using it, it's dead simple!
Website: sgs-rl.github.io
Paper: arxiv.org/abs/2610.12465
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
4
8
135
8,101