Assistant Professor @NYU_Courant; Staff Research Scientist @GoogleDeepMind. Previously Post-Doc @Stanford, PhD @UCBerkeley, M.Eng. / B.S. @MIT.
- Tweets 293
- Following 508
- Followers 5,242
- Likes 408
World models go beyond robotics.
Autoresearch and agentic ML engineering face a familiar bottleneck: expensive experiments.
Introducing MLE-World: learn from past experiments to predict new outcomes before running them.
Paper, model, data, code: mle-world.github.io/
🚀 Toward RSI: AI agents automate ML research, but each step trains and evaluates a model, limiting test-time scaling.
Our new paper, MLE-World, asks: can agents explore more ideas by predicting experiment outcomes before running them? 🧵⬇️
ALT MLE-World overview: an execution bottleneck, predicted terminal feedback with confidence gating, and higher mean leaderboard percentiles with less real execution.
Made with AI
5
6
68
4,238
This complements our Reward-Rate Policy Gradient work on MLE agents:
RPG learns to maximize reward per unit time.
MLE-World reuses experience to make experiment feedback cheaper.
Two complementary directions for making better use of a limited research budget.
nitter.siddz.com/sherryyangML/status/21…
Excited to share Reward-Rate Policy Gradient: RL that accounts for execution time.
For autoresearch and ML engineering agents, actions can take seconds or hours. A fixed-step MDP misses this distinction.
We optimize MLE agents' reward per unit time in a semi-MDP.
arxiv.org/pdf/2609.36393
muhang-tian.com/reward-rate
See details below:
1
1
4
187
Why should agentic robots rediscover what humans already know?
Excited to share our new work, MoCaP (mobile Code-As-Policy), which turns passive human videos of everyday life into a head start for agentic robots: where things are and how to approach them.
Human experience → less robot trial and error.
Coding agents are emerging as general robot policies, but they are still inefficient (e.g., requiring long real-world exploration), especially in mobile manipulation.
Humans become efficient by watching others.
Can coding agents learn from passive human videos, just like we do?
Introducing MoCaP: Mobile Code-as-Policies for Zero-Shot Mobile Manipulation from Passive Human Videos
↓🧵
1
3
29
2,961
Excited to share Reward-Rate Policy Gradient: RL that accounts for execution time.
For autoresearch and ML engineering agents, actions can take seconds or hours. A fixed-step MDP misses this distinction.
We optimize MLE agents' reward per unit time in a semi-MDP.
arxiv.org/pdf/2609.36393
muhang-tian.com/reward-rate
See details below:
1/7
Excited to share our new work on agentic RL! 🛠️
Some tasks in agentic RL involve actions that incur costs. One example is machine learning engineering (MLE) agents, where a proposed script might run for minutes or hours. We propose Reward-rate Policy Gradient (RPG) using a Semi-Markov Decision Process formulation that optimizes reward per unit time.
🌐muhang-tian.com/reward-rate/
6
6
116
6,730
I was most excited about the NanoGPT results: reward-rate RL achieves lower pretraining loss faster than vanilla RL.
With growing interest in autoresearch and MLE agents from @karpathy @quocleix @JeffDean, time-aware RL could be an important ingredient. Choosing the next experiment means deciding how to spend a limited research budget. Agents should learn to weigh an experiment’s potential payoff against the time it takes to run.
nitter.siddz.com/muhang_tian/status/210…
1
1
6
575
This work was inspired by the paper "Between MDPs and Semi-MDPs" by @RichardSSutton, Doina Precup, and Satinder Singh from nearly three decades ago.
Applying conceptual frameworks such as semi-MDPs in practical scenarios of modern agentic RL for LLMs is a reminder of how important the foundations of RL remain.
190
Sherry Yang retweeted
Au contraire.
C'est une nouvelle ère qui s'ouvre pour les mathématiques.
Une ère où la démonstration formelle est largement automatisée et où l'accent sera reporté sur le développement de nouveaux concepts, nouvelles abstractions, nouvelles définitions, et nouvelles conjectures.
L'invention du bateau a réduit l'importance de la nage, mais a permis la découverte de nouvelles terres.
145
641
4,498
246,348
Robotics researchers in academia including myself might be having a bit of an existential crisis watching systems like Astra get better so fast.
But academia has an unusual advantage: robots doing real science in physical labs.
That's why I'm excited about WetRobo 🧬🤖, where coding agents observe a wet lab, write + execute robot programs, and adapt through physical trial and error.
Paper: arxiv.org/abs/2609.18435
Code: github.com/tsudalab/WetRobo
6
18
213
16,758
I've been interested in the intersection of robot learning × AI for science for a while.
WetRobo makes me particularly excited about coding agents as an interface between foundation models and scientific instruments.
Moving from agents that reason about experiments to agents that can physically carry them out and adapt along the way is an extremely exciting future direction of robot learning × AI for science.
1
4
308
Amazing work led by @YunaOikawa1 in collaboration with @LerrelPinto and @kojitsuda and many other collaborators from @NYU_Courant and @UTokyo_News_en.
3
392