Assistant Professor @UNC @unccs | IDEAL@UNC | Dexterous/Loco-Manipulation | #robotics, #embodiedAI, #3Dvision, #foundationmodels.

Mingyu Ding retweeted
It’s really surprising to me that such a small change in the action exploration space can make such a big difference for dexterous manipulation -- across benchmarks, tasks, hands, and even different learning or optimization algorithms. @harshgupta tested it on so many settings, and it seems to consistently help. Maybe you try it too!
Per-joint Gaussian noise is a terrible way to explore a 20+ DoF hand We introduce EigenDEXplore, which explores along coordinated patterns from human hand motion, while keeping full control of every joint Better learning, from tool use 🔨 to reorientation 🔄 to bimanual 👐
2
10
131
14,035
Mingyu Ding retweeted
🎉 Excited to announce that AVIC has been accepted to #NeurIPS2026! More imagination is not always better for spatial reasoning, the key is knowing when to imagine and how much to imagine. AVIC learns an RL policy that adaptively invokes a video world model and controls how much to imagine, balancing answer accuracy against imagination cost for more accurate and efficient reasoning. We're also grateful for the AC's metareview comment "I strongly recommend accepting this paper as a Spotlight" 🙏 More details below 👇
🚨We updated Adaptive Visual Imagination Control (AVIC) with a new RL extension: AVIC-R. In AVIC, we proposed adaptive visual imagination for spatial reasoning and found that world models are most useful when invoked selectively. In AVIC-R, we take this one step further by training a policy with RL to decide when to imagine and how much to imagine. Using GRPO with QA correctness and imagination cost as rewards, AVIC-R learns to use world models more efficiently and improves spatial reasoning without relying on fixed test-time scaling. Notably, a Qwen2.5-VL-7B policy trained with AVIC-R outperforms GPT-4o / GPT-4 as policy models, showing that RL can teach smaller models to make better adaptive imagination decisions. See more details in Arxiv v2 👉arxiv.org/abs/2602.08236 (original thread below 👇)
1
17
44
3,005
Evaluated across four benchmarks: LIBERO, RoboTwin, RoboCasa, and RoboDojo. 📖 Blog: tianqi-zh.github.io/BenchMen… 🎥 Rollouts: huggingface.co/spaces/benchm…
A robot can follow a reasonable interpretation of an instruction and still fail the benchmark. Meet BenchMend: we use GPT-6-Astra to uncover instruction–verifier gaps in robotics benchmarks, then repair the wording while preserving success checkers and recorded demonstrations. Key takeaways: 📝 Instructions matter. Small edits can substantially improve alignment between task language and benchmark verdicts. 🤖 Even strong agents have limits when performing zero-shot robot control. Tasks requiring precise contact, placement, and coordination remain challenging for agents. 🛠️ An imperfect policy can still help debug benchmarks. Agents that diagnose gaps, propose fixes, and test them could enable self-improving benchmarks and potentially support recursive self-improvement (RSI). Read the blog: tianqi-zh.github.io/BenchMen… Watch the rollouts: huggingface.co/spaces/benchm…
3
22
2,936
Excited to share @AlanZhang0928's BenchMend! Robots may fail a benchmark not because of wrong actions, but a misaligned instruction-verifier. GPT6 Astra can repair this with minimal change, pointing toward benchmarks that uncover their failure modes and iteratively self-improve.
A robot can follow a reasonable interpretation of an instruction and still fail the benchmark. Meet BenchMend: we use GPT-6-Astra to uncover instruction–verifier gaps in robotics benchmarks, then repair the wording while preserving success checkers and recorded demonstrations. Key takeaways: 📝 Instructions matter. Small edits can substantially improve alignment between task language and benchmark verdicts. 🤖 Even strong agents have limits when performing zero-shot robot control. Tasks requiring precise contact, placement, and coordination remain challenging for agents. 🛠️ An imperfect policy can still help debug benchmarks. Agents that diagnose gaps, propose fixes, and test them could enable self-improving benchmarks and potentially support recursive self-improvement (RSI). Read the blog: tianqi-zh.github.io/BenchMen… Watch the rollouts: huggingface.co/spaces/benchm…
8
877
Mingyu Ding retweeted
Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws: • world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours, • this human data scaling law implied a scaling law on never seen robot data, • both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge 🧵
194
718
3,136
1,820,029
Thanks Shuran! Really appreciate your support. Website: handroid.org/ Also excited to see the amazing and inspiring Transformer Transformer for co-design transformer-transformer.gith…
Check out the @dingmyu and the team's amazing website to get the build details on how to build your own Handroid :)
9
1,420
Thanks Haochen! It was a pleasure collaborating with you as well. Your insights were incredibly valuable and look forward to pushing this direction even further together.
This is exactly the kind of "transformer" I imagine. Creatively extending the otherwise limited workspace of a robot arm in a way that just makes sense. A perfect example of a brilliant idea + amazing execution. Enjoyed collaborating with the team a lot!
4
696
🤖 A robot that transforms between humanoid and dexterous hand? Introducing Handroid, a reconfigurable robot with a shared 27-DoF body 🖐️→🚶→🖐️: • The same joints, different roles • Expanded robot task space • Shared sensing and control interfaces handroid.org
6
34
144
33,646
Great work with @RuoguLi0425, Chenyang Ma, @Ske_Vinci, Zhenyu Wei, @YaoYunchao, @HaochenShi74, Karen Liu, @SongShuran 🌐 Project: handroid.org 📄 Paper: arxiv.org/abs/2607.16187 CAD models and the complete BOM are also available on the project website.
2
947
Humanoids shouldn’t have to stop before they act. CoorDex enables humanoids to walk and manipulate simultaneously by coordinating body and dexhand motion priors in latent space. AnyBody enables coordinated whole-body control from sparse keypoints, while adapting to environment.
3
17
110
7,003
Mingyu Ding retweeted
We just released WatchAct, a benchmark for behavior-grounded robot manipulation (covered in the talk below). The robot has to watch a human video, infer what was done, make a plan, and then execute it. The best VLM (Gemini-3.1-Pro) only reaches 36.8% planning success rate, so still plenty of room for improvement. Paper: arxiv.org/abs/2606.26443 Project: baiqi-li.github.io/watchact_… Code: github.com/Baiqi-Li/WatchAct Dataset: huggingface.co/datasets/Baiq… w/ @baiqil0203 @cezhhh @yuffishh @dingmyu
Sharing my CVPR 2026 talk from the Vision for Intelligent Task Assistants workshop: "From Perception to Agency: The Cognitive Stack for Video Task Assistants." It covers our SVI-Bench project (svi-bench.github.io/) plus a video+robotics project we'll release soon. w/ @YuluPan_00 @mmiemon @Han_Yi_724 @mars_su0311 @baiqil0203 gedasbertasius.com/talks
1
7
24
2,246
Mingyu Ding retweeted
What if humanoids could manipulate without stopping, using a high-DoF dexterous hand? 🚨We introduce CoorDex, a learning pipeline for continuous dexterous humanoid loco-manipulation — coordinating whole-body motion and high-DoF finger control while the robot is still moving. This enables a Unitree G1 with a 20-DoF WUJI dexterous hand to: 🤖 grasp and carry a bottle while walking 🚪 open a fridge while stepping backward 🔄 pick up a cube and turn 180° 🔗Project page: skevinci.github.io/coordex/ 📜Paper: arxiv.org/abs/2606.23680 🧵(1/n)
2
9
85
7,034
Mingyu Ding retweeted
🔥 AutoResearchClaw tech report + v0.5.0 just dropped. 12,300+⭐ on GitHub. Two big additions this release: 🧪 1/ Domain-Expert Agents in the experiment stage: Specialized agents for high-energy physics, biology, and more. Real domain tools + knowledge plugged in — not a generic LLM pretending to run experiments. 📊 2/ ARC-Bench A 55-topic benchmark across ML, HEP, quantum physics, biology, and statistics. One of the broadest cross-disciplinary evaluations for autonomous research ever released. 🏆 The numbers: → Beats AI Scientist v2 by 54.7% on ARC-Bench → 7-mode HITL (human-in-the-loop) ablation: targeted intervention > full autonomy OR exhaustive oversight. The thesis (still): real research isn't a pipeline. Hypotheses fail. Lessons compound. AutoResearchClaw is a research amplifier — not a paper generator. 📄 Tech report: arxiv.org/abs/2605.20025 💻 Code: github.com/aiming-lab/AutoRe… Thanks @itsJiaqiLiu and @StephenQS0710 who lead the work and all other contributors @HaonianJi, @lillianwei423, @XinyeYee, @richardxp888, @HaoqinT, @Xinyu2ML, @WeitongZhang, @jiahengzhang96, @LINJIEFUN, @linjunz_stat, @yuyinzhou_cs, @CaimingXiong, @james_y_zou, @ZhengBerkeley, @cihangxie, @dingmyu
Everyone's excited about Karpathy's autoresearch that automates the experiment loop. We automated the whole damn thing. 🦞 Meet AutoResearchClaw: one message in, full conference paper out. Real experiments. Real citations. Real code. No human in the loop. One message in → full paper out. Here's what happens in between: 📚 Raids arXiv & Semantic Scholar, digests 50+ papers in minutes 🥊 Three AI agents FIGHT over the best hypothesis (one swings big, one sanity-checks, one tries to kill every idea) 💻 Writes experiment code from scratch, adapts to your hardware 💥 Code crashes at 3am? It reads the stack trace, rewrites the fix, keeps going 🔄 Results weak? It pivots to entirely new hypotheses and starts over 📝 Drafts a full paper with citations, every single one verified against live databases No babysitting. No Slack messages. No "hey can you re-run this." Karpathy built the experiment loop. We built the whole lab. Chat an idea. Get a paper. 🦞 Try it 👉: github.com/aiming-lab/AutoRe… Kudos to the team @JiaqiLiu835914, @richardxp888, @lillianwei423, @StephenQS0710, @Xinyu2ML, @HaoqinT, @zhengop, @cihangxie, @dingmyu, and we are looking for more contributors.
4
23
67
12,387
Mingyu Ding retweeted
🤖Can robots play Texas Hold’em? 🤔 We introduce DexHoldem, a real-world benchmark for dexterous manipulation and embodied agents. After fine-tuning leading VLAs and pairing them with frontier agents, we found: Robots can move the chips — sometimes. Agents can read the table state — sometimes. Unfortunately, poker requires both to happen correctly in the same hand. So no, robots are not ready to take your money at the casino — yet. More below 👇
2
23
58
13,384
Mingyu Ding retweeted
🚀Introducing DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLAs. By formulating latent visual foresight as a differentiable structural bottleneck, DIAL strictly grounds motor control in the VLM’s intent. This architecture alleviates representation collapse during joint optimization, enabling the model to absorb cross-embodiment human data for robust generalization, while mastering complex multi-stage coordination in the real world. 🚀Paper, code, and project page released! Fully open-sourced! Paper: arxiv.org/pdf/2603.29844 Project page: xpeng-robotics.github.io/dia… Code: github.com/xpeng-robotics/DI… Congrats to @ChenYi041699 and many thanks to our awesome collaborators @tttoaster_ @dingmyu @ge_yixiao
1
9
69
11,227
Mingyu Ding retweeted
🚀 AutoResearchClaw v0.4.0 is here — almost 10K⭐ in just over 2 weeks! Now supporting both fully autonomous AND human-AI co-pilot modes — you choose your level of involvement. What's new: 🤝 6 intervention modes — full-auto, gate-only, checkpoint, step-by-step, co-pilot, and custom. Same powerful 23-stage pipeline, your level of control. 🧪 Idea Workshop — brainstorm and refine hypotheses with AI before committing to a direction 📊 Baseline Navigator — review and customize experiment designs before execution ✍️ Paper Co-Writer — draft papers section-by-section, collaboratively 🧠 SmartPause — the system learns when to pause and ask for your input based on confidence levels 💰 Cost Guardrails — budget alerts at 50/80/100% so you never get surprised 🔀 Pipeline Branching — explore multiple hypotheses in parallel, compare, and merge the best Want full automation? It still does that. Want to stay in the loop? Now you can, at exactly the granularity you want. Try it 👉: github.com/aiming-lab/AutoRe… Kudos to the team @JiaqiLiu835914, @richardxp888, @lillianwei423, @StephenQS0710, @Xinyu2ML, @HaoqinT, @jiahengzhang96, @yuyinzhou_cs, @ZhengBerkeley, @cihangxie, @dingmyu, etc.
2
21
74
9,088
Mingyu Ding retweeted
Do Vision-Language-Action Models truly follow your language instructions? We present When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs. They promise to ground language instructions in robot control, yet in practice, often fail to follow language faithfully. 📄 Paper: arxiv.org/abs/2602.17659 🌐 Project: vla-va.github.io/ 💡 Highlights Vision shortcuts and counterfactual failures. When given instructions that lack strong scene-specific supervision, they default to well-learned scene-specific behaviors regardless of language intent. Counterfactual benchmark. We introduce LIBERO-CF, the first counterfactual benchmark for evaluating language following in VLAs. Our evaluation reveals that counterfactual failures are prevalent yet underexplored across state-of-the-art VLAs. Our solution. We propose Counterfactual Action Guidance (CAG), a simple plug-and-play dual-branch inference scheme that strengthens language conditioning without changing pretrained VLA architectures or weights. Experiments. CAG is effective across multiple dimensions of language grounding, consistently improving both language grounding and task success on under-observed tasks. #VLA #Robotics #Vision #Language
1
26
142
11,315
Can we build a universal brain for all dexterous robot hands? Zhenyu Wei, Yunchao Yao, and Mingyu Ding from University of North Carolina at Chapel Hill just tackled this! By creating a "canonical representation," they translate all kinds of dexterous robot hands into a single, unified description and control language. This allows a single AI policy to understand and control them all. The result: policies that instantly generalize to any new robot hand design, achieving an 81.9% zero-shot success rate on unseen hands and opening the door to universal dexterous manipulation. One Hand to Rule Them All: Canonical Representations for Unified Dexterous Manipulation Project: zhenyuwei2003.github.io/OHRA… Paper: arxiv.org/abs/2602.16712 Code: github.com/zhenyuwei2003/OHR… Our report: mp.weixin.qq.com/s/cp15BVTkx… 📬 #PapersAccepted by Jiqizhixin
3
9
1,462
Found an impersonation account @mingyding pretending to be me. I only have one account. Please do not interact with the fake account and help report it if possible, thanks!
3
2
15
2,096
Introducing OHRA (One Hand to Rule Them All) — a canonical representation that unifies diverse dexterous robot hands into a shared space, enabling cross-hand policy transfer and up to 81.9% zero-shot generalization to unseen morphologies 🌐zhenyuwei2003.github.io/OHRA arxiv 2602.16712
4
16
86
7,583