MS in Machine Learning at CMU | Researching Machine Learning Reasoning in Multiple Modalities
Pittsburgh, PA
- Tweets 102
- Following 143
- Followers 248
- Likes 158
Pinned Tweet
🎓I’m excited to share that I recently graduated with my Master’s in Machine Learning from Carnegie Mellon University’s School of Computer Science (@CarnegieMellon, @SCSatCMU).
This fall, I will be joining MIT (@MIT) as a PhD student in Electrical Engineering and Computer Science, advised by Paul Pu Liang (@pliang279) and Antonio Torralba, as part of the MIT Media Lab (@medialab) and CSAIL (@MIT_CSAIL).
I’ve been incredibly fortunate to learn from and work with amazing advisors throughout the years. I want to thank Deva Ramanan (@deva_ramanan) for his guidance and support during my time at CMU as well as Trevor Darrell (@TrevorDarrell) and Narges Norouzi for shaping my growth during my undergraduate years at UC Berkeley (@UCBerkeley).
No academic journey is complete without the vital support of mentors who challenge, encourage, and guide us along the way. I’m especially grateful to Roei Herzig (@roeiherzig), Zhiqiu Lin (@ZhiqiuLin), Rudy Corona (@_rodolfocorona_), Abrar Anwar (@_abraranwar), Jesse Thomason (@_jessethomason_), and Jean Oh (@jeanoh) for mentorship, collaboration, and for influencing the way I think about research. Finally, thank you to my family for their constant love and encouragement. A special thank you to my mom, whose unwavering support has been with me through every step of my academic journey.
I feel very lucky for the opportunities and people that brought me here. I'm excited for the next chapter and to build on my past work in sample-efficient adaptation, multimodal AI, robotics, and mechanistic interpretability while continuing to learn from the incredible communities at MIT.
Looking forward to all the things left to learn!
12
4
81
10,261
Chancharik Mitra retweeted
(1/8) 💡 To make better discoveries, it can be enough to train only the idea generator.
Guidance-TTT trains a small 8B model at test time to propose better ideas, while a frozen model turns them into code. The resulting programs are then evaluated to provide the reward for updating the small model. 🔄
Using only open-weight models and no web access, we outperform the best reported results on 4 discovery tasks. 🧵
Paper and code are out:
📄 Paper: arxiv.org/abs/2610.06269
💻 Code: github.com/Human-Agent-Socie…
1
29
124
10,084
Chancharik Mitra retweeted
Concept bottleneck models promise interpretable vision: predict named attributes first, classify from them. Where do the attributes come from? Usually an LLM prompted with the class name. No image is ever consulted. That turns out to be the weak link 🧵
ALT For "strawberry" on ImageNet-Sketch, an LLM lists red and ripe because it never sees an image. Scoring attributes against the sketch images returns what actually holds: colorless, hand-drawn.
1
5
11
542
Chancharik Mitra retweeted
🚨 Call for participation: the Child–Robot Dexterity Challenge at the CoRL 2026 𝗦𝗣𝗜𝗡 𝗪𝗼𝗿𝗸𝘀𝗵𝗼𝗽! Bring your robot, or run on ours. Apply by Oct 5.
🧒🤖 Robots and a child will take on the same everyday tasks, side by side, live on stage. The demonstration offers a simple way to see what today’s robots can already do and where there’s still room to improve.
We make it easy to join:
- Bimanual YAM setup (MolmoAct 2 style) + a dexterous Sharpa hand setup, provided on stage.
- Bring your own hardware (subject to approval).
🌐 Website: spin-workshop.github.io/#con…
🗣️ Speakers & panel from Stanford, Columbia, TU Darmstadt, and AMI Labs.
📍 Nov 12 · Austin, TX
#CoRL2026
2
12
36
5,915
Want to see how far a robot policy can go on just a handful of demos?
At #ECCV2026 in Malmö today I'm presenting Robotic Steering: adapt a VLA by tuning only the attention heads that matter. Same performance, fewer parameters, and it holds up on a real robot!
🎤 Spotlight Oral, 9:10, Palissad (Malmö Arena)
🖼️ Poster #344, 10:30–12:30, Exhall
Project Page: chancharikmitra.github.io/ro…
1
8
295
Chancharik Mitra retweeted
[1/6] Can one model fix rendering artifacts from any 3D representation? And what does it even mean to "fix" a rendering?
At #ECCV2026, we present FixAnything (fix-anything.github.io): a single generalist video model that refines 3DGS/mesh/sparse point-cloud renderings into photorealistic, 3D-consistent videos.
7
45
263
16,776
🎓I’m excited to share that I recently graduated with my Master’s in Machine Learning from Carnegie Mellon University’s School of Computer Science (@CarnegieMellon, @SCSatCMU).
This fall, I will be joining MIT (@MIT) as a PhD student in Electrical Engineering and Computer Science, advised by Paul Pu Liang (@pliang279) and Antonio Torralba, as part of the MIT Media Lab (@medialab) and CSAIL (@MIT_CSAIL).
I’ve been incredibly fortunate to learn from and work with amazing advisors throughout the years. I want to thank Deva Ramanan (@deva_ramanan) for his guidance and support during my time at CMU as well as Trevor Darrell (@TrevorDarrell) and Narges Norouzi for shaping my growth during my undergraduate years at UC Berkeley (@UCBerkeley).
No academic journey is complete without the vital support of mentors who challenge, encourage, and guide us along the way. I’m especially grateful to Roei Herzig (@roeiherzig), Zhiqiu Lin (@ZhiqiuLin), Rudy Corona (@_rodolfocorona_), Abrar Anwar (@_abraranwar), Jesse Thomason (@_jessethomason_), and Jean Oh (@jeanoh) for mentorship, collaboration, and for influencing the way I think about research. Finally, thank you to my family for their constant love and encouragement. A special thank you to my mom, whose unwavering support has been with me through every step of my academic journey.
I feel very lucky for the opportunities and people that brought me here. I'm excited for the next chapter and to build on my past work in sample-efficient adaptation, multimodal AI, robotics, and mechanistic interpretability while continuing to learn from the incredible communities at MIT.
Looking forward to all the things left to learn!
12
4
81
10,261
For anyone looking for an end-to-end, multi-platform, and easy-to-use framework for building and deploying robot systems, RIO from CMU looks really exciting.
RIO, a new framework from CMU, provides a unified interface for robot control, data collection, teleoperation and AI deployment!
It gives researchers a lightweight, modular foundation for deploying robots quickly on any platform.
ri.cmu.edu/robot-io/
2
295
Chancharik Mitra retweeted
This work would not have been possible without an incredible roster of co-authors and collaborators.
First and foremost, a massive thank you to the leads who spearheaded this research and turned a massive two-year vision into reality: 🔸Yuvan Sharma, 🔸Dantong Niu @Dantong_Niu, and 🔸Anirudh Pai @apai253.
Building robust robotics takes a village. A huge shoutout to the dedicated team who spent countless hours collecting data and pushing through rigorous evaluations. Flawless execution from: 🔸Zekai Wang, 🔸Zhuoyang Liu @liu_zhuoyang13, 🔸Baifeng Shi @baifeng_shi, 🔸Stefano Saravalle, 🔸Boning Shao, 🔸Ruijie Zheng, 🔸Jing Wang, 🔸Konstantinos Kallidromitis, 🔸Yusuke Kato, and 🔸Fabio Galasso!
Finally, special thanks for the invaluable mentorship, support, and help shaping the narrative: 🔸Yuke Zhu @yukez, 🔸Danfei Xu @danfei_xu, 🔸Linxi Fan @DrJimFan, 🔸Trevor Darrell @trevordarrell, and 🔸Jitendra Malik @JitendraMalikCV.
A truly monumental team effort from BAIR @berkeley_ai and NVIDIA @nvidia! 🚀
1
721
Incredibly grateful to have been a part of this project with @ZhiqiuLin! VQAScore is one of the most generalizable and lightweight frameworks for measuring all types of text-image or text-video alignment. Highly recommend everyone take a look with the new updated models in play!
Two years ago we released VQAScore with a simple idea: ask a VLM "does this image show {prompt}?" and use P(Yes) as the score.
It has since become the go-to evaluation metric and reward model for visual generation, replacing CLIPScore across the field. Adopted by Google DeepMind (Imagen 3 & 4), NVIDIA, ByteDance, and other frontier labs. Our open-source model has 2M+ downloads on HuggingFace.
Today: a major upgrade. VQAScore now supports text-to-video evaluation using 20+ SOTA VLMs including GPT, Gemini, and Qwen, capturing generation accuracy for prompts like "a shallow depth of field shot rack focusing from a foreground crumpet to a dog entering the background, catching it at the focus shift."
As VLMs get stronger, VQAScore gets stronger. For free.
📄 Paper: arxiv.org/abs/2404.01291
💻 Code: github.com/linzhiqiu/t2v_met…
Thanks to @chancharikm for driving this upgrade, and @gneubig and @RamananDeva.
4
200
So excited to present CHAI, a long-term project that tackles video understanding in the right way 🎥
If your research or interests even loosely connect to video or multimodal reasoning, please check it out. Huge thanks to my co-authors and advisors!
Before AI can generate professional videos, it needs to see like a professional.
We spent a year with 100+ content creators teaching AI to describe video like a filmmaker would.
Introducing CHAI: Critique-based Human-AI Oversight for Building a Precise Video Language [CVPR'26 Highlight, Top 3%].
Try prompting a video generator for a dolly zoom, dutch angle, point of view, or camera roll. Most fall back to the same bland defaults: a push-in, a level shot, a third-person view. Why? These techniques require a language of cinema that current models rarely speak.
We built that language:
1️⃣ Precise specification: 5-aspect structured captions co-designed with professional cinematographers covering subject, scene, motion, spatial, and camera dynamics
2️⃣ Scalable oversight: LLMs draft captions, humans critique what's wrong and how to fix it
3️⃣ Post-training recipes: Qwen3-VL-8B surpasses Gemini-3.1 and GPT-5
4️⃣ Video generation: fine-tuned Wan follows 400-word cinematic prompts with precise control
Here's how each works 🧵
Work led by CMU and Harvard with @chancharikm, @du_yilun, and @RamananDeva.
📄 Paper: huggingface.co/papers/2604.2…
🌐 Site: linzhiqiu.github.io/papers/c…
1
3
10
1,272
At a high level, CHAI is about building a compositional language for video: not just modeling subject motion, but also subtle, high-impact factors like camera motion and shot composition that are often ignored.
Explicitly modeling these leads to more faithful understanding and better control for generation.
Excited about implications for video understanding, text-to-video, robotics, and 4D reasoning!
Full details (and great examples) in the main thread + paper reposted above!
2
370
Chancharik Mitra retweeted
[1/7] Video diffusion has come a long way, generating more & more realistic videos.
Can we revisit sparse-view novel view synthesis through these video priors?
Meet FrameCrafter: a permutation-invariant multi-view model built on video diffusion 🧵
🌐 frame-crafter.github.io
2
33
156
11,860
They say a picture is worth a thousand words, precisely why the usefulness of visual representations will depend on the semantic context.
Check out this exciting work on steering visual representations with language!
Pretrained ViTs like DINOv2 or CLIP are great, but they produce fixed, generic representations that encode the most salient visual concepts (e.g., "cat").
In human vision, prior priming with language changes how people parse an image. We believe visual encoders should do the same
🚨 Introducing Steerable Visual Representations, a new family of visual features you can steer with text towards specific visual concepts.
1
3
323
Chancharik Mitra retweeted
Most of today's AI can see the world, but it doesn’t **feel** it.
Capturing the sense of touch is crucial for dexterous robotic manipulation, user modeling, and understanding physical interactions.
Introducing OpenTouch: bringing full-hand tactile sensing into real-world AI🖐️
OpenTouch is collected in-the-wild using tactile sensing gloves, hand pose tracking gloves, and egocentric glasses. It includes:
• 5 hours of real-world data,
• 3 hours densely annotated contact-rich interactions,
• 2,900 curated interaction clips,
• across 800 objects, 14 environments, and 29 grasp types.
all open at: opentouch-tactile.github.io/
10
35
220
49,361
Robotic reward modeling needs to consider failure cases and represent partial progress. This work takes an interesting approach to dealing with these challenges, among other valuable contributions. Definitely worth a read!
A reward model that works, zero-shot, across robots, tasks, and scenes?
Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories.
Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more!
🧵 (1/12)
3
17
1,554
Chancharik Mitra retweeted
Current Vision-Language Models completely struggle with complex 4D dynamics. We fixed that. 🤯
🚨 Introducing 4D-RGPT: distilling perceptual knowledge directly into LLMs for precise space & time reasoning.
🎉 Excited to share our @NVIDIAAI work has been accepted to #CVPR2026! @CVPR
A quick dive into how it works 🧵👇
2
16
81
14,085
Chancharik Mitra retweeted
Introduce CRISP, a real-to-sim pipeline that recovers human motion and simulatable scene geometry from monocular video!
CRISP builds contact-faithful 3D scene for simulation - 8× fewer sim failures, +43% faster sim, and improves human motion!
Interactive demos👉: crisp-real2sim.github.io/CRI…
Exciting collaboration w/ @JiashunWang @jefftan969 @_Tsukasane @ Jessica Hodgins @shubhtuls @RamananDeva
6
64
350
50,690