It’s great to see growing interest in MBRL. Last year in RWM, we demonstrated that policies trained entirely in world-model imagination transfer zero-shot to the real world. Sometimes, ideas are ahead of their time. Glad to see this direction getting the recognition it deserves.

Can feed-forward 3D models improve without 3D labels? 🚀 PURE (ACM MM’26): post-train VGGT, Pi3 & DA3 on unlabeled videos using monocular geometry + frame matches. Better dynamic 3D, no added inference cost. Project: funnyguy525.github.io/PURE-P… Code: github.com/IGL-HKUST/PURE

3D shape decomposition matters!

🌍Meet our World2HOSI: 3D Gaussian scenes + text → long-horizon human–object–scene interactions. Existing motion experts collaborate through a shared, evolving world—without training a new unified motion generator. nintreesssss.github.io/World…

Use 1️⃣3D Gaussian scenes + 2️⃣Text for 3️⃣long-horizon human–object–scene interactions.

Sharing our new work: UniForge3D generates detailed 3D meshes from sparse-view inputs with high fidelity! The code has been released and welcome to try it! Paper: arxiv.org/abs/2610.15357 Project Page: xishuxishu.github.io/uniforg… Code: github.com/xibushu/UniForge3…

✨RigidFormer: Learning Rigid Dynamics using Transformers — accepted at #NeurIPS2026! 💻 Code is now open source: github.com/Frank-ZY-Dou/Dyna… We propose an object-centric Transformer for learning rigid-body dynamics, enabling efficient and scalable mesh-free simulation. It achieves higher accuracy and faster simulation. We hope this architecture can be extended to a wide range of rigid-body dynamics scenarios and potentially to deformable-body dynamics by adapting its rigid-alignment component. Contributions are very welcome! It’s been a pleasure working with this wonderful team: Minghao Guo (@GuoMh14), Haixu Wu (@Haixu_Wu_1998), Doug Roble, Tuur Stuyck (@TuurStuyck), and Wojciech Matusik (@wojmatusik)! See you in Atlanta! #Simulation #PhysicalAI #MachineLearning #Robotics #AI #DeepLearning #Simulation

GaussiAnimate compresses a 4D capture into a rigged 3D asset while keeping its secondary dynamics. No physics simulation or learned deformable model. Page: net-maker.github.io/gaussian… #SIGGRAPHAsia2026 🧵

Tested on humans, animals, and garments, using 4D Gaussian and mesh sequences. Each becomes a rigged 3D asset you can animate with new motions. Paper: arxiv.org/abs/2604.08547 Project: net-maker.github.io/gaussian… GitHub: github.com/Net-Maker/GaussiA… Kudos @Net_Maker @Small_Blue_Wolf @ZeyuCai111168 @frankzydou @_cheng_lin @AnpeiC

Segment anything for your 3D meshes! PartLLM (SIGGRAPH Asia 26, ACM ToG) connects 3D meshes to LLMs for flexible 3D segmentation with point prompts, text prompts, or arbitrary granularities. Codes released! GitHub: github.com/czvvd/PartLLM Paper: arxiv.org/abs/2609.25832

Excited to share that our paper, TrackFish3D: Self-Supervised 3D Tracking of Schooling Fish from Multi-view Videos, has been accepted to #NeurIPS2026 🎉 Tracking individual fish in dense schools is challenging due to frequent occlusions, highly similar appearance, and the lack of identity annotations. In this work, we present TrackFish3D, a geometry-driven self-supervised framework for multi-view 3D tracking. Instead of relying on appearance-based re-identification or manually annotated identities, TrackFish3D uses calibrated multi-view geometry to generate pseudo-associations, then learns cross-view correspondence and temporal identity consistency directly from unlabeled footage. TrackFish3D achieves strong performance on both our synthetic benchmark and the 3D-ZeF zebrafish benchmark, outperforming existing baselines while requiring no identity labels or 3D trajectory annotations for training. I am very grateful to my coauthors @frankzydou, Yifan Wu, Kinfung Chu, Yuan Liu, Lei Yang, and Wenping Wang, and especially to my supervisor, Prof. Taku Komura, for their guidance and support throughout this work. 📄 Paper: lnkd.in/g6SwfYfd 💻 Code: lnkd.in/gqX5uAEf 🌐 Project Page: lnkd.in/gzXwbfv3 🤗 Data & Checkpoints: lnkd.in/g52Bd49t #NeurIPS2026 #ComputerVision #3DTracking #SelfSupervisedLearning

🐟 🐠🦈 Happy to see Patt(@PPhurtivilai )’s TrackFish3D work accepted to #NeurIPS2026. Challenges: Reconstructing identity-consistent 3D trajectories in dense groups of visually similar individuals, such as schooling fish, is a challenging vision problem, further complicated by frequent occlusions and degraded underwater visibility. Significance: Beyond perception, these trajectories provide a powerful empirical tool for studying collective behavior—allowing us to test how local interactions give rise to coordinated group motion and moving us toward a more first-principles understanding of its underlying mechanisms. Project Page: pattgene.github.io/trackfish… Code: github.com/pattgene/TrackFis…

Over the past few weeks, GPT-6 Astra, Claude Code with Opus 5.5, and other recent foundation models have surprised many of us in graphics, CAD, and robotics. They can now work directly with Blender, CAD kernels, game engines, physics simulators, and real robots very well. New examples appear almost daily, faster than the publication cycle can capture. 📝To keep track, we maintain a Living Survey of frontier AI for Design & 3D Modeling & Robotics: 🌐 mit-cdfg.github.io/Survey-AI… Currently, it organizes 243 cases from 345+ public showcases and developer reports across 3D modeling, CAD and industrial design, animation, simulation, and robot control, each linked to its original source. (If you are working on related projects, it's better to check this out: ) Different from a traditional survey, three ideas guide it: 1. Horizon scanning beyond publication lag In an era where frontier foundation model capabilities evolve rapidly, traditional academic publishing cycles could lag behind public community developments and empirical findings. This platform establishes a centralized, high-velocity empirical synthesis repository to provide researchers and engineers with timely visibility into ongoing developments across 3D generation, parametric CAD, and embodied robotics—anchoring these observations in objective evaluations of strategic opportunities and critical safety boundaries. 2. Demonstrations as a distributed record of use We analyze the corpus of over 345 publicly documented showcases and developer reports as an extensive, distributed "crowdsourced user study." This framing captures how models operate when prompted across diverse geometry kernels (CGM, Open CASCADE), DCC software (Blender), physics simulators (Isaac Sim, MuJoCo, Genesis), and physical robot hardware—revealing real-world workflow friction, prompt overhead, and boundary failures that static benchmarks miss. 3. Ranking by open verifiability A central challenge is balancing the timely collection of rapidly emerging results with the need for high-quality, evidence-based evaluation. Our approach is to rank cases by open verifiability—the amount of evidence that others can directly inspect. Equally impressive demos can carry very different evidence, so every case is ranked by what others can inspect: - Rank 1 · Demo + Implementation Code: public code, scripts or CAD/robot harnesses (availability/verifiability) - Rank 2 · Demo + Interactive Web Link: a live web app, 3D viewer or cloud CAD project - Rank 3 · Demonstration Media Only: video or screenshots only Note: Rank 1 means the evidence is inspectable, not necessarily reproducible from a clean setup. Gallery: mit-cdfg.github.io/Survey-AI… Contributions are very welcome via pull requests on GitHub. We especially welcome Rank 1 contributions (demos with code) and Rank 2 contributions (demos with interactive webpages). Many thanks to our collaborators: Jamison Meindl(@meindljamie), Akihisa Watanabe(@Akihisa_Wat ), Anna Deng, Tianyu Huang(@tyanyuy3125), Igor Sadalski(@igorsadalski ), Harrison Liang, Minghao Guo(@GuoMh14 ), Benjamin Tod Jones, Wojciech Matusik(@wojmatusik ).

🤩

One model to control/animate them all: humans, animals, and articulated objects... Check out @LinzhanMou’s UniMate: linzhanmou.com/unimate/ One unified model can animate diverse rigged assets from text prompts, with no per-skeleton retraining. This idea could go beyond kinematic motion generation— a unified representation across different embodiments could also be very useful for robot control and action prediction. Very nice work from @LinzhanMou! Congrats! #SIGGRAPHAsia2026 #Animation #Robotics #RobotLearning #EmbodiedAI #GenerativeAI #3D #MotionGeneration #AI #AIGC

Our work #S4R has been accepted to SIGGRAPH Asia 2026 / ACM Transactions on Graphics (TOG). Removing unintended interpenetration is an important step toward making generated or assembled 3D scenes usable for physical simulation and robotic interaction. S4R tackles this problem through a simple scale-space strategy: we first shrink objects to create clearance, then progressively scale them back to full size while optimizing their poses. This transforms a difficult deep-interpenetration problem into a sequence of mostly shallow-contact quadratic programs (QPs). We are also integrating S4R into an agentic framework for automatically generating and optimizing penetration-free, simulation-ready scene layouts. Project Page: frank-zy-dou.github.io/proje… Code: github.com/Frank-ZY-Dou/Dyna… #Geometry #3D #Graphics #Simulation #Optimization #Robotics

Please check out this comprehensive evaluation of #GPT6 Astra as an embodied agent on #RoboDojo. robodojo-benchmark.com/repor…

We put GPT-6 Astra in the RoboDojo. 🥋🤖 The RoboDojo Team conducted a comprehensive evaluation of GPT-6 Astra as an embodied agent, including: • RoboDojo Sim & Real, compared with GPT-5.5 and DeepSeek-Flash • Humanoid high-level control • Dexterous piano playing with RoboPianist 🎹 • A systematic study of in-context learning (ICL) Our key takeaway: GPT-6 Astra demonstrates remarkably strong semantic and spatial understanding, together with impressive in-context adaptation. At the same time, physical commonsense remains a clear bottleneck — revealing an important gap between understanding the world and truly reasoning about its physics. Full report & demos: robodojo-benchmark.com/repor… @_wenbozhang (project lead), @wenhaocha1, @frankzydou, @JinWeiyang18434, @YutaoOuyang, @minifullcapsule, @x_h_ucb, @YutaoOuyang, @YueChen614

Tested #GPT6 Astra with xhigh reasoning to control my #LeRobot SO101 for a real-world pickup using a third-person RGB camera. The result looks good. Note that I deliberately placed the pen where a straightforward pick-and-place approach wouldn't work: the arm had to bend back to reach it. My prompts: 1. "We have connected a camera (for a third-person view) and a LeRobot SO101 robotic arm for control. Please control the robotic arm to pick up the red pen and hold it. You need to figure out for yourself how to adjust the robotic arm's pose." 2. "Please record a video while operating." 3. "Did you record a video of our entire process? Give me a final MP4." (After I saw the robot complete the task) It automatically reused some of my existing code for motor control, joint coordinate conversion, configuration loading, camera capture, and robot kinematics. Similar code and robot models are available online. It wrote new motion/recording scripts, adjusted its approach from camera feedback, and successfully picked up and held the pen. The obvious limitation is speed: 22 minutes for one pickup. To be fair, slowing down makes manipulation easier, since quasi-static motion removes most of the dynamics. The flip side is that the robot gets to trial-and-error: every attempt comes back through the camera, giving the model a better grip on the 3D scene and the physics than a single shot would. (As many studies have shown, adding expert modules such as depth estimation could further improve efficiency.) Video is at 50× speed.

I kept seeing #GPT-6 Astra modelling results on here and got curious enough to build the thing myself: an agentic pipeline that turns a plain RGB video into 3D assets. The living room for robots @MIT_CSAIL. Input: one handheld phone walkthrough of our lab kitchen (~20s). Nothing else — no depth sensor, no CAD, no asset library. ~1 day (I actually slept overnight), including human-in-the-loop. Live: frank-zy-dou.github.io/kitch… It is still not perfect — thin and shiny things are still weak, a few objects are drafts, the room shell needs another pass. The loop, roughly: - Monocular video → metric scan (ViPE): camera poses + depth. This is the measuring instrument, not the output. - GPT-6 lists what should exist as separate objects, then open-vocabulary detection + tracking gives per-object masks; each object is fused and measured in metres. - For every asset, GPT-6 works in its own sandbox with tools: it writes the object as a program in a small Blender DSL (closed solids, PBR colours, hinges/drawers), builds it, renders it over the original video frames, compares it with the scan points in 3D, and iterates. - A separate GPT-6 session is the verifier — it can render the model on any frame it wants and must justify every complaint with a frame. The modeller never grades its own work. - A completeness pass renders the whole modelled scene from the video's own cameras, puts it next to the real frames, and says what's still missing. What the detector keeps missing (a row of identical cabinets) gets placed geometrically instead. - Out comes MJCF/URDF with joints. Now, adding simulation to this env.