NVIDIA released Foundation4D. Give it multi-camera driving video and a single forward pass returns depth, camera poses, 3D boxes, semantic occupancy and Gaussians you can render. One shared backbone, a decoder per task.

Where this gets practical is simulation. Using its predicted poses and tracks as priors for lidarless NuRec cut turnaround from 16.0 to 3.7 hours per clip, about 4.3x faster.The renders hold up too: 23.94 dB PSNR, against 22.66 for Instant NuRec given ground-truth poses.

Most video world models try to predict future actions directly in pixel space, which hides internal physics and makes precise control difficult. Adam Abecid and team introduced DeXplicit to fix this by making physical state explicit for dexterous manipulation. The model first forecasts 3D geometry motion for objects and hands, then renders those explicit physical states into future video. It handles joint level hand articulation, keeps track of dynamic object deformations, and runs fast enough with Causal DeXplicit to output four future video frames in 0.54 seconds on a single A800 GPU.

To prevent visual flickering across long sequences, DeXplicit carries appearance features directly with the 3D geometry. Initial image descriptors attach to persistent 3D points and project forward into future viewpoints. Depth ordering handles dynamic occlusions, ensuring that visual details travel with the object even when hands pass over surface textures.

Robots can assemble parts without seeing or feeling anything. “Blind Precise Assembly” from MIT CSAIL shows that joint encoders alone give enough proprioceptive feedback for precise assembly. The tasks include peg insertion, gear meshing, and nut threading. No cameras. No tactile sensors. No object pose knowledge.

The results from Blind Precise Assembly: Near perfect success in simulation Zero-shot sim to real transfer 73% success on peg insertion across diverse 3D printed parts with tight clearances. The surprising part is that blind policies generalized better than matched visual policies on out of distribution parts. The trade off is that they were slower on in-distribution ones.

Generative models are blind to the physics of complete trajectories. They predict the next step but fail to understand the entire path. Researchers from Harvard and Microsoft just fixed this with LEAP. By treating the stochastic action of a system as the primary learning target, this framework turns raw trajectory observations into a fully reusable objective for scientific control.

Once the action is learned, it never requires retraining. The exact same objective can be instantly reused for path ranking, mechanism steering, or constrained optimization. Researchers proved this by learning a molecular trajectory at 600 Kelvin and successfully reusing that exact model to separate and steer paths at 500 Kelvin.

What if we could improve pretrained 3D reconstruction models using unlabeled videos - without adding inference-time overhead? A promising new direction for dynamic 3D reconstruction: PURE PURE explores exactly this, targeting more accurate geometry and better temporal consistency in dynamic scenes.

Approaches like this could be valuable for dynamic scene understanding, immersive 3D environments, and building more realistic digital representations of the world. The balance between reconstruction quality and computational efficiency remains a key challenge.