Sharing my CVPR 2026 talk from the Vision for Intelligent Task Assistants workshop: "From Perception to Agency: The Cognitive Stack for Video Task Assistants." It covers our SVI-Bench project (svi-bench.github.io/) plus a video+robotics project we'll release soon. w/ @YuluPan_00 @mmiemon @Han_Yi_724 @mars_su0311 @baiqil0203 gedasbertasius.com/talks

What to remember in the past to act—spatial, temporal or motion details? We often assume what’s useful, but that’s hard to know in advance. Meet IMPRINT 🧠: retain rich memory first, then learn to use it for control. Compact latents keep inference fast as history grows. 🧵 1/10

I’ll be at #COLM2026 this week presenting EgoMemReason! Our benchmark asks how well multimodal agents can build and use memory to reason across a week of everyday life-tracking object changes, connecting past events, and inferring recurring patterns. 📍 Poster #71, Imperial Ballroom 🗓️ Tuesday, Oct. 6, 4:30–6:30 PM PDT Stop by to chat about the work and what comes next: how can we build persistent memory that helps agents learn from experience over time? Exciting work with Yue Zhang @zhan1624 (co-lead), Shoubin Yu @shoubin621, Ce Zhang @cezhhh, Zengqi Zhao, Jaehong Yoon @jaeh0ng_yoon, Hyunji Lee @hyunji_amy_lee, Gedas Bertasius @gberta227, and Mohit Bansal @mohitban47 (@unc_ai_group @unccs)! 🌐 Website + Benchmark: egomemreason.github.io/ 📄 Paper: arxiv.org/abs/2605.09874 I’m also on the industry job market for opportunities starting summer 2027. Would love to connect!

Check out @abhirammaddukur's work! 🚀 We processed ~523 hours of Ego-Exo4D video 🎥 and release the SMPL-H reconstructions for the camera wearer. We put a lot of effort into curating this data and hope it will be useful to the community! 🔗 Data & Code: abhiram824.github.io/egoexo4…

📣 Our work MolmoMotion was accepted into NeurIPS 2026 as Spotlight 🏆 We showcase that 3D motion *prediction* is a strong yet very learnable prior for robotics and video generation tasks. All artifacts are released, please give it a try! See you in Atlanta 🥳

Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI. Perhaps counterintuitively, I think it's a great time to do so. I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wri…

Toru raises fundamental concerns with which I agree. It is amazing to see the progress in VLMs such as Astra, building on the work of generations of scientists. But Astra doesn't cite what it is building on (I have co-authored a couple of in-hand rotation papers which might have been used in the Astra pen-spinning demo). I fully acknowledge that VLMs make the result of previous research much more accessible to the general public, just like encyclopedias and Google Search did earlier, and that is a good thing. However, in the past, society had mechanisms like patent disclosures and paper citations as ways of doing credit assignment. Should we not accuse these AI models of plagiarism if they don't cite their sources? Somewhat relatedly, I note the open letter by 25 Fields Medalists in response to the Navier Stokes result complaining about the disruption caused to the mathematics research process. Some people interpreted it as turf protection, but the point was more subtle. If certain kinds of creative work stops because incentives are disrupted, it is like farmers eating their seed corn.

We’re teaching a new course at Penn: CIS 6280 · World Models 🌍 cis.upenn.edu/~cis6280/ Every day, LLMs amaze me with one more thing they can solve. But I’ve always felt there’s more to learn than what humans have written down—from observing the world, interacting with it, and experiencing what happens next. To me, that’s what world models are about—and why I see them as one of AI’s next big bets. However, I struggled to find a systematic course on them—so we are building one! Topics will include: representation learning, generative models, simulation, model-based RL, video and 3D generation, world models for robotics, reasoning, and code-based world models. Hands-on work will include: - Building an environment - Training a world model - Learning a policy and a final research project with leaderboard. We’re already six lectures in. Slides, demos, and readings are public, and we’ll keep adding materials throughout the fall. Big thanks to our TA team— @TongMutianTMT @hagsaeng_bag @EnxinSong @KeelyAi04 —for helping bring this course to life!

Video models have become good at recognizing actions but still struggle to infer what people are trying to accomplish, why they chose a particular action, and what they will do next. We built SVI-Bench to measure this gap. We'll present it at #ECCV2026 next week. The video below gives a short overview of the work. I also wrote a short post on the broader research direction: gedasbertasius.com/post/from…

Really excited to see the VideoTree idea evolve from a hierarchical representation into an interactive search space for long video agents! 🌲🔍 Our new work, VideoTreeSearch (VTS) lets agents hierarchically navigate the long video, revisit earlier decisions, backtrack from unpromising paths, and self-correct as they search for the right evidence. This makes long video reasoning more adaptive and leads to strong gains across both grounded and general long-video QA. Great work led by Ce, excited to see where this direction goes! 🚀 Check Ce's thread for more detail 👇

For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. images.nvidia.com/pdf/Open-W…

Excited to share our new work, “Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA.” Grounded LVQA requires a model to answer a question about a long video and localize the short interval supporting its answer. We introduce VideoTreeSearch (VTS), which organizes the video as an adaptive temporal tree and lets an agent search, backtrack, and revise its search path. VTS brings strong gains on three grounded LVQA benchmarks and also transfers to general long-video QA. (1/9)

Our proposed VideoTreeSearch achieves state-of-the-art results across all grounded long-video QA benchmarks while also being more efficient than prior agentic video methods. We also show that we can achieve strong temporal grounding results even if we don't use any manually labeled long video data. Check out Ce's thread below for more details. Work with @cezhhh @YuluPan_00 @ZiyangW00 @mohitban47

Highly performant open weights frontier models such as Kimi are a competitive threat to OpenAI & Anthropic, but probably for everyone else these are a win. Hope more US entities will release top quality open weights models as well. Government regulation of AI models to prevent public harm could be done equally for proprietary or open weight models. Classic arguments in favor of open source software apply to AI models too.

Thrilled to share that EgoMemReason is accepted to #COLM2026 ✨ EgoMemReason is a benchmark for memory-driven reasoning over week-long egocentric videos, designed to test whether existing methods can build and use persistent visual memory: tracking how entities evolve over time, connecting related events separated by hours or days, and inferring recurring behavioral patterns from sparse observations. It contains 500 questions spanning three memory types and six reasoning challenges, with each question requiring evidence from 5.1 video segments and 25.9 hours of memory backtracking on average. Across 17 MLLMs and agentic frameworks, the best method achieves only 39.6% accuracy, revealing substantial limitations in current long-horizon visual memory systems. 👇🧵

Happy to share my position paper that appeared @icmlconf this week! I make a case about the last mile problem of longitudinal local adaptation being a necessary goal of assistive AI - one that hasn't received as much attention, especially in embodied contexts

WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation Baiqi Li, Ce Zhang, Yu Fang, Yue Yang, Shangzhe Li, Mingyu Ding, Gedas Bertasius arxiv.org/abs/2606.26443 [𝚌𝚜.𝚁𝙾 𝚌𝚜.𝙰𝙸 𝚌𝚜.𝙲𝚅]