We're looking forward to the Industry Track at #EMNLP2026! Two keynotes: • Mohit Bansal (UNC Chapel Hill): Agentic Challenges & Industry–academia Collaboration (Sun, 16:30) • Verena Rieser (Google DeepMind): Constitutional Alignment (Mon, 9:30) 2026.emnlp.org/program/indus…

We are hiring at @unccs 🎉 ▶️ Two TT Asst. Professors: 1 in Safeguarding AI in Health, 1 in Neuroengineering. ▶️ Full Professor jointly with Biostats. Part of campus cluster hires (review begins Dec 1). Plus, Teaching Faculty (review begins Nov 1). cs.unc.edu/about/faculty-hir…

🎉 Excited to share our #NeurIPS2026 paper, "Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge"! Latent Safety Signals in LLMs: Can we detect a harmful query before the model generates even one token? We show that the answer lies in the model's dark knowledge, i.e., the first-token probability distribution contains safety signals that transfer across different LLMs. We use this insight to build LADE (Latent Safety Signals for Defense), a model-agnostic defense against jailbreak attacks that requires no access to internal hidden states. 🛡️ Key takeaways: 1️⃣ We discover latent safety signals in the first-token probability distribution: contrasting harmful and benign queries reveals safety-discriminative tokens that transfer consistently across safety-aligned LLMs, despite different architectures, tokenizers, and refusal styles. 2️⃣ We introduce LADE 🔍: a lightweight framework that identifies harmful queries before generation begins by applying a simple kNN classifier to latent safety signals in the first-token probability distribution. 3️⃣ LADE also transfers across models 🔄. Across all reference–target LLM pairs, it achieves 96.6% average harmful-vs-benign classification accuracy. It also performs well against several jailbreak attacks while keeping over-refusal low.

Multimodal LLMs (MLLMs) excel at reasoning, layout understanding, and planning—yet in diffusion-based generation, they are often reduced to simple multimodal encoders. What if MLLMs could reason directly in latent space and guide diffusion generation with fine-grained, spatiotemporal control? 🤔 Introducing MetaCanvas 🎨 A lightweight framework that translates MLLM reasoning into structured spatiotemporal conditions for diffusion models. 🧵 👇

🤔 We rely on gaze to guide our actions, but can current MLLMs truly understand it and infer our intentions? Introducing StreamGaze 👀, the first benchmark that evaluates gaze-guided temporal reasoning (past, present, and future) and proactive understanding in streaming video settings. ➡️ Gaze-Guided Streaming Benchmark: 10 tasks spanning past, present, and proactive reasoning, from gaze-sequence matching to alerting when objects appear within the FOV area. ➡️ Gaze-Guided Streaming Data Construction Pipeline: We align egocentric videos with raw gaze trajectories using fixation extraction, region-specific visual prompting, and scanpath construction to generate spatio-temporally grounded QA pairs. This process is human-verified. ➡️ Comprehensive Evaluation of State-of-the-Art MLLMs: Across all gaze-conditioned streaming tasks, we highlight fundamental limits of current MLLMs. All MLLMs fall far below human performance. Models particularly struggle with temporal continuity, gaze grounding, and proactive prediction.

🎉 @NSF renewed the EngageAI Institute for 5 more years and $20M. UNC CS researchers, led by Mohit Bansal (@mohitban47) will run the institute's core AI research: the world modeling tech behind story-based learning tools for K-12 science classrooms. unc.edu/posts/2026/09/14/ai-… @unc

LLM agents & memory systems operate in continuously updated environments (Git repos, evolving docs). They must process long contexts, recover earlier information, and reason over many updates that create interference between old and new information. How well do they handle this? We introduce MINTEval: ✅ Frequent context changes & interference (avg. 86 updates) ✅ 5 challenging question types, including long-range lookback & reasoning over multiple targets distributed across context ✅ 4 realistic domains: state tracking, multi-turn dialogue, Wikipedia revisions, GitHub commits ✅ Avg. 138.8k tokens per instance (up to 1.8M) ✅ Human verification on generated QAs = 95.6% 📊 Across 7 representative systems, MINTEval remains difficult, showing an avg. acc of 27.9%, and the best system reaches only 33.4%. 🔎 Our analysis shows: • Memory construction failures cause a 41.7% drop • Memory agents are highly sensitive to design choices • Memory systems have a strong bias toward insertion operations (76.8%) over deletion/update

Sparse binary rewards bottleneck LLM RL, motivating the use of privileged information in self-distillation as dense teachers. How can we use and balance multiple types of privileged info: leveraging stable cross-view info, while preserving view-specific info? Current on-policy self-distillation methods often condition the teacher on only one type of privileged view: full solution, partial rationale, answer-only, reference code, feedback, etc. This can be suboptimal: 1️⃣ No single privileged view consistently performs best when used as a teacher. 2️⃣ Views can introduce teacher-specific artifacts from information unavailable to the student. 🧠 Adaptive-View Self-Distillation (AVSD) considers multiple privileged views jointly as a teacher family, balancing cross-view consensus and view-specific signals through a token-level gate to construct better dense learning signals. 🧵👇

🚨We updated Adaptive Visual Imagination Control (AVIC) with a new RL extension: AVIC-R. In AVIC, we proposed adaptive visual imagination for spatial reasoning and found that world models are most useful when invoked selectively. In AVIC-R, we take this one step further by training a policy with RL to decide when to imagine and how much to imagine. Using GRPO with QA correctness and imagination cost as rewards, AVIC-R learns to use world models more efficiently and improves spatial reasoning without relying on fixed test-time scaling. Notably, a Qwen2.5-VL-7B policy trained with AVIC-R outperforms GPT-4o / GPT-4 as policy models, showing that RL can teach smaller models to make better adaptive imagination decisions. See more details in Arxiv v2 👉arxiv.org/abs/2602.08236 (original thread below 👇)

🚨 Excited to introduce PhyMotion🤸: Structured 3D Motion Reward for Physics-Grounded Human Video Generation! ❌ Existing 2D video rewards misleadingly assign high scores to videos with floating feet, self-penetrating limbs, and physics-violating motions. ✅ PhyMotion lifts generated videos into 3D, grounds them in a physics simulator, and scores motion along kinematic / contact / dynamic feasibility. ➡️ RL post-training with PhyMotion improves 1.3B model to match 14B models performance in human prefence. 🧵(1/n)👇