Department of Computer Science - University of North Carolina at Chapel Hill Choose to #GIVE today - learn more here: linktr.ee/unccompsci
Chapel Hill, NC
- Tweets 2,761
- Following 431
- Followers 3,310
- Likes 4,114
Pinned Tweet
We are hiring at @unccs 🎉
▶️ Two TT Asst. Professors: 1 in Safeguarding AI in Health, 1 in Neuroengineering.
▶️ Full Professor jointly with Biostats.
Part of campus cluster hires (review begins Dec 1).
Plus, Teaching Faculty (review begins Nov 1).
cs.unc.edu/about/faculty-hir…
4
9
4,526
UNC Computer Science retweeted
Congrats to Yue on the Rising Star Award! 🚨 She is on the faculty job market this year so if you are looking for an awesome faculty member in the important and timely topics of spatial intelligence, embodied AI, world models, and learning from failure (with a focus on building more capable and reliable embodied agents), make sure to hire her! 🙂
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
1
9
34
1,266
🚀 I'm on the 2026–27 faculty job market and would love to connect!
🌟 I recently had the pleasure of speaking as a Rising Star at the 2026 Michigan AI Symposium, sharing my research on spatial intelligence: how embodied agents understand their surroundings, imagine beyond observations with world models, and learn from failures to improve.
It was wonderful to meet and exchange ideas with so many inspiring researchers!
Huge thanks to @mohitban47 for his continuous support, to my amazing collaborators and labmates @unc_ai_group, and to the organizers @michigan_AI for this wonderful opportunity!
4
25
80
11,517
UNC Computer Science retweeted
🤔 Rewarding correct answers doesn’t guarantee the model relies on the CoT, and LLM reasoning can often be post-hoc rationalization. Can we improve both accuracy and CoT faithfulness by optimizing CoT causality?
💡 Our key idea: If reasoning causally contributes to the final answer, changing the order of its steps should affect the probability of that answer. We propose CRoS (Causality Reward on Scrambled CoTs).
🔥 Outcome rewards improve accuracy at the cost of faithfulness, as models may learn shortcuts rather than rely on their CoTs. CRoS mitigates this trade-off, improving accuracy, reasoning quality, and faithfulness together!
⭐️ On Gemma-3-4B, CRoS improves CoT faithfulness (+13.8%), out-of-domain accuracy (+3.0%), and CoT quality (+2.0%) over the strongest outcome and causality reward baselines trained with GRPO.
7
22
59
5,973
UNC Computer Science retweeted
🎉 Congrats on the UNC Postdoctoral Award for Research Excellence! 🚨 Definitely hire Hyunji (Amy) if you are looking for an amazing faculty member & researcher in the important timely topics of long-horizon memory, multi-agent systems, trustworthy and safe AI, information retrieval, or continual learning and knowledge adaptation across text and multimodal data!
Honored to receive the 2026 UNC Postdoctoral Award for Research Excellence (PARE) recognizing research, mentoring, service, and leadership! 🎉
Also: I’m on the job market this year! Please feel free to contact me if you have relevant positions 👀
It was great to share my work on "adaptive AI systems under evolving knowledge (across modalities)" during the UNC Research Week.
Huge thanks to @mohitban47 for the constant support, and to amazing labmates at @unc_ai_group!
Website - amy-hyunji.github.io/
2
12
45
3,173
UNC Computer Science retweeted
Honored to receive the 2026 UNC Postdoctoral Award for Research Excellence (PARE) recognizing research, mentoring, service, and leadership! 🎉
Also: I’m on the job market this year! Please feel free to contact me if you have relevant positions 👀
It was great to share my work on "adaptive AI systems under evolving knowledge (across modalities)" during the UNC Research Week.
Huge thanks to @mohitban47 for the constant support, and to amazing labmates at @unc_ai_group!
Website - amy-hyunji.github.io/
3
23
86
6,234
🚨 Excited to share our new paper “Representing the Unseen: Rethinking Visual Imagination for Spatial Reasoning”!
World models increasingly help agents imagine unseen environments by generating future RGB views. But is RGB always the most useful way to present an imagined world to a reasoner?
Not always. Even a ground-truth imagined view can hurt: on farthest-object questions, adding the oracle endpoint RGB drops accuracy. Yet adding depth or a top-down map of the same imagined state improves performance.
So the question isn’t just whether to imagine, but how to represent what is imagined.
We introduce：
🔹 IMAGIREP: A controlled benchmark that fixes the imagined state and question while varying the representation (RGB, depth, normals, panorama, top-down, text, video). Different questions benefit from different representations, and the benefit persists even when the state is estimated by a world model.
🔹 GEARS: A lightweight test-time strategy that selects when to trust imagined evidence and which representation to use.
➡️ Spatial imagination is more than generating views. World models should produce representations reasoners can actually use.
🧵👇
4
28
100
5,959
UNC Computer Science retweeted
🤔 How can we efficiently train terminal agents to avoid misbehaviors?
😮 We try error-targeted learning on terminal agents and find that, with < 200 envs in TMax-15K & 35% of TMax's rollout, we can match TMax-4B full recipe.
🚨Introducing MiRL (Misbehavior-driven RL): MiRL first profiles the errors of a policy to find mistakes that recur across many environments, which we call generalizable misbehaviors. It then trains the policy with RL, starting rollouts at the steps where these misbehaviors occur and rewards a rollout only if it passes the tests and avoids the misbehavior.
Key takeaways
🔍 On Qwen3.5-4B, we find that 8 dominant misbehaviors account for 52.1% of wrong steps and appear in 75.4% of the environments.
🔥 MiRL outperforms TMax-4B, the released checkpoint trained with the full TMax recipe on Terminal-Bench Lite (51.8% vs. 47.4%) and matches it on Terminal-Bench 2.1 (21.3% vs. 20.6%), while using 35% of its rollouts and 189 selected training envs.
🧵👇 [1/9]
3
18
39
4,708
UNC Computer Science retweeted
👋 Looking forward to attending #EMNLP2026 where I will be giving an industry keynote on the hard, imminent AI challenges of trust, world discovery, and memory, plus an important, timely discussion on re-strengthening industry-academia collaborations!
We're also hiring at all levels: phd, postdoc, faculty! Feel free to ping if you want to meet up to discuss research, hiring, life 🙂
We're looking forward to the Industry Track at #EMNLP2026! Two keynotes:
• Mohit Bansal (UNC Chapel Hill): Agentic Challenges & Industry–academia Collaboration (Sun, 16:30)
• Verena Rieser (Google DeepMind): Constitutional Alignment (Mon, 9:30)
2026.emnlp.org/program/indus…
5
21
89
3,997
UNC Computer Science retweeted
It was a pleasure visiting ETH Zürich for their CS Distinguished Colloquium Series -- amazing scenic views from the hilltop hotel + everywhere 🙂 -- also had various engaging discussions about AI safety/trust and its future with the faculty, students/postdocs, and scientists there -- thanks again for the kind invitation + hospitality Mrinmaya Sachan, Thomas Hofmann, Marc Pollefeys et al. (and for these nice photos incl. from our dinner at the world's oldest "certified" continuously running vegetarian restaurant 😯)
Also had a fun short visit at Google Zurich thanks to Srini Narayanan, Ivan Vulić, Jonas Pfeiffer, Emanuele Bugliarello!
--> Now off to TU Darmstadt for the 'The Future of Science' pre-EMNLP international symposium --> And after that, you can find me at EMNLP in Budapest where I will be delivering an Industry Keynote on Oct25 afternoon. Looking forward to seeing everyone!
9
13
79
4,032
UNC Computer Science retweeted
🚨 A seemingly safe AI agent can become dangerous after benign fine-tuning due to dormant poisoning.
We show that dormant poisoning, where models are poisoned before release such that the hidden harmful behaviors are dormant until activated by benign downstream fine-tuning, can extend beyond text generation to unsafe actions in language and multimodal agents.
We introduce LookAhead Defense 🛡️ to prevent dormant poisoning from activating harmful behaviors in language and multimodal agents.
Key takeaways:
1️⃣ Dormant poisoning in AI agents: We show that benign fine-tuning can activate harmful actions, such as leaking private information via email tools.
2️⃣ Self-generated Safety Buffer: We introduce a defense that uses only the model's self-generated responses to prevent dormant poisoning
3️⃣ LookAhead Defense: Selectively penalizes safety-degrading updates, reducing poison activation by 90% (93.9% → 3.5%) on Qwen3-4B across InjecAgent and ASB, while maintaining a favorable utility–refusal trade-off.
2
23
42
5,174
🚨 Excited to share WEDGE: From Failures to Learnable Worlds, our new work on policy-conditioned environment generation for navigation agents!
We found that even on the same navigation task, agents tend to fail under different environmental conditions. Yet most environment generators do not consider what each agent actually needs to improve.
🤔 This led us to a question: Can we use an agent's own failures to guide the generation of training environments?
➡️ We introduce WEDGE, a framework where an agent's failures shape the worlds it trains in next. WEDGE identifies the conditions under which each agent struggles and generates new environments that target those weaknesses while remaining learnable. After each round of adaptation, it re-diagnoses the agent to guide the next round of environment generation.
Thread🧵👇
4
21
58
5,782
UNC Computer Science retweeted
🚨 We are hiring Tenure-Track and Full Professors in important and timely areas such as AI Safety**, Neuroengineering, as part of exciting campus-level strategic cluster hires with CS, and with several positions in the clusters. The Full Professor position in 'Safeguarding AI for Health' is jointly between our strong BioStats department and CS -- please spread the word and feel free to apply & reach out for any questions!
**broad emphasis incl. but not limited to machine learning theory, AI systems, interpretability, robustness, verification, human-AI interaction.
We are hiring at @unccs 🎉
▶️ Two TT Asst. Professors: 1 in Safeguarding AI in Health, 1 in Neuroengineering.
▶️ Full Professor jointly with Biostats.
Part of campus cluster hires (review begins Dec 1).
Plus, Teaching Faculty (review begins Nov 1).
cs.unc.edu/about/faculty-hir…
2
16
47
3,502
UNC Computer Science retweeted
Can agents learn useful skills from frozen logs that people and agents already leave behind?
🚨Introducing TeleTune: We turn noisy offline telemetry into agent skills w/o access to a live environment during training!
+6.7% on WorkArena & +7.7% on Mind2Web 👉 up to 75x fewer tokens used for optimization. 🧵
Activity logs like user telemetry capture how people use software, but often (1) lack explicit task goals, (2) cannot be replayed to test each skill update, and (3) can be a noisy mix of multiple tasks.
We address these jointly in TeleTune, where we propose skill-guided progress, an offline metric that uses action-prediction errors to improve agent skills. Not only does optimizing it improve downstream performance, but it also saves token usage by 5 to 75x compared to validating skill updates in live environments!
Key takeaways:
1️⃣ TeleTune achieves 77.1% average success on WorkArena and 80.6% on Online-Mind2Web, outperforming the strongest baselines by +6.7% and +7.7% absolute
2️⃣ Optimizing offline skill-guided progress transfers to improvements in downstream task success
3️⃣ Offline optimization uses 5 to 75x fewer tokens than validating the skill edits in a live environment
2
25
89
4,169
UNC Computer Science retweeted
Excited to share our #NeurIPS2026 paper "Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge"! 🎉
We show that LLMs contain latent safety signals in their first-token probability distributions and these signals can transfer across different models.
🎉 Excited to share our #NeurIPS2026 paper, "Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge"!
Latent Safety Signals in LLMs: Can we detect a harmful query before the model generates even one token? We show that the answer lies in the model's dark knowledge, i.e., the first-token probability distribution contains safety signals that transfer across different LLMs.
We use this insight to build LADE (Latent Safety Signals for Defense), a model-agnostic defense against jailbreak attacks that requires no access to internal hidden states. 🛡️
Key takeaways:
1️⃣ We discover latent safety signals in the first-token probability distribution: contrasting harmful and benign queries reveals safety-discriminative tokens that transfer consistently across safety-aligned LLMs, despite different architectures, tokenizers, and refusal styles.
2️⃣ We introduce LADE 🔍: a lightweight framework that identifies harmful queries before generation begins by applying a simple kNN classifier to latent safety signals in the first-token probability distribution.
3️⃣ LADE also transfers across models 🔄. Across all reference–target LLM pairs, it achieves 96.6% average harmful-vs-benign classification accuracy. It also performs well against several jailbreak attacks while keeping over-refusal low.
15
32
1,862
UNC Computer Science retweeted
Recurrent–attention hybrid LMs (e.g., Qwen3.5) have two complementary memory pathways: attention and recurrence. But do they use both effectively?
❌ We find that they rely heavily on attention, even after SFT.
💡 We show that encouraging recurrent memory use improves performance (esp. on long-context tasks)!
🌐 Strong generalization across:
- Multiple hybrid LMs
- Question answering (avg. 4.6% ⬆️) and agentic tasks (avg. 12.1% ⬆️)
- Attention-only models with multiple memory types (28.6% ⬆️)
3
26
73
5,783
UNC Computer Science retweeted
Glad to announce that MetaCanvas has been accepted to #NeurIPS2026! ✨
MetaCanvas lets MLLMs reason and plan directly in spatial and spatiotemporal latent spaces, instead of reducing them to global text encoders for diffusion models. A set of learnable "canvas tokens" carries the MLLM's planning into the diffusion latent space through a lightweight connector, and visualizations show these canvas tokens act as reasonable visual planning sketches guiding the final synthesis.
Implemented on three diffusion backbones and evaluated across six tasks: text-to-image, text/image-to-video, image/video editing, and in-context video generation, each requiring precise layouts, robust attribute binding, and reasoning-intensive control. MetaCanvas consistently beats global-conditioning baselines, narrowing the gap between multimodal understanding and generation.
See you in Atlanta! 🚀
Multimodal LLMs (MLLMs) excel at reasoning, layout understanding, and planning—yet in diffusion-based generation, they are often reduced to simple multimodal encoders.
What if MLLMs could reason directly in latent space and guide diffusion generation with fine-grained, spatiotemporal control? 🤔
Introducing MetaCanvas 🎨
A lightweight framework that translates MLLM reasoning into structured spatiotemporal conditions for diffusion models.
🧵 👇
1
12
25
918
UNC Computer Science retweeted
It was a pleasure to give a keynote at the 36th IEEE International Workshop on Machine Learning for Signal Processing (#MLSP2026), in Atlanta! 🙂
In my talk, “AI Challenges: Trustworthy Collaboration, Skill+World Discovery, and Long-Horizon Memory,” I discussed important and timely AI challenges in trustworthy+safe collaboration (incl. some latest adversarial scenarios), world discovery, and long-term memory planning, across diverse domains & modalities & real-world applications (e.g., early medical diagnosis and classroom education engagement)!
Thanks again to the conference organizers (Sergey Plis, Vince Calhoun, Ran Xiao, Sundaresh Ram, Carl Yang, Hyeokhyen Kwon, Ph.D., Bradley T. Baker, TReNDS Center) for the kind invitation and great organization (incl. the aquarium gala dinner with the whales+sharks, and the cool soundwave clip gift idea perfectly fitting a signal processing conference 🙂), with a very engaging audience/attendees! 🙏
1
15
54
2,225
UNC Computer Science retweeted
Excited to share that StreamGaze 👓, developed in collaboration with Adobe Research, has been accepted to #NeurIPS2026 E&D track (with score 5/5/4)! 🎉
We introduce the first benchmark for evaluating how well MLLMs can use human gaze signals 👀 for temporal reasoning in streaming egocentric videos.
More details below 👇
🤔 We rely on gaze to guide our actions, but can current MLLMs truly understand it and infer our intentions?
Introducing StreamGaze 👀, the first benchmark that evaluates gaze-guided temporal reasoning (past, present, and future) and proactive understanding in streaming video settings.
➡️ Gaze-Guided Streaming Benchmark: 10 tasks spanning past, present, and proactive reasoning, from gaze-sequence matching to alerting when objects appear within the FOV area.
➡️ Gaze-Guided Streaming Data Construction Pipeline: We align egocentric videos with raw gaze trajectories using fixation extraction, region-specific visual prompting, and scanpath construction to generate spatio-temporally grounded QA pairs. This process is human-verified.
➡️ Comprehensive Evaluation of State-of-the-Art MLLMs: Across all gaze-conditioned streaming tasks, we highlight fundamental limits of current MLLMs. All MLLMs fall far below human performance. Models particularly struggle with temporal continuity, gaze grounding, and proactive prediction.
14
51
2,601
UNC Computer Science retweeted
🎉 Excited that our NSF AI Institute for Engaged Learning has been renewed for another 5 years 🙏
Looking forward to leading the core AI efforts on 3D world models for multimodal narrative environments and AI that adapts to student needs -- and collaborating with our wonderful partners (top experts in education and learning sciences) to produce transformative advances in our understanding of how students learn and fundamentally improve the learning outcomes for K-12 students!
🚨 We'll be hiring postdocs+ phd students etc. for the next 5 years for this, so apply+ping with any questions!
🎉 @NSF renewed the EngageAI Institute for 5 more years and $20M. UNC CS researchers, led by Mohit Bansal (@mohitban47) will run the institute's core AI research: the world modeling tech behind story-based learning tools for K-12 science classrooms. unc.edu/posts/2026/09/14/ai-…
@unc
5
21
108
8,023