The most important technology in history is outpacing our ability to understand it. We’re funding exceptional talent to work on it.

🚨New paper (possibly the most important one I have written yet). We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios. Led by @KamacheeMax.

It is important to re-state that the threats are not hypothetical, and that we must take action to mitigate the risks of AI-enabled terrorism.

New paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciting latent knowledge in a code vulnerability scanning task. See below how we made the sandbagging model organism robust to both prompting and training! ( 1/14 )

Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!

> The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research. I'm glad MATS is experimenting with new ways to support promising AI Safety research!

Preprint from my work at @MATSprogram is now up: arxiv.org/abs/2610.07476 Looking at technical governance / verification Qs, specifically if power draw can be used to constrain covert compute.

The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research. → Up to $285K/yr → Berkeley, London, or DC → Research compute, relocation & visa support Apply by Oct 31 AoE ↓ matsprogram.org/residency

With support from @MATSprogram and @BlueDotImpact I’m running a mixer event in London for quants who are concerned about AI risks in the evening of 15th Nov. Come meet researchers working in the field (including @NeelNanda5 , MTSs from Anthropic, Apollo Research, and more)! Travel from Europe covered. Apply here by 1st Nov (takes less than 10 minutes) - forms.gle/pL3UjPeWc2yswfM99 Limited space, so earlier applications are prioritised.

New research led by @PeterNutter1 @DJROYTBURG via @MATSprogram: a better way to update model beliefs that we call grafting. Fine-tune the base model with next-token prediction on synthetic documents (SDF), and simply apply the weight update onto the post-trained model. This works! 1/7

Frontier models can now detect AI control interventions. In particular, GPT-6 Astra is incredibly good at this. When we published the initial version of our CIAware benchmark in May most models were not much above chance level, now a step change happened. I think this is a big problem and challenges the effectiveness of control protocols. High control intervention awareness means that interventions leaks information about the monitor which helps circumventing the monitor. Details in the thread 🧵

Building Comparative Motivation Profiles with Symmetric Belief Editing will appear at NeurIPS. We refined the methodology and messaging of our performative misalignment work. We are excited to extend this to white-box auditing. by @David00845018 @1700x0 @MinTaywon @JinghuaO via @MATSprogram 1/7

New @NeurIPSConf paper: we're releasing ORBIT: A Framework for Multi-Agent Safety and Security Evaluations. Open source, with configurable attacks & defenses and dynamic topologies, all built on @AISecurityInst's Inspect. When we started this work a year ago, we didn't imagine how urgent it would become by the time we published. Below are some of the interesting results and where we think this should go next. If you want to talk it through, let us know 🧵

New paper: instead of decomposing a whole neural network, we only dissect the parts used on a task we care about. It's much cheaper, and gives mechanisms you can inspect, erase, or edit to change how the model behaves 🧵

Using AuditBench? Make sure you have the updated models! We traced an unexpected hallucination behavior to specific training examples, corrected the data, and retrained the affected models. Here’s what we found and what we checked. 🧵