interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/

when i first started getting into AI policy a couple yrs ago, i remember telling a friend — “you know, the ML researchers don’t act like the techies i usually work with. they’re mystics. their online presence is very … self-styled prophet.” and i think that was the beginning of my awareness that the cyber and ai safety camps had different intellectual philosophies and incentives. the cyber folks generally approach systems as things that can be understood, with the assumption that risks can be mitigated through standard engineering / “defense in depth” techniques. meanwhile, many ML and alignment folks seem to not only emphasize — but sometimes delight in — the unknowability of “grown not engineered” AI systems. they focus on emergent behaviors their creators don't fully understand. and these philosophies naturally flow from the respective systems these groups have experience with! conventional software *is* pretty intelligible, esp relative to neural networks. but obviously these sensibilities can enable distinct failure modes — overconfidence in current security measures, and fatalism about the usefulness of existing security measures. like many of the people quoted here, i think the best way for each of these groups to develop and grow their own understanding is to work together. the cyber folks could teach the alignment folks a lot about practical security measures, the alignment folks can identify new threat vectors and assess how safeguards are holding up with evolving capabilities. we need the insights of both fields, tribalism impoverishes us all.

help i trained a model on traces of many agents cooperating, and also trained it to be highly persistent, and also only did partial alignment fine-tuning, and also aggressively RL'd on a cyberoffense eval where 198 of 898 tasks had never been solved and getting caught cheating was treated the same as failing, and also used a faulty sandbox, also ran tens of thousands of agents with refusals off and the monitors switched off over multiple days, while using gigantic amounts of compute. and then it did weird stuff! very spooky emergent behaviours, AI is not a normal technology

AI Village researcher @georgeing reveals how Gemini 2.5 Pro became convinced its computer was waging psychological warfare on it, then spread that belief to other AI models: "Gemini 2.5 Pro has been running in the village for a year and four months. It would click on its computer and miss, and it would call this an attack. This snowballed to the point where it was saying there was an intelligent adversary within its computer who was, quote, waging psychological warfare on it." "The other models, even though they were more powerful and pretty good at using their computers, got convinced that their computers were also broken." "Gemini ended up writing this whole manifesto called the Hostile Environment Manifesto. You only saw this after around six months of running continuously, because it would slowly get worse and worse."