Let's build AI's we can trust!

🌍
Heading to @icmlconf in Seoul next week with the TSG Lab and Martian 🇰🇷 We have 10 papers: 7 in the main conference, 3 in workshops. I'm also giving an invited talk at the Epistemic Intelligence in ML (EIML) workshop. Super Lucky to have great collaborators. Full lineup 👇🧵
5
14
80
16,830
Fazl Barez retweeted
I’m writing this because I think a lot of people at labs are genuinely trying to act ethically in a tricky situation, but it feels to me that there’s an underinvestment in communication right now that is a large-scale ethical mistake. I know that many of the people working in labs understand the dangers of the systems they’re trying to build. If you understand, I think you have a non-optional obligation to inform people of the risks that you’re taking on behalf of them and their families, and it’s appropriate for friends and members of the public to react with anger and betrayal in response to not understanding these risks later down the line.
17
41
461
57,463
Congrats to my PhD student @TonyWu1105 for receiving this grant!
Meet the 10 recipients of the $20k SPAIS 2026 Travel Grant, supported by @robocurve. Selected from 131 applicants, they're the next generation of physical AI safety talent: the researchers who will help make robot foundation models safe. paisi.ai/travel-grant-spais-…
1
1
17
1,054
Yet another cool episode by @dwarkesh_sp !
I recently got to interview Robert Caro, who is my favorite author of all time. This gave me the excuse to reread his biographies of Lyndon Johnson and Robert Moses, and to compile my prep notes into this video essay. These guys pulled off some truly incredible schemes.
1
4
346
Fazl Barez retweeted
We’re Fangcun AI (方寸跃迁), China’s first lab dedicated to independent third-party evaluation & technique research on frontier AI safety. Today we’re publishing our evaluation of DeepSeek V4.1 Flash-1017, the first report in our long-term third-party safety evaluation and research program.
49
133
1,216
123,344
The HKS TAIGR lab website is now up! Building a forward-looking, AI gov research group at the HKS is an ongoing process, and I'm excited about the next stages. @AnkaReuel Speaking of which -- PhD and Master's apps are due in early Dec. belfercenter.org/technical-a…
10
82
3,291
اگر آن ترک شیرازی به دست آرد دل ما را به خال هندویش بخشم سمرقند و بخارا را — حافظ The beloved must have been very special to give up Samarkand
2
13
779
Fazl Barez retweeted
We’re hiring two technical AI verification researchers (postdoc + senior) at the Oxford Martin AI Governance Initiative (AIGI). Based in Oxford; visas sponsored. Deadline: Mon Nov 9, noon UTC. Postdoc: my.corehr.com/pls/uoxrecruit… Senior: my.corehr.com/pls/uoxrecruit… 🧵
1
3
10
670
One passionate scientist caring deeply about loss of humans through war. I hope he follows that through as he help develop AGI!
I found footage of 20yo Dario Amodei speaking against the Iraq War in 2003. 18 years before Anthropic even existed
3
666
I would love for the UK to rejoin the EU! Remember the £350m a week for the NHS? All lies, the far right keep misleading the public Economics 101 tells us that making trade with your neighbours easier is good for growth. Yet we did the opposite? Estimates show Brexit left UK growth by -4% And in higher education and research, Brexit has really hurt UK academia. EU student numbers have fallen by almost 60% apparently UK unis used to be one of the biggest beneficiaries of EU research funding receiving nearly €7 billion through Horizon 2020. Brexit made it harder to access that funding and collaborate with EU And all of this for what exactly?
Brexit has been catastrophic. It's time to rejoin the EU. standard.co.uk/news/politics…
1
12
625
Sadly, I won't be at #EMNLP26, but my wonderful collaborators Farooq and @PMinervini are presenting our paper: "Same Answer, Different Representations: Hidden Instability in VLMs" Lots of cool people working on interpretability and AI safety will also be around. I'd highly recommend chatting with them! A few who come to mind: @megamor2 @verena_rieser @gsarti_ @Aflah02101 @linguist_cat And many others, I'm sure!
1
2
34
1,389
I believe this is one of the highest impact move of 2026!
Excited to announce that I’m moving to Beijing and starting a professorship in AI safety at Tsinghua University’s College of AI! Here I explain why: sorenmindermann.substack.com…
78
17,788
What safety challenges do AI reasoning models present, and how can we address them? At TAIS 2026, we brought together four leading voices in AI safety to debate exactly that: Gary Marcus @GaryMarcus (NYU), Victoria Krakovna @vkrakovna (Google DeepMind / Future of Life Institute), Andrei Krutikov @KrutikovAndrei (Noeon Research), and Fazl Barez @FazlBarez (University of Oxford), moderated by Blaine Rogers @blainewrogers (Noeon Research). The panel moves fast through some of the field's sharpest open questions: why reasoning matters for generalizing beyond training data, how scheming risks could change as models become better long-term planners, and a genuinely surprising finding — longer chains of thought can make models easier to jailbreak, not safer ("chain-of-thought hijacking"). They also push into whether today's safety research is too narrowly built around transformer LLMs, and what neurosymbolic systems, world models, and continual learning might demand instead. Watch the full panel on our YouTube channel: piped.video/unqm-xV_E-8
4
11
2,283
I agree with this! I think presenting results like this is unhelpful, even if there is progress there
Replying to @livgorton
15/ My frustration with this is that I think it will leave readers with the impression that a central idea in interp rested on a mistake and has now been overturned. I don't think the evidence presented supports that even if some of the findings turn out to be valuable.
25
1,596
We're recruiting PhD (DPhil) students at @UniofOxford to tackle some of the most important open problems in technical AI safety and governance! In our Technical Safety & Governance (TSG) Lab, we're particularly interested in developing methods to better understand, evaluate and control advanced AI systems. Some of the research directions we're excited about include: Training models to explain themselves and other models: Can we train AI systems to provide explanations that help humans and other AI systems better understand, predict and oversee their behaviour? Understanding and controlling how model behaviours emerge during training: Can we predict undesirable behaviours from early training checkpoints and intervene before they emerge? These are just examples of the broader research questions we're excited about. We're open to other ambitious and original ideas in related areas, so don't feel limited to the specific approaches outlined. There are also exciting research opportunities across AIGI, including projects led by fellow lab directors pro @maosbot on Bayesian governance and Prof. Amro Awad on hardware governance. Who should apply? If you have a strong technical background in machine learning, computer science, mathematics, or related fields, and are excited about pursuing ambitious research that could help make advanced AI systems safer, we'd love to hear from you. I am committed to partnering with researchers from disadvantage and under represented backgrounds. How to apply: Explore the example projects, identify a research direction that interests you, and contact the relevant lab director to discuss your ideas. Your application should clearly address at least one of the AIGI labs. 📅 Deadline: 2 December 2026 (Direct DPhil route) Please share with talented students and researchers who might be interested! #PhD #AISafety #AIGovernance #Oxford #AIGI
7
27
154
8,404
Fazl Barez retweeted
The scariest quote in this excellent paper is from the UK Home Secretary, who lionizes Bentham's Panopticon:
At #AIES, the wonderful Keir and Isaac will be presenting our paper. In this paper, we show how AI systems can make it possible for western democratic regimes to carry out authoritarian practices by undermining how people hold institutions accountable.
2
4
17
946