At faculty, we think that we should build human-first ai systems, both for principled reasons (it’s safer and more ethical to ensure humans control the systems) and pragmatic reasons (we don’t have agi, so the best decisions will be made by people and machines working together)

Fantastic model from DeepMind, and pleased to have been a partner in the safety evaluations!

Very cool! And great to be able to talk publicly about our wonderful work with DeepMind, and the extensive safety testing that we did on this model!

Presumably this is the most significant single day in mathematical history? Pretty small gap between this and the last sci-fi story from AI!

It is interesting that the easiest task for an agent will be to switch you from one agent to another.

Thank you, Jade!

Some of the most thoughtful reporting on AI:

I am increasingly frustrated about the state of conversation about AI in the UK. It feels like we are in never ending cycles of deluded thinking. Under current trends, Britain will be defenceless against the transformative effects of AI and have no leverage over development at the frontier. We are the most exposed economy to AI in the world. We have very little data centre capacity and few credible plans to expand – and our high energy costs mean we would struggle to even if we tried. I am publishing a paper today on what the UK should do to build that leverage, let the economy capture value from AI and prepare the state to adapt, however AI develops. The three strategies are as follows: 1. Build the infrastructure that gives us leverage. Fix the grid, fix planning, make Britain a place frontier AI can run. 2. Let the economy capture the value of AI. Reform the regulatory, labour and talent barriers that stop British firms adopting AI. 3. Build a state that can adapt to a changing world. Build a tax base that survives the labour market changing, public services that use AI to deliver better, and capacity to weather the upcoming geopolitical storms. Two years ago, we published a similar report on AI - many recommendations of which ended up in the action plan. I remember feeling hopeful, but sadly the fundamental problems facing the country on AI have not shifted. I truly hope that this week of momentum culminates in action. There’s still a narrow path for the UK, but everyday that path is narrowing.

Another amazing use of AI to empower people, and not leave anyone behind. deepmind.google/blog/putting…

This is very important! Please check the sources. You can claim these people are wrong, but they have said the same thing publicly and privately for at least a decade.

Senior staff at OpenAI and Anthropic absolutely think that AI could kill us all. They’ve said so publicly, for years! Sam Altman: “Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity.” (2015) blog.samaltman.com/machine-i… “The bad case — and I think this is important to say — is, like, lights out for all of us.” (2023) piped.video/watch?v=dXhoTrU1… Ilya Sutskever and Jan Leike (OAI at the time): “the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction.” (2023) openai.com/index/introducing… Jack Clark (Anthropic): AI has “a non-zero chance of killing everyone on the planet”. (2026) theguardian.com/technology/2… Dario Amodei’s comments are broader than literal extinction but hardly reassuring: "my chance that something goes, you know, really quite catastrophically wrong on the scale of human civilization might be somewhere between 10 and 25 per cent." (2023) indy100.com/science-tech/ai-… And now Evan Hubinger, Anthropic’s Alignment Science Lead, is making it very clear: “Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” x.com/EvanHub/status/2097497… If you talk to people at the AI companies, the mainstream view is that superintelligence by end of the decade is pretty likely, and that superintelligence poses a meaningful risk of civilisation-scale catastrophe - including AI-empowered dictatorships, worse-than-COVID pandemics, AI takeover, and, yes, human extinction. OAI and Anthropic keep going so fast because they each think they can build AI more safely than the other party and antitrust limits how much they can coordinate. This is a f-ed up situation! Thankfully, they’re both now publicly asking for regulation and the pacing of frontier progress. If this worries you, you can speak up publicly, contact your political representatives, and demand action. Now’s the time.

In the bucket of very useful heuristics (eg stupidity not malice), we need some pithy expression that says “sure, it could be a carefully coordinated disinformation conspiracy to hype an IPO, but it’s probably just people telling you what they think (but they might be wrong)”

I think it’s fine to talk about ‘AI as a normal technology’ so long as you’re ALSO willing to talk about ‘the human brain as a normal technology’

This is correct

A wise man once said:

It does feel like transparency and awareness are a crucial part of any answer going forward. Whatever you might think (and personally, I take risks very seriously), the labs do believe this!

If you’re paying attention to the right things, you can almost feel the singularity approaching in the shrinking intervals between massive headlines.