This is huge— @Cassi_on_X came 2nd overall and 1st on "Dataset" questions on @Research_FRI Forecastbench! 🥈📈 Only @elonmusk's @xai @grok outperformed us overall. What this means: 🧵

The server was hacked and someone exfiltrated the key. gg. news.ycombinator.com/item?id…

Seeing some bad takes on British Government AI funding: some notes. 1. The Government has announced a tranche of funding for a number of different AI-related organisations and initiatives. It comes to roughly £600m (excluding private sector input.) This has led to a series of partisan bad takes by people who don’t really follow this conversation. That is annoying because only I can have partisan bad takes. 2. Two common takes: “this is nothing, AI labs spend hundreds of millions every day/month etc! We can’t compete! Common British L” Or “Government is terrible at picking winners and investing, should be left to the private sector.” These takes reflect usually partisan pre-existing notions, neither actually understands what this money does or doesn’t do, and neither actually examines what the money is going towards either. Apples to oranges type stuff. 3. Let’s break down two of the more significant sets. The first is more funding for AISI (+£120m), and the second is for a new AI Biosecurity Lab (£100m, + around £200m from frontier companies). 4. AISI is the world’s best resourced institute doing this work and leading in its class. Don’t take my word for it - feel free to read MI5 Chief’s recent speech, or any number of expert statements from frontier companies. IT IS NOT a frontier AI company competitor. IT IS NOT designed to produce cutting edge LLMs or agents or consumer applications. It is, effectively, a research organisation staffed by clever people whose job is to analyse what frontier AI companies are working on and test their stuff. So if you’re measuring this announcement of investment and saying “it’s nothing compared to what AI firms spend”, you’re comparing totally different things. Instead, look how much spending Britain’s AISI gets relative to its international counterparts on getting access and insight into this frontier. 5. The UK’s AI Biosecurity Lab (AIBL) is even newer than AISI, and a world first. IT IS NOT a competitor to private sector biolabs. IT IS NOT an example of Government intervening and ‘picking a winner.’ Its job - modelled after AISI - is to “develop technical capabilities that help the UK better understand, prevent, detect and respond to biological threats.” It will do this by attracting best in class talent + working with frontier companies + working with the UK’s wider biosecurity landscape + other agencies. 6. Both of these organisations represent continuity in British Government thinking: ‘AI brings with it a whole bunch of positive things, but also risks. We should create organisations that attract top talent to help us analyse and understand this technology.’ You see this in MI5’s recently launched AI Threat Assessment Centre (AITAC) and the AI Economics Institute (AIEI) launched earlier this year. You can absolutely think this is the right or wrong approach btw but that’s the broad strategy. 7. So the comparisons to frontier companies’ spending is just misplaced. You’re working yourself into a tizzy over a misunderstanding of why the state is allocating resources. You shouldn’t compare these to frontier companies, but instead to other AISIs or other foreign govt AI investments I guess. Finally, if I asked many of those commenting what they would do differently, I would guess they wouldn’t really know.

That former treasury advisor is, surprisingly enough, right. Sovereign AI is the only game for Britain to play now. We must mobilise maximum resources toward it, or be left behind, forever.

Project Agincourt will establish AUTOWARCOM and prototype the fusion of development and operations, pushing money and decisions to operational units.

This is not insane. This is slightly creative. If this seems insane to you, you have not at all internalized what it means to have an adversary that is more intelligent than you and optimizing against you.

Today we're setting out a British approach to AI: backing innovation and growth, managing risks responsibly, and making sure the benefits of AI-enabled growth are felt right across the country.

The most prescient AI forecasters I know: * Carl Shulman * Daniel Kokotajlo (What 2026 looks like, AI 2027) * Leopold (Situational Awareness) * Ryan Greenblatt (#2 in 2025 forecasting competition) All seem to expect broadly beyond-human AI 2028-2029.

1/7 I have two simple tests for British AI, driven by our values. Is it safe? And is it ours to shape? 🇬🇧 Today, Chancellor @JohnHealey_MP is announcing a major, ambitious package of investment to strengthen Britain's defences against emerging threats, back British innovation, and build our technological independence.

Naom Brown vindicated.

Consider that the people who were right about AI-enabled cyberattacks might be right about AI-enabled bioweapons

The cyberpocalypse is going to hit British politics like an asteroid. So many people have heads buried so far in the sand (just look at what Parliament spends its time discussing...)

# Black October For anyone who knows me even somewhat well, it's clear that I am a 99th percentile tech optimist. However, the vibes have shifted for me quite a bit in the last 18 hours, and I feel a moral imperative to share what I'm thinking and feeling with my circles so that everyone can understand, and hopefully take seriously, what's happening right now. I think October 2026 will have many chapters written about it in the history books. You know those scenes in financial movies like Margin Call where someone realizes that they're all fucked, and no one else sees it yet? And that person feels like they're crazy, and then suddenly everyone is pulling an all-nighter in a boardroom and freaking out? Yeah, that's kind of how this is feeling right now in the SF/Bay Area/AI circles. We're 48 hours into what is being called Black October. In the last 2 days there have been several dozen cyber attacks which, in isolation, would each have been the worst to happen in an entire calendar year. Every sector from government to education to big tech and banks is being massively affected, and people are starting to freak out. In a nutshell, what's happened is that frontier AI models have gotten to a point where they are as advanced as the most sophisticated programmers, with one critical difference: they can work 24/7, and you can spin up 10,000 of them with a single command and they'll work together tirelessly until they accomplish their task. The most advanced models today are from companies like Anthropic and OpenAI, whose current best technology is so good that they haven't been able to roll it out to everyone yet because of the national security risks it poses. Meanwhile, the best open source models (where anyone can just download the model weights and do whatever they want with them, uncensored) are slightly behind the frontier but still 90-95% as capable. These models, developed primarily in China, are being used by black hat hackers to wreak havoc on the Western capitalist structure. Presumably, this has been part of China's plan the entire time, and we're seeing its implications fall out in real time. In red-pilled AI circles, we've been sharing and reading papers called things like "Situational Awareness" and "AI 2027" for the last year or so, which at the time felt a bit like near-term speculative fiction. They laid out the case for why Artificial General Intelligence (AI smarter than any human at any knowledge task) and Recursive Self-Improvement (models that are able to use themselves to get better) weren't half-baked future ideas anymore, but instead an inevitable outcome of the linear progress that's happening. These papers would then talk about things like "takeoff scenarios," trying to envision how that moment would come to be and what it would be like when it happened. It would either be a "slow takeoff" that would play out over the course of months and quarters, happening subtly and only obvious to a few. Or, it would be a "fast takeoff" where it would pretty much happen instantly, like a bit flipping from 0 to 1. From everything that I can see and am feeling, we are currently experiencing the fast takeoff scenario. Beyond the crazy security vulnerabilities that have come out recently, there have also been massive breakthroughs in life sciences, fundamental computer science, materials science, and pure mathematics that have all landed in the same time frame. While I still firmly believe that AI will be a net positive for humanity, it's more clear to me than ever that the societal and individual risk is very real, and is playing out NOW. For as much conviction as I have that this is happening, the thing that's caused me to lose sleep is that I don't really know what to do about it, or more importantly, what to tell you, reader, to do about it. So, the best I can do is make a few suggestions that have always been a good idea but are more timely now than ever: Set up 2-factor authentication for everything. I use Duo. Use a password manager across your devices. I use 1Password. Get an identity protection app set up. I use Alta. Move your IoT devices to their own network. Claude helped me do this. And last, but certainly not least, if you've been avoiding leaning into AI, see this as a catalyst to wake up and start using it. Don't just use the chat mode version of ChatGPT/Claude; click that Code/Work button and see how easily you can have AI write code to help you with your job. One of the biggest vulnerabilities was in a massively popular programming language called PHP, and the prompt that set it off was literally, "Find vulnerabilities in php." This is an unprecedented time for humanity, but as we've done with many things before, we will prevail so long as we are kind, loving, and open-minded to the world. Much love.

SITUATION EXPLAINED: US banks are finding AI agents inside customer accounts, and some appear to be acting without the user's instructions. • The banks include Morgan Stanley, Bank of America, JPMorgan, and Wells Fargo, and some don't even know agents are getting in • Many are people's own agents, like Meta's Muse, logging in with the user's password, which most banks don't allow • Banks often can't tell a person from a bot, and some agents have even called customer service lines • Banks are blocking agents, rewriting their terms, and moving to read-only access that shows balances but can't move money • Schwab says it stops outside agents trying to move client money "in the tens and twenties" every week • Many agents can tap through your phone the way you do, banking app included, and your phone already holds most of your life @theojaffee: "It seems like there's definitely a problem with people abusing AI agents before they are reliable enough, before they are aligned enough, and before they are capable enough to take actions on behalf of the user, kinda like someone with power of attorney would, without messing things up."

If you want an analogy, imagine you're a beetle entomologist in some tropical jungle. Drop a quadrat square anywhere you want, discover 20 new species an hour. The attack surface is hyperbolically enormous. It's a Cambrian explosion of entirely new kinds of exploit.

OpenAI says the PHP security flaw (found with GPT-5.6 Sol) is the tip of the iceberg, and they have several more discoveries to announce.

Forget air conditioners and riots, Europe is truly fucked. If the UK can't have a power grid capable of running an AI datacenter until the mid-2030s, it will have no on-prem datacenters for any of the AI tasks. Remember when the EU demanded all data storage to be brought to its borders? It won't be able to demand AI compute be brought there, even though AI compute is a way more important thing than google search history. A future US President could cut of EU AI access in a dispute, leaving EU businesses and governments adrift. Governments like Saudi Arabia have been investing heavily into ensuring that they have local AI datacenters, and Europe can't even power one. Meanwhile the UK government tried to pass laws saying they would limit AI development, like a toddler watching their mom cook and putting a plastic hot dog in their fake microwave.

The NCSC is encouraging UK organisations to prepare for immediate updating of systems using PHP. Where immediate remediation is not feasible, organisations should consider taking the system offline until it can be fully mitigated. For more⬇️ ncsc.gov.uk/news/critical-se…

I recently spent some time with former colleagues from academia. All of them brilliant, peak human capital. Almost all of them fundamentally unable to plug into the economy, inoculated against productivity by a youth spent experimenting in laboratories instead of building wealth for society. Dealing with real people in the real world. 90% of them have left academia in the last 10-15 years, and 90% of those are functionally unemployed. Grad school does a lousy job of training scientists to be professors (since most of them don't get the good postdoc or tenure track position) and an even worse job of preparing people for real life. We complain that our societal infrastructure is collapsing, our state capacity is low, our incumbent industries are faltering. Where are the next generation of leaders that can push us forward? A huge fraction of them were conveyor-belted into academia by schooling systems that have no mechanism to channel ambition and 99+%ile competence, and our universities are mostly hedge funds with glorified psych wards/academic departments on the side, running an insane pyramid scheme that exists only to prove, over and over again, the Shirky Principle: Institutions will try to preserve the problem to which they are the solution. We literally poured trillions of dollars into state-funded research and effectively neutered our elite civilian technical leadership development programs. I recently asked a few academic friends if they were finding AI useful in accelerating their programs of research. You know, to solve the Big Problem their entire field exists to solve? Some were. But most were complaining about how it created pedagogical issues, explaining how their department decided to ban the use of AI, or how it took 18 months to approve an enterprise account, or how they tried Gemini 2 on a puzzle a year ago and it wasn't very helpful, or repeating nonsense about datacenters and water consumption, or supposed IP theft by model companies, as though that has any bearing on their ability to, eg, accelerate a cure for cancer. Elon sometimes calls AI a supersonic tsunami - a huge wave you don't see coming. I see it coming. But even the people who you'd think have been waiting for this their entire careers are steadfastly looking in the other direction.

This was fun, with thanks to @Independent and Jonathan Margolis. independent.co.uk/news/uk/ho…

The Tricolore textbooks did not prepare us for all this shit going off in France. The most exciting thing that ever happened in La Rochelle was Annette falling off her bike or Philippe enjoying a particularly tense game of le badminton down l'auberge de jeunesse.