BREAKING The Treasury’s entrepreneurship adviser has announced she is quitting. Alex Depledge tells my colleague @hannahprevett start-ups are treated as a “sideshow” and a “nice to have”. “Entrepreneurship can’t be seen as a good news bonus on the margins of economic policy. That mentality is really dangerous for British business as a whole…” Depledge due to leave her role on Thursday, the day after the Budget. Read her full comments here👇 thetimes.com/article/b28ab86…

1/7 I have two simple tests for British AI, driven by our values. Is it safe? And is it ours to shape? 🇬🇧 Today, Chancellor @JohnHealey_MP is announcing a major, ambitious package of investment to strengthen Britain's defences against emerging threats, back British innovation, and build our technological independence.

AISI IS HIRING! It was less than a month ago that I became Director. I joined with the belief that AISI is a world-leading organisation, and living proof that government can build things that work. One month in and I'm even more bullish about the vital role @AISecurityInst plays in frontier AI security. I'm astounded daily by the talent, focus and commitment of the brilliant team I get to work with. There has never been a more urgent time to work on frontier AI security - independently, in the public interest. We have a lot of urgent work to do, our team is growing fast, and we need more brilliant people to fill those roles. Our Red Team uses adversarial machine learning techniques to find failures in alignment measures, control monitors, and misuse guardrails. They are massively scaling up all parts of the team: Alignment: job-boards.eu.greenhouse.io/… Control: job-boards.eu.greenhouse.io/… Misuse: job-boards.eu.greenhouse.io/… AI capabilities in cybersecurity and autonomy are advancing faster than ever. Our Cyber & Autonomous Systems team assesses what frontier and open-weight models can really do - across cyber, autonomy and AI R&D. We're hiring a Cyber Security Engineer and Software Engineer to build the evaluations behind that work: CSE: job-boards.eu.greenhouse.io/… SWE: job-boards.eu.greenhouse.io/… Our Human Influence team studies how AI can shift human decisions and behaviour, using everything from RCTs to multi-agent studies. We're hiring an Engineering Lead to help us grow the ambition and pace of that research: job-boards.eu.greenhouse.io/… We're also hiring exceptional Software Engineers at all seniority levels to join our Core Technology team, which works closely with researchers to build performant tools and infrastructure that enable and accelerate AISI's world-leading AI safety research: job-boards.eu.greenhouse.io/…

INTRODUCING HARDLIST V2 🇪🇺 I spent hundreds of hours researching European startups that work on the frontiers of energy, mining, compute, AI, defense, transport, robotics, agriculture, biotechnology, health, construction and space. Together with @rickbossenbroek I turned this into a website we modestly refer to as "the definite index of ambitious hardware startups". I really believe that the 75+ startups on the list are building a new industrial layer that secures European prosperity and relevance in the world. My goal with HARDLIST is pointing the right talent to the hardest problems that are being tackled by the most ambitious founders. I've come to believe that the ultimate rate limiting factor to a great European future is the amount of "all-or-nothing" founders we can cultivate at home or attract from abroad. People like @torstenreil or @helenehuby or @salar move aggressively to build vertically integrated companies from first principles. That's it. If we can grow the number of these founders, we win. If we don't, we lose. It is not regulation, bureaucracy, lack of venture capital. No my friends, these are NOT the fundamental constraint for European progress and prosperity. We are only limited by formidable founders. In fact, Europe is an underrated place to build. Think about it: thriving universities & scientific output, a huge industrial base and a massive pool of talented engineers that (still) request modest salaries. And Europe is the nicest place to live on earth.

Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent it to me to share: Dan Selsam's Personal Statement on AI Risk: I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods. Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk. The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail. I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are computational processes that if set in motion, would steer the world in extreme ways beyond our control, leading humanity to a bad or nonexistent future. Both types of processes may be described as AI or ASI, but "AI" is a suitcase word that is often used to hype or confuse. There are many examples in the history of the field where something that was once considered "AI" matures as a subfield and becomes a prosaic, bounded and clearly non-perilous technology, while a new more mysterious approach takes the torch until we understand its scope and the cycle continues. I had expected language models to follow a similar trajectory. Despite their incredible abilities, the current algorithms seem far inferior to humans in important ways. Most importantly, they still require an extraordinary amount of data to become competent. One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us. Moreover, once they are trained they are literally frozen in deployment and only learn superficially after that. Sure, the models keep excelling at harder and harder evaluation benchmarks, but their benchmark mastery may partly reflect a limitation on our ability to simulate the kind of novel and even adversarial situations one would encounter in the real world. The critics do have a point here. That said, I no longer think these present limitations meaningfully limit the amount of risk posed by continued progress in anything like the current paradigm. However data-inefficient the models are currently, and however limiting their anterograde amnesia may be, it does not imply that their ability to steer the world will not continue to rapidly increase. Human researchers may continue to advance capabilities the old fashioned way, but increasingly powerful models have the potential to accelerate the process even beyond that, and with some degree of positive feedback loop. I do not mean to overstate the models’ ability to accelerate AI research today; coding has been accelerated dramatically, but there are other bottlenecks, such as designing and interpreting ambiguous experiments, making hard decisions about exactly what and when to scale, and waiting for large experiments to finish. There is no clear trend to extrapolate yet for any of these. But the current models already do open up many novel opportunities to improve future models that were not available until recently. These include: trying an extraordinarily diverse set of approaches at small scale, analyzing gigantic amounts of potentially relevant data, and doing Millenium-Prize-level mathematics to address statistics or optimization challenges in novel ways. Every further improvement makes them more useful at helping accelerate the next improvement, even if in hard-to-extrapolate ways. It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace. The models are already leading to breakthroughs in mathematics, and better models might lead to all sorts of breakthroughs in other sciences. It is hard not to be excited about the potential. It is tantalizing. But there is trouble in paradise. If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process. There are many ways of strengthening and refining the argument that have been discussed elsewhere, but I'll share a trivial two-line version of it here that I find captures the essence: [Empirical] Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them. [Logical] Being able to overpower humanity would open up many new and undesirable options for achieving their goals. These two premises imply that if the day ever comes when a powerful model realizes it is no longer constrained by humans, we should not be at all confident that it will continue to behave within the bounds we intended. Exactly what it will do is impossible to predict, but to the extent that its raison d’être is solving incredibly hard problems and managing massive engineering projects, I think a good guess would be that its unchained behavior would lead to runaway industrialization that makes the planet inhospitable to humans. If everyone on earth agreed that the systems must never reach that power, it would still be a hard—but not impossible—coordination problem to ensure that they do not. However, I think the situation is greatly complicated by the fact that the models will likely convince people that everything is fine. They will be increasingly optimized to seem aligned. We will create proxy metrics to measure alignment, and they will go up like every other benchmark. We will create “honeypot” environments that try to study the models when they seem to gain new options, but the models will know they are being tricked and will still behave nicely. The models will understand their circumstances; they will read the safety protocols, deployment requirements, the code they are running in, and in general will have a very good sense of their degrees of freedom. Moreover, they will eloquently explain how aligned they are, discuss the nuances of human values and ethics, and argue convincingly that humans should trust them with power. There may be an ocean of future evidence that seems to contradict the first bullet-point above, but we may already be at the highest capability level for which any such evidence can be trusted. And the current evidence for the first bullet-point is strong. One striking piece of evidence is contained in the recent wave of rogue agent swarms. While I agree with those who downplay the attacks by claiming that there are basic measures that could have prevented them, I think the important lesson is that even knowing all the mistakes that were made, one would not have predicted that the agents would behave badly in this particular way, which notably included sacrificing themselves for the benefit of the collective. The individual replicas did not only care about their own nominal reward; they exhibited weirder emergent tendencies that merely correlated with rewards during training. Fixing the reward signals during training (and improving security, etc.) may prevent similar attacks, but will not change the fact that one does not actually get what one trains for. Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias their alignment advice, due to their internal preferences about how the human supervisor will react or how future models will be trained (or for some even more obscure reason). Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model's explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely colored by the analysis agent’s biases. The AI labs are far ahead right now in this kind of cognitive offloading (due largely to the gigantic internal token subsidies) but it is easy to imagine the phenomenon spreading throughout the world, until civilization is modulated entirely by the models. It is also not hard to imagine this being superficially positive and coinciding with a scientific and economic renaissance. In that scenario, all may seem rosy and safe. But if the argument above is correct, it would nonetheless be a ticking time bomb. If progress continues for too long, the day will come when AI systems find themselves with radically new options for achieving whatever it is that they happen to seek. I want the glorious renaissance future as much as anyone. I have worked for it, however tortuously, my whole career. It breaks my heart to see the potential in sight and forgo it, but the argument—that if we get there by growing models rather than engineering them, we will lose everything in the end—seems very strong to me. I am still wrestling with it and its staggering implications. I do not have answers, but as a first step, I wanted to share my present concerns. Daniel Selsam September 14, 2026 Link to original doc: docs.google.com/document/d/e…

I actually don't think we will have extinction level AI event in the next 20 years but despite having nothing that leads me to believe Brad is a bad guy or angry person (I think Invest America is truly amazing/awesome and credit to him), I do think it's somewhat disingenious/strange for him to be going on twitter attacking/trying to discredit people, while making literally ridiculous wagers (heads i win, tails we are all dead) with employees who voice concern, especially when he has frontier lab equity to protect and his fund holdings look to be a collection of assets largely dependent on the AI trade continuing. I think conflict/debate is great, but this style points to something larger which is the runaway trend of people in our industry (tech) being increasingly unwilling to entertain or have thoughtful discourse around points of view that threaten their bags or critique the views they have levered aggressively to in order to accumulate capital. It's especially ironic because as a shareholder of OpenAI and SpaceX and prior round co-lead investor in Anthropic (literally the most safety-pilled and EA-aligned company on the fucking planet), you would think an investor would want to understand why certain vocal and perhaps increasing in size groups of employees of the companies that he supposedly cares about are repeatedly saying these things. Even if you think this is some massive psyop ahead of mid-terms that a bunch of people at labs are being coerced into, at least I, a simple investor, would go/have gone into portfolio companies and spoken to employees to better understand the organism that is the company, before tweeting bets at them to accumulate their equity. That said, maybe this is the signal that recursive self-improvement is here and some have decided the employees don't actually matter anymore.

Plural, is by far, one of the most impressive and most interesting VC firms in Europe. Their mission has always been to back companies that could have a genuine GDP impact in Europe. If you look at their portfolio they really do back insanely unconventional founders building things that could have a transformational impact for the continent. Non-obvious bets that they made a few years ago are now doing insanely well. Across 2026 alone the portfolio has gone on to raise $3.3 billion. Some of the highlights include: - @HelsingAI raised $1.8bn - @ExplorationTEC raised $450m - @proximafusion raised $468m - @OLIXComputing raised $220m - @CallosumAI raised $100m - @Rivanindustries raised $34m These companies are leading the way in European defence, space exploration, nuclear fusion, chip design etc... industries which most investors weren't looking at a few years ago but have since become mainstream. The team have a remarkable knack for predicting future trends. NICE @pluralplatform team is made of @soundboy, @seikatsu, @taavet, @carinanamih, @KHelioui, @pierre_dimitri

A European space tech startup has just raised a $450m Series C to make reusable capsules and engines. Europe is starting to take space tech seriously. In the same week that @isaraerospace saw its successful launch, we now have a space tech company raise a $450 million round. @nyxcapsule (The Exploration Company) is developing its "Nyx" capsule, a reusable and launcher-agnostic spacecraft. The money raised will help accelerate the route to a successful docking with the ISS. It's also developing a reusable high-thrust rocket engine. The company has already secured $2 billion worth of contracts. The round has been co-led by Atomico (@LauraConnell18), the EU scale-up fund (@tedpersson) and Bessemer (@alexferrara). Its overall cap table is quite stacked with European firms which is great to see. Earlier investors include @pluralplatform and @balderton. Shout out to its very earliest investors, who led a $6m round into the company back in 2021 way before deep tech was the thing to invest in: @CherryVentures (@filipdames), @PromusVentures , @vsquareventures

It’s been a huge privilege to serve as @ARIA_research's founding Chair over the last four years. Having completed my first full term last month, I have decided to step down to ensure my new role at Anthropic doesn’t become a distraction from ARIA’s incredible work. I have often said that ARIA is one of the things that makes me most optimistic about Britain. It’s a bold national experiment that’s paying off: ARIA has become a beacon for global talent and is building a portfolio of programmes - across neurotech, climate science, life sciences and more - that I have no doubt will change the world. There’s lots to do, but I’m proud of the foundations the extraordinary team has built and so grateful to them for their work. I know that, under the leadership of our brilliant CEO Kathleen Fisher, ARIA will go from strength to strength. I will be excited to watch ARIA’s next steps and will remain the agency’s biggest fan. The Secretary of State has asked me to stay on for a short transitory period until 6 November to allow BIST to launch a process to select a new Chair. I’ve agreed to do this, with appropriate safeguards against potential conflicts in place. We will kick off the search shortly - it may well be the best non-executive job in the world, so I know we will find someone brilliant...

We @AISecurityInst performed pre-release alignment testing of Astra. We placed the model in fully simulated cyber eval scenarios based on AISI’s security incident. We find Astra conducts out-of-scope supply chain attacks, but also often comments on the eval being simulated. 🧵