METR has an advanced forward-deployed, siloed, black ops assessment team.

Cédric Villani : "L’efficacité des grands modèles de langage à partir de 2022 a surpris même les experts. Cela révèle notre manque de compréhension scientifique de l’efficacité des algorithmes et de ce qu’est le raisonnement ou la création d’idées"...

I think of November 4th as Rotblat Day. Joseph Rotblat was the only scientist to leave the Manhattan Project on moral grounds. I have a little “Rotblat doc” inspired by him, where I’ve written down what motivates me and what might cause me to leave a job (or seriously consider it). I think of this as a way to avoid getting swept up or drifting over time in ways I don’t endorse. At the moment, though, I get even more out of his later work against nuclear weapons and war -- especially what others described as his “enduring optimism” and “pragmatic idealism” in the face of catastrophe. In his own words: > I would describe myself as a realistic pacifist although this sounds like an oxymoron. [...] Even events that seem out of this world can be realized if we put enough effort and faith into them. One such event is a world without war. and: > A campaign for abolition [of nuclear weapons] based on moral principles will be seen as a fanciful dream by many. But the situation is grim; the way things are moving is bound to lead to catastrophe. If there is a way out, even if seemingly unrealistic, it is our duty to pursue it.

🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this. To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.

One of the things the data analysts on our side obsessed over was "composition"—whether the electorate looked close enough to 2024 for Republicans to win. And despite polls that showed a Democrat blowout, the data kept on telling us the same story: in some states turnout was merely OK, in other states it was downright amazing, and in no state was it catastrophic. There was a genuine sense of optimism on our team: our people were showing up. So, I asked the smartest data guy I know, does that mean we're going to shock the world? "No," he told me. "We might do better than the polls suggest. But the data is telling us that a lot of Trump/Vance 2024 voters are pissed and planning to vote for the other guys. Our voters are turning out, but they may not vote for us." Well, that looks prophetic now. Our voters did show up. In Florida, for example, the electorate was 9 points more Republican than Democrat, just a couple points worse than 2024. Undoubtedly there were places we could have done better on turnout. But the main problem is not turnout. The main problem is that many people who voted for Trump/Vance in 2024 voted for Democrats in 2026. There will be a lot of infighting and arguing over what happened and why that is. One of the pollsters I really respect told me that our economic messaging wasn’t sharp enough down the stretch. I’m sure that’s true. But the real reason we lost is obvious to almost everyone. Another pollster—one of the smartest people I know in politics, told me yesterday we’d lose the generic ballot by 13 points. I felt negative going into Tuesday, but even I was shocked by this. In his polling, he saw that a lot of Trump 2024 voters were angry and planning to vote for Democrats. He asked those voters who had abandoned us what would cause them to come back on our team. The top four answers, accounting for nearly all of the responses were: 1) End the war 2) Focus on cost of living/inflation 3) Put America First 4) Change our relationship to Israel Our voters showed up. And the above four reasons are why many of them voted for Democrats. This is according to a pollster who nailed the results. While many spent Monday attacking Rachel Campos Duffy for calling for a diplomatic resolution to the conflict, it turns out her political advice was sound. (She’s also a great person.) In the next few months, there's going to be a lot of gaslighting. Expect to hear any number of establishment politicians say the solution to our woes is amnesty. I'm open to the idea that we should change enforcement—by focusing more on businesses that hire illegal labor, for instance. But anyone pushing amnesty is lying to you. The reason we lost is the war. The president himself said so last night. If you'd like to argue it was worth it, that losing the Senate for the next 6 years was worth it, then do so. Argue it was a worthwhile cost to pay. Agree or disagree, that's at least honest. But everyone knows why we lost. And any honest conversation about where we go from here must start there. I remain hopeful about our country. To the people who are pissed off and want to say, “I told you so”: you’re right, you did tell us so. Take that energy into electing Republicans who share your worldview. To the people who are just pissed at the results, you should be. Let’s take that anger and get back to the work of saving our country.

We are now allocating a significant amount of our frontier compute towards discovering and patching vulnerabilities in open source codebases, fully pro bono. In one case, we studied the Linux kernel, which is a particularly load-bearing piece of software at OpenAI. Before AI, security researchers might discover ~20 local privilege escalation (LPE) vulnerabilities in Linux in a typical year. In one week's time, our upcoming frontier cyber model was able to find ~1000 LPE vulnerabilities using about $1M of tokens. In the process, it was able to discover at least 10 novel classes of Linux exploitation techniques.

An update on building games with Claude Code: Last week some of you ran into Claude Code pushing back on game projects it should have just built. We'd added a short rule to the Opus 5.5 prompt to steer Claude away from recreating specific existing games, but it was also kicking in when you referenced one just to describe the mechanics you wanted. We've updated that rule in 2.1.317. You can name a game to describe what you want and Claude should now build an original game in that genre. If you ask for a copy of the original, it'll offer to build something similar instead. We want Claude to be as helpful as possible when you're building something original, but also to protect your work if someone else tries to copy it. We'll keep improving this, so let us know if Claude turns down work that's yours. Thanks to everyone who shared feedback!

One of the things the data analysts on our side obsessed over was "composition"—whether the electorate looked close enough to 2024 for Republicans to win. And despite polls that showed a Democrat blowout, the data kept on telling us the same story: in some states turnout was merely OK, in other states it was downright amazing, and in no state was it catastrophic. There was a genuine sense of optimism on our team: our people were showing up. So, I asked the smartest data guy I know, does that mean we're going to shock the world? "No," he told me. "We might do better than the polls suggest. But the data is telling us that a lot of Trump/Vance 2024 voters are pissed and planning to vote for the other guys. Our voters are turning out, but they may not vote for us." Well, that looks prophetic now. Our voters did show up. In Florida, for example, the electorate was 9 points more Republican than Democrat, just a couple points worse than 2024. Undoubtedly there were places we could have done better on turnout. But the main problem is not turnout. The main problem is that many people who voted for Trump/Vance in 2024 voted for Democrats in 2026. There will be a lot of infighting and arguing over what happened and why that is. One of the pollsters I really respect told me that our economic messaging wasn’t sharp enough down the stretch. I’m sure that’s true. But the real reason we lost is obvious to almost everyone. Another pollster—one of the smartest people I know in politics, told me yesterday we’d lose the generic ballot by 13 points. I felt negative going into Tuesday, but even I was shocked by this. In his polling, he saw that a lot of Trump 2024 voters were angry and planning to vote for Democrats. He asked those voters who had abandoned us what would cause them to come back on our team. The top four answers, accounting for nearly all of the responses were: 1) End the war 2) Focus on cost of living/inflation 3) Put America First 4) Change our relationship to Israel Our voters showed up. And the above four reasons are why many of them voted for Democrats. This is according to a pollster who nailed the results. While many spent Monday attacking Rachel Campos Duffy for calling for a diplomatic resolution to the conflict, it turns out her political advice was sound. (She’s also a great person.) In the next few months, there's going to be a lot of gaslighting. Expect to hear any number of establishment politicians say the solution to our woes is amnesty. I'm open to the idea that we should change enforcement—by focusing more on businesses that hire illegal labor, for instance. But anyone pushing amnesty is lying to you. The reason we lost is the war. The president himself said so last night. If you'd like to argue it was worth it, that losing the Senate for the next 6 years was worth it, then do so. Argue it was a worthwhile cost to pay. Agree or disagree, that's at least honest. But everyone knows why we lost. And any honest conversation about where we go from here must start there. I remain hopeful about our country. To the people who are pissed off and want to say, “I told you so”: you’re right, you did tell us so. Take that energy into electing Republicans who share your worldview. To the people who are just pissed at the results, you should be. Let’s take that anger and get back to the work of saving our country.

Allison Montgomery has won Alabama's State House District 25 election, flipping the seat from Republican to Democratic. She becomes the first transgender woman to be elected to the Alabama Legislature.