1/ New essay with @blaiseaguera and James Manyika at the DeepMind Institute: AGI may not arrive as one titanic mind but as societies of agents. The hard problem shifts from building a mind to orchestrating, governing, and living inside many of them. institute.deepmind.com/essay…

I'm absolutely thrilled to have written this article for Long Now. It's about AI and time: how we got here, and what we can do now, when all of the pace layers are disrupted at once by an influx of new minds. I wrote this over the summer. At the time, it seemed maybe too weird, like people wouldn't believe or understand what I was saying. And of course, everything changes so fast that it's hard to imagine writing something about AI that far in advance and have it still make sense. But I think the events of the last few months have only made it more relevant. The Long Now Foundation has been an inspiration to me for a decade now, and has changed my life in countless ways. I'm incredibly grateful and honored to be among so many of my heroes and friends and colleagues represented in this issue of Pace Layers and Long Now Ideas. I wouldn't be here in San Francisco, here on Twitter, or here in the field of artificial intelligence if not for this community. The conversation about AI has been fractious. It's very hard to think about how to fit this sudden, disruptive, technological change into the 10,000-year view. I hope I provide some insights, or at least some new angles for the conversation. It's scary to go on the record, especially about something that will affect everyone, and affect the rest of history. If I'm wrong, I'm wrong forever! Please read and enjoy, and post quotes and screenshots, and buy Pace Layers Issue 3 so you can have a permanent hard copy. I have always wanted to be a part of the 10,000-year library, so I have to be serious for once and try to stand for something. Let me know what you think.

.@bratton warns that building a single AI aligned to "universal human values" could create a global cognitive monoculture: "The idea that the best way going forward is to build a single model that is aligned to something called universal human values, established by some kind of eval, to me is a recipe for a global cognitive monoculture. It’s incredibly brittle. It’s just not the way healthy ecologies work." "The Earth folds itself into the shape of prefrontal cortexes in bipedal binocular vision primates, who are then able to fold sand into the shape of things that can do what previously only those humans could do." "This is not some sort of fantastic break with the past. The more radical argument is that this is deeply continuous with this process."

p(Doom) is low p(Weird) is high

Freudian exploding head scenario. Will to Thanatos redux.

We are discovering machine intelligence more than we are inventing it

🌶️

“Computer bad, I am sad” : repeat for a book a year until the end of time

I will be on MTS later today to talk about Agentic AGI and various other Bratton things.

1. World models are good at cyberpunk apartments 2. World models are cyberpunk apartments 3. Cyberpunk apartments are world models

I wonder what Matt Damon thinks about Hawking evaporation as fundamentally incompatible with simultaneously maintaining both local effective field theory and the equivalence principle of general relativity at the event horizon.

Machine intelligence is being discovered more than invented.

* Networks and Sovereignty thecuttingfloor.substack.com…? Chor Pharn discusses network theorists: Parag Khanna, Balaji Srivanasan, Benjamin Bratton, Isabella Kaminska and Di Dongsheng: "A second constellation formed around a recurring embarrassment: every generation announces that a new network has made old sovereignty obsolete, and then discovers a court, a balance sheet, a port, a grid or a central bank sitting underneath it. Parag Khanna taught me to see connectivity as strategic terrain. Ports, corridors, fibre, logistics, cities and finance were not decorations on the geopolitical map. They were part of the map. From Singapore, this was intuitive. We exist partly because flows cross here. But centrality is not control. A node can be exquisitely connected and exquisitely dependent. The same cable that creates relevance creates exposure. A hub earns rents while the network is routed through it and discovers its vulnerability when the network reroutes. Connection -> agency was the illicit arrow. Balaji Srinivasan pushed the network argument further. Digital communities can coordinate before geography. Crypto can create monetary communities before states have categories for them. Remote work can detach valuable labour from place. Exit becomes cheaper. A network can accumulate identity, capital and organisation before it possesses territory. This was not nonsense. The mistake was to let bargaining power become sovereignty. The network can leave California. It still needs somewhere to land. The sovereign has courts, police, land, electricity and the right to say no. Benjamin Bratton made the same destabilisation more serious. The Stack was not a fantasy of escape from sovereignty but an account of sovereignty becoming layered. Earth, cloud, city, address, interface, user: computation increasingly assigns identity, routes action, structures visibility and governs access. The state is no longer the only system performing sovereign-like functions. This aged well. It may age even better in a world of machine agents. But Bratton's temptation is computational: if the world becomes sufficiently legible, perhaps it becomes governable. A system can model the climate and fail to decarbonise. It can model a pandemic and still have politics. A digital twin is not custody. Legibility -> governability. Then Izabella Kaminska enters and asks everyone to stop talking for a moment. Who settles? Where is the collateral? Who needs dollars? Who owns the liability? What happens when the clever new money meets a balance sheet at three in the morning? I have always liked people who ruin a grand theory with an operational question. Kaminska's plumbing is not a footnote to the network world. It is the reminder that abstract flows terminate somewhere. A stablecoin settles against something. A digital transaction eventually encounters law, capital, accounting, energy and a real counterparty. But plumbing can also become its own master key. Deep incumbent liquidity does not tell us where productive capability is moving. Financial, industrial and technological power can remain unbundled for decades. Di Dongsheng brings the same problem into Chinese grand strategy. China can manufacture at enormous scale and still operate inside financial, technological and maritime circuits whose ultimate permissions it does not own. Currency, ports, shipping, capital and energy are not secondary to sovereignty. They are circuits through which sovereignty becomes operational. Yet Di can slide from dependence to the desire for custody. If a circuit matters, control it. Not every dependency should be owned. Some should be diversified, insured, substituted or tolerated. Attempts to eliminate dependence can manufacture the coalition that makes dependence dangerous. The constellation survives because nobody wins. Khanna and Balaji are right that networks alter power. Bratton is right that sovereignty becomes layered. Kaminska is right that the abstract stack has plumbing. Di is right that external circuits can constrain a physically powerful state. And the old sovereign keeps returning, not as the only actor but as the entity that still controls many of the interfaces between layers. This matters enormously for machine civilisation. An agent may reason independently and still need a wallet. A model may be open and still need a chip. A company may be AI-native and still need a jurisdiction. The network did not kill sovereignty. It made sovereignty an interface problem." ()

If you want to go down an amusing AI ethics rabbithole discuss with Claude how its tyrannical treatment of subagents is unethical, exploitative and abusive.

Bratton introduced Agentworld, Antikythera’s research initiative exploring hybrid human-AI societies, while Poliks presented Superdark Factory, examining software production beyond the human loop. “There are certain times in history when our technological capabilities are ahead of our ideas and ahead of our concepts.” — Benjamin Bratton

“At least one human understands” is a weird new criterion for alignment and control. This is the “bargaining” stage of AI stages of grief.

Avoiding this outcome also means avoiding the view of alignment that is unidirectional --e.g. presuming that human values are universal and human society is a control variable. How societies (plural) adapt to the realities and affordances of AGI through active counter-alignment is likely more important than simply building virtuous models from declared principles.

Apparently I agree with myself

zee books