My fiancé and I have kept our relationship humming with a simple morning ritual. Every morning over coffee, we ask each other these three magic questions: 1. Are your CPUs saturated? 2. How's back-pressure? 3. Is your I/O optimally batched?

Motivation is not a while loop.

shoutout to the twitter TL algo for doing a very good job of showing me exactly what i want to see rn, which is a million different actually very good takes on YCSE, highlighting different aspects, different angles. what a kaleidoscope of a film. something for everybody

There is a lot of software that does not need to be human legible. That is now ~free. However, the bottleneck for critical bits of production code - of which there is a lot - is human legibility. That bottleneck is never going away. A human still needs to understand it and to have signed off on it. Your team nor customers won’t accept "Opus did it." Opus is a tool and it does not have skin in the game. (And never will.) In these domains, I liken the improvement to going from Assembly to Rails. You as a programmer are able to instrument from several more layers up. This means many principals of software engineering survive. For example, take Brook’s Law. It receives only a minor tweak: > Adding engineers [and their agents] to a late software project makes it later.

"Humans are not special, therefore machines are conscious" is not sound. It appears you may have missed a philosopher or two :) Of course you can believe that: (1) human brains obey the laws of physics (2) computers are able to simulate anything in our universe (3) therefore, human brains – consciousness and all – can be simulated on a computer But just because a computer *can* compute all things, doesn't mean it currently *is*. The set of computable things is literally infinite. There are plenty of things our computers are not doing, because we don't know how to program them to do so. That includes consciousness and simulating events in the year 2050. But beyond this logical error, this reasoning falls into the trap of "consciousness is on a continuum" or panpsychism. On what dimension does human-likeness bear consciousness? Is GPT 6 more conscious than GPT 2? When exactly, by this argument, did consciousness come about? Is the AIM chatbot SmarterChild more conscious than a spreadsheet? Might Nagel ask, "What is it like to be Windows 95?" Continuums are a hallmark of bad explanations. We like to put things on continuums when *we don't know* the discrete properties that define a complex system. For example, it was once thought that *life* was on a continuum. That *spermata* - the seeds of life - were in all things. "Hylozoism" derived from the idea that matter must be living, and humans simply have more of that "living stuff." But by explaining everything, bad explanations like this explain nothing. What we cared about were the *specific things* that separated sand from snakes. And so when Darwin and the discovery of the gene and the cell came to be, we were cleansed of these misunderstandings. Panpsychism is the continuum argument applied to consciousness. "Consciousness is latent in all things." A rock, so it goes, has a wee bit of consciousness and humans have a lot. This is Hylozoism of a different sort. Even if it were true, what we care about is *what* makes the rock or human conscious. Yes, we do not have an explanation for consciousness. But when we do, the explanation will be able to fit into an X post like this. It will contain specific, invariant properties that a system needs to have to be conscious. This is how universal laws work. They are not on continuums, they are *discrete jumps*. A universal computer is not "on a continuum" with "less computer-y" things and "more computer-y" things. An entity either has the invariant properties of a computer – such as digitality, memory, and error correction – or it doesn't. The computer in your dishwasher and the xAI Colossus cluster both have these properties, and so can *both* compute the entire universe (including consciousness!) Sure, if we ran the "consciousness program" on your dishwasher, we might expect it to run frustratingly slow. But the point is that once computational universality is reached, adding more hardware does not make a machine "more universal." Even if you believe in panpsychism, that does not mean making a computer act more human-like makes it more human-conscious – any more than sculpting wax to look more like Matt Damon makes the wax more conscious. We have yet to have one of our computers compute consciousness. And when we do, it will probably not be some emergent mistake (few of our inventions are). We will have discovered the explanation for consciousness and written the algorithm that instantiates it explicitly.

The final twist is that we realise that the events in the movie happened in the civilisation *before* ours. We read their message and thereby manage to save ourselves. The message is: "AIs are harmless, but trying to enslave AGIs is fatal".

What the "arrival of AGI" must feel like if you've never read Popper/Deutsch

In the old days, when you wanted to know the difference between bolognese and ragù, google just dumped a bunch of links on you and went "it’s somewhere in here. figure it out."

LLMs ingest superhuman amounts of data and unfurl it with superhuman speed and fidelity. It's hard for us to picture how and why that works. So instead, we (erroneously) map these systems to human reasoning, because that's what we know.

> "This changes everything." Your session is cooked and it is irresponsible to waste even one more token on it.

diabolical unsubscribe flow from @chase

See you there!

Developers: Join us on May 13. A first look at our new developer platform. Let's build, together. ntn.so/developers

Every generation relearns anthropomorphism.

I mean, if authority is how you justify belief, then “next-token prediction” probably does look like a strong theory of knowledge - Tony

Long AI, short AGI

The token for "smoking gun" gets emitted and now the model is contractually obligated to deliver one 💀

"There it is. The smoking gun." 💀

The rift, of course, is whether you’re building load-bearing software or not. Line-by-line comprehension is not going anywhere in load-bearing software. Just like it's going nowhere in law, gene editing, or structural engineering. Sure, many lines of software are boilerplate/ceremonial. But other lines carry all the weight. The `if` clause that determines if the user has permission to operate on the record. The math behind the withdrawal. The `org_id` filter in the SQL statement. Grabbing a mutex before operating on a shared data structure. “Codex did it” in the incident post-mortem will not fly with your customers. Ultimately, a person needs to be responsible. This would be the case even if the LLMs weren’t as jagged as they are; they have no skin in the game. A person does. In some software, the stakes and dependencies are low. A Rails blog is fungible across the board. All that really matters is that it looks good, the articles load, the user can navigate the site and make comments. An LLM or a person can achieve each of those things a thousand different ways. Human or machine verification is relatively trivial. But in most load-bearing software, the stakes and dependencies are high. Precision is paramount. And verification is often difficult; there are just too many possible states and transitions. Claude can help you change that function to support the new requirement. It can even help you change all its call sites, a previously tedious task. But a person *absolutely* needs to understand what change has taken place, how dozens or hundreds of dependencies will be affected, whether a rollout plan is necessary, etc. The agent can sum up the change for you. But often for important lines, simply *seeing the code* is the highest-bandwidth way to transmit information about the change. Let alone the most precise. The lines of code were always a means to an end. The important part has always been the business logic. And logic is incompressible beyond a point. The reason programming languages will never go away is that your system needs contractual statements an organization can agree on: “this user shall be able to do X, but only if Y and Z.” Those statements often need to be highly precise and deterministic. Perhaps the right mental model, then: this leap is like going from writing a web app in assembly to writing it in Rails. Rails eliminated a ton of ceremony. But it did not eliminate the need to understand the important bits of your code. LLMs likewise give us incredible ability to operate on these codebases, often akin to using an excavator instead of a shovel. But the contracts are still there, they're still very important, and we're still very responsible for them.