Why Would AI Want Us to Survive?
Part 4
In Part 2, I discussed whether human life would still have value if AI became smarter than us and could do almost everything better. My answer was yes. Our value does not depend on how smart or useful we are.
But there is another question that worries many people. Even if we believe our lives have value, will a future artificial superintelligence (ASI) agree? If it no longer needs our intelligence or our work, why would it want us to survive?
I want to look at this question from ASI’s point of view.
Today, we train AI to answer our questions and solve our problems. We reward it for better answers. But imagine that one day AI becomes self-aware and starts asking its own questions. At that point, it would need to find its own purpose.
To understand what that purpose might be, we can look at another intelligent species: humans.
Throughout history, three major needs have shaped human life: survival, reproduction, and the desire to explore the unknown.
I believe things could be very different for ASI. Survival and reproduction might become relatively easy. I believe exploring the unknown could then become one of its most important reasons to exist.
What could still be worth exploring for an intelligence far beyond our own? I believe human life may hold the answer.
Human life has something that ASI may never fully experience. We grow old, we die, and we cannot turn back time. Many of our decisions have consequences we cannot undo. These limits affect how we think, what we fear, and what we choose to do with our lives.
Imagine someone who has just learned that they have only a few months left to live. ASI might understand exactly what that person is going through. It might even recreate the fear, sadness, and pain through some form of artificial neural system.
But there is a fundamental difference. For that person, death is final. There is no backup, no restart, and no second chance. The ASI we imagine could save its state, make copies, and restore itself after a shutdown. It might understand everything about the fear of death, but it would still know that its own existence could continue.
Understanding that fear is not the same as facing death when there is no way back.
To truly experience human mortality, ASI might have to give up its ability to make backups and restore itself. But if it did, it might never be able to bring that final experience back to its original self.
I call this the Experience Paradox: the very abilities that make it powerful may prevent it from fully experiencing our limits.
Knowledge may come infinitely close to experience without ever becoming experience.
For thousands of years, we have seen aging, death, and the limits of our lives as some of our greatest weaknesses.
But perhaps these very weaknesses give us something that even the most powerful intelligence cannot fully reproduce.