Gabe had the initial idea and provided an outline, Adam fleshed out the outline, found more concrete examples, drew the graphs, and restructured the outline to make it flow better, with regular feedback from Gabe.
Why has technology been such a destabilising force in the last few decades?
It feels like new technologies keep appearing, and keep causing massive societal problems: social media and the whole attention economy wrecked our brains, industrially engineered junk food massively increased obesity rates, and AI seems bound to lead to gradual disempowerment or full human extinction.
The models of technology I see floating around are unable to answer this question. On one hand, technological optimists claim that more technology is going to make things better, completely ignoring the clear negative effects of many technologies. On the other hand, people focused on the damage caused by technology on public health, the environment, and culture often frame technology as an inherently perverting invention, forgetting all the benefits on which their existence depends.
My answer is that society has lost its ability to consistently absorb technological shocks.
Every new technological discovery expands the range of what we can do through its many applications. Unfortunately, destructive applications are easier to find than constructive ones. This has nothing to do with some moral failing inherent in technology, and everything to do with the physical reality that it is much easier to destroy than to build.
This means that when a new technology is introduced, it induces a shock, a temporary period where we don’t yet know how to handle all the negative applications of that technology. It is through the work of our institutions (laws, cultural norms, international treaties…) that these shocks have historically been absorbed and channeled toward beneficial results.
What has changed over the last few decades is that society has become less and less able to absorb the successive shocks of new technology. In part this is because of the acceleration of technological development, and in part because the very institutions that historically absorbed technological shocks have been weakening and ossifying.
In this post, I explain this model in detail, from the initial destructiveness of technologies to the shock absorption capabilities of institutions, and what it means to live in a world unable to handle the shockwaves of technology.
I’ll conclude by hinting at how this model applies to AI development, which is the basis for my belief that if AI development continues as it has done, the default result is human extinction. I will write more about how this model connects with extinction from AI in a follow-up article.
Technological shocks
Technological progress has radically accelerated in the last century, and technology is becoming more impressive by the day. We can now make a round trip around the world in a few days, we have eradicated major diseases like smallpox (to the extent that its Wikipedia page is written in the past tense), diseases like the plague that destroyed whole populations are virtually extinct, and the poorest people on earth are able to send whole libraries of data across the world in seconds, when even kings and emperors in the past had to wait weeks or months to send messages across continents.
This can be put into numbers. If we plot measures of technological progress like the time it takes to travel around the world, the time it takes to send a book’s worth of information from Paris to Shanghai, the height of the tallest building, or how many calories can be acquired from the average daily wage, the graph always has one of two shapes:
For most of history, the measure did not increase in any relevant way. Then, around 1850, it zooms up. After this inflection point, the metric either saturates at the physical limit, like how quickly information can travel, or it keeps growing faster and faster, like how much information we can pack in our pockets.
As this explosion of technological progress happens, so does the range of what we can affect. We can now act on the very small: we read an entire human genome for a few hundred dollars, edit genes with CRISPR, and arrange individual atoms one by one. We can act on the very large: a single bomb can erase a city, and our probes have left the solar system. We can sense the very far, measuring the composition of the atmosphere of planets hundreds of light-years away. We can act very fast: within days of a new virus’s genome being published, we can design a vaccine against it, or engineer a more dangerous version of it. And we can harness the very strange, building computers out of quantum superposition and cooling matter to a billionth of a degree above absolute zero.
—
Sadly, out of these new possibilities, the destructive ones are easier to find and exploit. This is not because humanity or technology is evil, but simply a consequence of the laws of physics. Thermodynamics favors disorder over order: it takes far less energy to break an egg, a glass, a building, than to build them, keep them intact, or fix them. It is written in the laws of the universe that destroying is easier than maintaining or building.
This is why it is much easier to poison someone than to cure them, to wreck the environment than sustainably engineer it, and to spread parasitic destructive ideologies rather than deep meaningful ideas.
As a result of this, technological progress comes with a shockwave of potential destructive applications.
How institutions absorb technological shocks
New technology creates an initial destructive shock. However, the net effect of technology is rarely negative, and never as negative as its initial shock.
For example, we can look at nuclear technology. The initial shock happened when nuclear weapons were invented during World War 2, and used on Hiroshima and Nagasaki. It fits with the model above that the first real application of nuclear technology was weapons instead of energy: civilian electricity only appeared a decade later as a continuation of the same research programs that led to the bombs. Indeed, there are many more ways to destroy using a nuclear chain reaction than to produce safe energy.
Still, in the 80 years since, there has been no worldwide nuclear conflict, or even use of nuclear weapons in war. Nuclear energy has also been channeled for civil usages, with lower accident and mortality rates per unit of electricity produced than most other methods.
—
Why is it that for most dangerous technologies, including nuclear technology, we have mostly managed to handle the initial shock?
My model is fairly simple: a new technology starts by percolating into society, which induces a shock. After this shock is absorbed, society reaches a new baseline. For most technologies, the new baseline for society is better than before: after the dust settles down, the impact of technology is usually positive.
I usually illustrate it with the following graph:
This shock absorption is not automatic: it takes a lot of work from various institutions. We would have been in a very different world if right off the bat, every country and every individual had access to nuclear weapons. Instead, many institutions across the world worked together to ensure a much more positive result:
The U.S. quickly worked to avoid nuclear proliferation, both to individuals and to other countries
Once some amount of proliferation happened (notably to the USSR), the U.S. pushed repeatedly for international agreements to limit proliferation, and took actions to prevent new countries acquiring nuclear weapons1. The result, the NPT, leverages civil nuclear as a reward for committing to not develop nuclear weapons
An international organisation, the IAEA, was created and empowered to monitor and verify nuclear industry across signatories of the NPT, and check if nuclear material is diverted for weapons
Even between states with nuclear weapons, detente agreements were pursued to reduce the number of nuclear warheads and their strength
Technology was developed to detect nuclear testing anywhere in the world, making it much harder to redevelop the technology in secret
The point is that a lot of bad effects of technology tend to come from their initial shock rather than from the nature of the technology.
For instance, Social Media has a lot of negative impacts on society right now. But I believe that this is mostly because we have not yet regulated them, and because we have not developed healthy cultural norms about their proper use.
This could look like society as a whole putting moral demands on influencers and social media companies, similar to the code of conduct of journalists and the regulations on what can be broadcasted in traditional media. The dangers of social media for children would be understood to an extent that most parents would put reasonable limits on them, and society would assist them in doing so with the right tools and regulations. Beyond that, the content itself would be managed not to maximise attention or profit, but to strike a healthy balance: between entertainment and information, between preventing propaganda and fostering cultural exchange, and between low-brow and high-brow content.
After some experimentation and a lot of effort, social media would eventually become a net positive: connecting the entire world ought to lead to more good things than negative ones.
—
The model above is about what happens with a single technology. But in the real world, multiple technologies often appear together or one after the other. Their shocks then stack with each other, leading to more complex effects.
This leads to one of two outcomes, which I will call here the Progress World and the Decline World.
In the Progress World, there’s enough capacity to deal with technologies as they’re being developed, either because our institutions are great at absorbing technology, or because technology is developed slowly enough. And thus, society smoothly improves.
In the Decline World, technologies are not absorbed quickly enough. They percolate into all of society, a technological revolution happens every month, our institutions are too slow at dealing with them. Regardless of the reason, the shock introduced by each subsequent piece of technology happens before we’re finished dealing with the previous one. As a result, society progressively deteriorates.
These two possibilities are illustrated by the following graphs:
What decides whether we end up in a Progress World or a Decline World is how much technological shock our institutions can absorb at any given time, our institutional capacity. Concretely, this includes:
The ability of governments to quickly pass regulations to handle the negative consequences of the technology (such as the Atomic Energy Act in the U.S., passed only a year after Hiroshima and Nagasaki)
The ability of civil society to educate the general public about the new technology, its dangers and promises (such as the educational campaigns about the ozone hole which helped create public pressure for international action)
The existing process ensuring safety before a new technology is released (such as drug trials for medicines)
This capacity should not be taken for granted: the crumbling of our institutions and constant political infighting have made it much harder recently to pass any law or to educate the general public. Still, as long as there is capacity left, shocks are absorbed smoothly and we get the benefits with little lasting harm. But if a single shock is too big for our institutions to absorb, or if new shocks pile up before the previous ones are dealt with, the unabsorbed damage accumulates and spreads, and society deteriorates.
This deterioration could take many forms. It could look like laws taking years to pass, so that they were already made obsolete by technological change before being voted on. It could look like people barely five years apart being unable to understand each other and communicate, because their technological experiences are so drastically different. And it could look like parents and teachers completely unable to educate children on how to engage with new technologies, because these did not exist even a year or two ago, let alone when the parents and teachers were kids themselves.
How to deal with a declining world
I think that pretty clearly, up until the middle of the 20th century, we were in a Progress World. Sadly, I think since the 2010s, we are now in a Decline World.
There has been a hard-to-pinpoint turning point. Something to do with the successive shocks of television, the computers, the internet, social media all compounding onto each other. But I think that dating it to a single event or a single date, like the early 1970s that are often seen as the start of the “Great Stagnation”, would be wrong.
—
To understand what it means to live in a Decline World concretely, let’s imagine what would happen if nuclear chain reactions were discovered in 2026 instead of in 1945. I think we would be in a much worse situation:
There are far more sovereign states with the manufacturing and scientific capacity to turn the technology into weapons quickly: the UN went from 51 member states in 1945 to 193 today, many of them with modern industry and universities
It is much easier to communicate a lot of information across the world in a way that cannot be caught, now that we have the internet and end-to-end encrypted messaging accessible to anyone. In the past, spies on early nuclear programs like Klaus Fuchs had to meet representatives of foreign intelligence agencies in person, and share written notes that were easy to intercept.
It is much easier to send concrete artefacts through trade-links (think plane, containers, trucks): the A. Q. Khan network already used commercial shipping to smuggle nuclear centrifuge parts to Libya, Iran and North Korea in the 1970s
Spying on a foreign country requires far fewer resources because of the transition from training and deploying actual spies on the ground to cyber intelligence operations run from a laptop: the Stuxnet computer worm showed that even a heavily guarded nuclear facility could be reached remotely
As a result, I expect that nuclear technology would spread to many more countries in a matter of days or weeks, and most of these countries would manage to turn the technology into weapons in a few months. In this context, the Mutually Assured Destruction equilibrium would be much harder to maintain, and a Non Proliferation Treaty would have much less tooth.
How long does it take until nukes are used tactically, and then strategically, with worldwide escalations? I can’t know for sure, but my guess is that the result would be much more violent and drastic than in the Progress World of 1945, when the US detonated two strategic nuclear bombs on Hiroshima and Nagasaki, and then no nukes were used ever again.
—
I believe this is the hard problem that humanity is facing right now.
How do we deal with the fact that we moved from a Progress World, where we naturally incorporated technological progress into society for a net positive result, to a Decline World where technological developments look more destabilising by the day.
We are not even able to absorb the shock of technologies that should almost be beneficial by default. I think the best example might be lifespan improvements. It should be clearly good to make people live longer with a decent quality of life. There should not even be a “shock” phase for technologies of this type. They ought to immediately improve society.
And yet, we are now suffering through what is sometimes known as “the second demographic transition”. Many countries are now having sub-replacement fertility rates, as well as an extremely ageing population. As a result, these countries are facing a strong imbalance between the growing economic needs of their older population and their relatively shrinking young population.
France is a striking example. Its pensions are not funded by savings invested over each worker’s career, but paid directly out of the contributions of people currently working, which creates a growing crisis as more and more people retire and fewer join the workforce. The ratio of workers paying for each retiree fell from 4 in the 1970s to about 1.7 now. Every extra year of life expectancy, one of the greatest achievements of modern medicine, now shows up as a bill that a shrinking number of young workers must pay.
I want to emphasise how unnatural this is. As a society, we have handled nuclear weapons well enough to extract nuclear power plants without WW3, but we cannot make it sustainably good to have people live longer lives. To me, this feels extremely bad: we have a big problem with technology.
—
While we are struggling with technologies that increase life expectancy, people are working on technologies that are so far beyond our capacity to absorb. For example, a growing number of startups are working on reading people’s thoughts, including one funded by a former OpenAI employee.
The magnitude of the shock that this technology would create is not something that society has the capacity to absorb:
Authoritarian governments would gain a powerful new tool for propaganda, control, and repression, the technological means for a perfect Thought Police, more dystopian than Orwell’s 1984
Companies that already optimise to capture attention could reach directly into people’s thoughts to keep them even more engaged
The people developing these technologies do not plan to address these problems. Many techno-optimists seem to assume that we still live in the Progress World of the 1950s, where governments have enough bandwidth to absorb the initial shocks caused by new technologies. More cynically, many simply do not care, as long as it is exciting to develop and lucrative. “Après moi, le déluge”2
If thought-reading technology is developed, it would take considerable legislative, governmental, cultural and international effort to prevent it from being used for authoritarian purposes or to maximise people’s screen time at the expense of their sleep and wellbeing.
Sadly, we don’t have that much legislative and executive spare capacity to go around. On the contrary, we are experiencing a deep institutional crisis, where governments can’t address the numerous problems of a society that has become incredibly complex.
Conclusion: Where This Leaves Us With AI
I expect AI development to lead by default to human extinction.
Part of my reasoning comes from applying this model of technological shock to AI. AI exacerbates the magnitude and speed of the shockwave by accelerating the creation of new shocks, compressing decades and centuries of technological progress into years.
Even in a Progress World, it would take an extraordinary effort to handle it. But as it is, our Decline World and its weakening institutions have little hope to absorb the tsunami of shocks caused by AI: at some point, one of these shocks will be too large to recover from. This could be a new technology that enables anyone to produce bioweapons at home for a few dollars (or any technology whose initial shock is so destructive that it ends humanity), the slow disempowerment of humanity as AIs replace us in every decision that matters, or Runaway Machine Intelligence that simply overpowers us.
On this, cheers!
This includes pressuring allies such as Taiwan and South Korea. The U.S. also secretly airlifted about 600 kg of weapons-grade uranium out of Kazakhstan, and bought 500 tonnes of Russian weapons-grade uranium to convert it into civilian reactor fuel.
(“After me, the flood”), a phrase attributed to Louis XV that expresses indifference to the consequences of one’s actions after one is gone.