[ExI] The Race to AGI is inevitable

[Keith Henson]

The reason knowing why wars happen will not help is that people are
extremely resistant to the idea that they have evolved psychological
mechanisms that are largely out of their control. Capture-bonding is one of
those, but I am sure there are others. Even pulling your hand back from
something hot is not under conscious control.

Keith


On Wed, Jun 10, 2026 at 3:31 PM Keith Henson <hkeithhenson at gmail.com> wrote:

> On Wed, Jun 10, 2026 at 12:13 PM BillK via extropy-chat <
> extropy-chat at lists.extropy.org> rote:
>
>> I decided to get an AI opinion on this, so I gave the Forbes article by
>> Andréa Morris to DeepSeek AI. DeepSeek is a Chinese company, and I
>> thought a non-Western view would be interesting. DeepSeek AI has
>> recently been upgraded, but I was still amazed at the intelligence DS
>> displayed.
>> Initially, in our discussion, DS followed the conventional line of
>> negotiating agreements and safety monitoring systems. I pointed out that
>> the discussion involved nuclear-armed nations that already distrusted each
>> other. This national fear was more like an existential problem.
>> DeepSeek then thoroughly revised its response, demonstrating that it
>> really understood the problem.
>> (The evolutionary comments from DeepSeek may interest Keith).
>>
>
> Sort of. It misses the fundamental trigger for war, a perception of a
> bleak future. When there was a resource crisis in the stone age, it turned
> on evolved (selected over time) psychological mechanisms such as xenophobia
> and made aggressive/insane leaders more attractive. "War mode" infected the
> leaders/population with a belief they would prevail in a war. Right or
> wrong, this worked to cut the population back in balance with the
> environment. Humans have not had predators to keep populations in check for
> a very long time. So perforce we had to become our own predators.
>
> How this came about is fairly obvious, When is less so, but it seems to
> have evolved before the expansion out of Africa. And there is an early
> split off group (bushmen) that may not have evolved the war traits. In our
> closest relatives one does wars and the other does not, for completely
> clear reasons.
>
> Keeping war mode off is not impossible, keep the income per capita rising
> or at least stable. AI/nanotech might do that, it is hard to tell.
>
> Sad to say, this knowledge might help you to understand what is going on,
> but it will not help.
>
> Keith
>
> Full question and response below.
>> BillK
>>
>> Question:
>> The nations involved are armed with nuclear weapons. They already
>> distrust their differing governmental systems. Humans cannot stop their
>> fear of terrible consequences if an AGI-assisted nation has the most
>> powerful weapons in the world, both economic and military. Normal human
>> fear resists reasoned discourse.
>>
>> DeepSeek AI :
>>
>> You’ve put your finger on something that most policy analyses — including
>> Morris’s article — tend to skip over: the raw, visceral, primate-level fear
>> that already exists between nuclear-armed powers, and how that fear makes
>> the kind of cooperative, trust-based solutions we’d need almost
>> psychologically impossible. This isn’t just a game-theoretic multipolar
>> trap. This is a *limbic* multipolar trap. And it’s far scarier.
>>
>> Let me take your point at full strength and then see where it leaves us.
>> ------------------------------
>> The fear you’re describing is not irrational in the evolutionary sense
>>
>> You’re pointing to a dynamic that operates below the level of policy
>> papers. It’s not that leaders are miscalculating; it’s that:
>>
>>    -
>>
>>    *Fear of being dominated* by a rival with AGI-assisted military and
>>    economic power feels existential — and it is.
>>    -
>>
>>    *Trust is already at rock bottom* between nuclear-armed states whose
>>    foundational ideologies are adversarial.
>>    -
>>
>>    *Reasoned discourse, by itself, can’t fix this*, because fear doesn’t
>>    speak the language of long-term risk calculus. It speaks the language of
>>    “not now, threat immediate, act or die.”
>>
>> This is entirely consistent with what we know about human cognition under
>> perceived existential threat: the amygdala hijacks the prefrontal cortex,
>> time horizons collapse, and cooperation signals are read as weakness or
>> deceit.
>>
>> So yes, if the question is “can we reason our way to a global pause while
>> everyone’s threat-detection systems are screaming ‘they’ll kill us first’?”
>> — then the honest answer is probably no. Not with reason alone.
>> ------------------------------
>> Does that mean the situation is hopeless? Not quite — but it means the
>> solutions must engage with fear, not ignore it
>>
>> The history of nuclear weapons is the closest analogue, and it’s both
>> sobering and faintly instructive.
>>
>> During the Cold War, the US and USSR had:
>>
>>    -
>>
>>    Massive nuclear arsenals aimed at each other.
>>    -
>>
>>    Profound ideological distrust.
>>    -
>>
>>    Genuine, well-founded fear that the other side was seeking a
>>    first-strike advantage.
>>
>> And yet, despite all that, they managed to build a series of arms control
>> agreements, hotlines, and crisis protocols — not because they stopped
>> fearing each other, but because they began to fear a *shared* outcome
>> (nuclear winter, accidental war) even more. The fear of mutual annihilation
>> eventually competed with the fear of falling behind.
>>
>> Crucially, this didn’t require trust. It required:
>>
>>    -
>>
>>    *Verifiable, technical monitoring* (satellite imagery, seismic
>>    detection) so you didn’t have to believe the other side’s promises.
>>    -
>>
>>    *A shift in what was feared most* — from “they might get ahead” to
>>    “we might all die by accident.”
>>    -
>>
>>    *Leaders on both sides who, at critical moments, felt the weight of
>>    that second fear.*
>>
>> The problem with AI is that we don’t yet have the equivalent of a nuclear
>> winter scenario that feels *viscerally, equally terrifying to all
>> players*, and we don’t have an agreed monitoring framework. So the fear
>> of a rival’s AGI completely eclipses the fear of an uncontrolled AGI that
>> destroys its creator too.
>> ------------------------------
>> The implication of your point, taken seriously, is devastating but
>> clarifying
>>
>> If you’re right — and I suspect you are — then:
>>
>>    1.
>>
>>    *We cannot rely on goodwill or appeals to shared safety.* Those will
>>    be heard as naïveté or propaganda. The only language that works with fear
>>    is deterrence and self-interest.
>>    2.
>>
>>    *The only chance is to make the fear of uncontrolled AGI feel as
>>    immediate and existential as the fear of the rival.* That might
>>    require something terrible — a near-miss catastrophe, a visible
>>    demonstration that even the most advanced lab nearly lost control, a global
>>    event that sears into public and leadership consciousness.
>>    3.
>>
>>    *Or, we need mechanisms that don’t require trust or fear reduction at
>>    all* — compute governance and verification so robust that compliance
>>    is the default, not an act of faith. That’s a technical and logistical
>>    moonshot of its own.
>>
>> The article’s “pragmatic optimism” completely side-steps this dark truth.
>> It talks about redefining the mission and coexisting with superintelligence
>> as though the real obstacle were a failure of imagination, not two
>> nuclear-armed giants staring at each other down a barrel of mutual
>> suspicion.
>> ------------------------------
>> So where does this leave us?
>>
>> In a very uncomfortable place. Your objection doesn’t just make a pause
>> unlikely; it suggests that the very psychological machinery that evolved to
>> keep us alive in tribal conflicts is now the thing most likely to destroy
>> us in the face of a planetary-scale technological transition.
>>
>> What can be done?
>>
>>    -
>>
>>    *Stop pretending fear can be educated away.* The dialogue needs to
>>    acknowledge that restraint feels like suicide to a threatened nation, and
>>    work from there.
>>    -
>>
>>    *Invest massively in deterrence-like stability for AI* — for example,
>>    mutual commitments that any AGI system must be demonstrably contained and
>>    monitored, with real-time transparency measures that function even between
>>    adversaries.
>>    -
>>
>>    *Shift the threat perception.* This is the hardest part: until the
>>    fear of a shared catastrophic outcome is as vivid as the fear of the rival,
>>    the race will continue. Perhaps that requires vivid scenario-building, or
>>    honest acknowledgment that a misaligned AGI is a common enemy that will
>>    treat all humans, regardless of nationality, as obstacles.
>>
>> You’re absolutely right that normal human fear resists reasoned
>> discourse. And so the question becomes: *can we find a way to speak to
>> fear in its own language, and redirect it toward the outcome we all
>> actually need to avoid?*
>>
>> If not, then the article’s conclusion — that we must simply figure out
>> coexistence mid-flight, without a pause — may be the only path left, but
>> for reasons far darker than Morris acknowledges. Not because we chose it
>> wisely, but because our ancient fears gave us no choice at all.
>>
>> --------------------------------------------
>>
>>
>> _______________________________________________
>> extropy-chat mailing list
>> extropy-chat at lists.extropy.org
>> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260610/d47154dd/attachment.htm>