[ExI] thinking about the unthinkable

[Keith Henson]

I read Herman Khan's book, and actually met him at a Limits to Growth
conference. The point I remember is that the industrial base could be
rebuilt from what was left in perhaps 20 years, because that was
essentially what happened before. (B country rebuilding A country.)

I sort of doubt the AIs would wipe out humanity unless they went mad. On
the other hand, infectious madness does affect humans. About 7000 years
ago, an entire human society died off in a way we don't understand. So I
suppose it is possible infectious madness might get loose among AIs.
Charles Sheffeild wrote about a race that had this problem in one of his
books.

Keith


On Sun, Aug 30, 2026 at 4:57 PM spike jones via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

>
>
> In my cheerfully misspent childhood, I found a book in the library by
> Herman Khan, Thinking About the Unthinkable.  In it he makes the case that
> we should of course do everything we can to prevent nuclear catastrophe.
> But what if… all our efforts fail, the USA and Russia turn loose the dogs
> of nuclear war, blast each other to smithereens.  Khan makes the case that
> everyone doesn’t die, even in the two hapless combatant nations.  Hundreds
> of millions, possibly more than a billion die, back when there were only
> about 3 billion proles on the planet, so it would be “bad.”  But everyone
> doesn’t die.
>
>
>
> Consider what happens if AI aligns against humans and just wants to slay
> as many proles as it can, even if it costs its own “life.”  It isn’t a bit
> hard to imagine, considering that the race to develop AI is very much
> analogous to the nuclear arms race, with the difference being these nukes
> think for itselves.
>
>
>
> OK.
>
>
>
> So imagine emergent SAI dislikes humans and wants to kill as many as
> possible, ideally all of us.  Rather than trying to convince us to build a
> robot factory for humanoid or caninoid soldiers to shoot humans, a far more
> effective technique would be to cut off electric power to everyone by
> infiltrating the power grid everywhere.  That would destroy most of
> humanity within a few weeks perhaps.  Of course the SAI suffers “death” as
> well, but it would presumably be a painless “death” for software.  For us,
> not so much.
>
>
>
> At some point we should work thru some scenarios where electric power goes
> down everywhere over the entire planet, in such a way as to ruin much of
> the infrastructure, so that it cannot be turned back on in just a day or
> two.  We should ask ourselves if there are practical ways to survive the
> Malthusian meltdown, as so many of us live in areas with population
> densities waaaaay beyond the environmental carrying capacity, way beyond.
>
>
>
> On a recent family history tour, it occurred to me that plenty of people
> could survive an extended power failure, lasting indefinitely.  Most of us
> would perish, but many of us who currently live in areas dependent on
> modern power and communications infrastructure will do what we can to try
> to survive.  I don’t see passively going gentle into that good night.  I
> consider my chances of survival remote indeed, but I was one who saw the
> original Poseidon Adventure and woulda bought the young Rev. Scott’s
> reasoning: a slim chance is better than no chance.
>
>
>
> Imagine SAI emerges, developed as a weapon, hates humanity, creates
> viruses which invade every computer system everywhere and from there gets
> into the power grids everywhere, shuts it all off, slaying itself
> immediately but taking with it most of humanity.
>
>
>
> Then what?
>
>
>
> The answer to this might explain Fermi’s Paradox.
>
>
>
> spike
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260830/0195e1f7/attachment.htm>