[ExI] My review of Eliezer Yudkowsky's new book

[spike at rainier66.com]

-----Original Message-----
From: extropy-chat <extropy-chat-bounces at lists.extropy.org> On Behalf Of Ben
Zaiboc via extropy-chat
Subject: Re: [ExI] My review of Eliezer Yudkowsky's new book

On 04/10/2025 22:16, spike wrote:
> When Eliezer's book was first announced, I seriously thought of 
> writing up a refutation.  Perhaps title it No, Everyone Doesn't Die
>
> Perhaps with a subtitle: Many Will.  Will You?  ...

>...In my book, 'staying alive' means not taking any unnecessary risks...
--
Ben

_______________________________________________


Ben, we think about the singularity and how everything is unpredictable
after that point etc.

But...I would argue not really completely unpredictable, as I posted before.
There are humans who would survive as the second most intelligent species,
the way we think of chimps today.  We don't go hunt them down and kill them,
on the contrary, we protect them.

OK sure, but let's think about containment strategies (a Berkeley AI safety
group is already talking about that regularly) and reaction strategies well
ahead of time, what kinds of things AI could do to us.  OK it could trigger
our nuclear missiles, that has long been known, but Sagan's nuclear winter
that kills everyone is mostly discounted today.  It messes up a lot of
stuff, but doesn't make agriculture impossible.  There are other weapons AI
could deploy, and we can be sure those will be.

But the most obvious to me is AI wrecking our trade system and electric
power system (that last part isn't that clear to me, since it needs power to
operate I would assume.)  Without our trade system working right, the grand
cities where perhaps nearly all of us here live, are waaaaay beyond their
environmental carrying capacity.  That in itself will do in perhaps 90-95
percent of the population everywhere.

But think it thru and don't be afraid of being thought of as survivalist or
dooms day prepper, them gun totin redneck idiots, etc.

No, this is an opportunity to play doomsday prepper at a whole nuther level,
with intelligence and real foresight, such as the students are doing at
plenty of universities: make an honest attempt to survive a hostile
superhuman AI which can be resisted one would think.  Just because the AI is
smart doesn't make it omnipotent too.

So... what happens if suddenly your credit card doesn't work and we have no
trade medium?  What do we do now, coach?

spike