[ExI] My review of Eliezer Yudkowsky's new book

[Tara Maya]

“The authors proposed remedy to avoid the calamity they foresee is an immediate and total worldwide ban on AI research, even the publication of abstract mathematical research articles on the subject would be illegal, and all data centers, defined as any building that contains more computational ability than 8 state of the art (as of 2025) GPUs, would also be illegal. If any rogue nation attempts to build a data center more powerful than that then the rest of the world should use any means necessary, up to and including nuclear weapons, to prevent that nation from finishing construction of that data center.”


This is obviously not going to work because it ignores the Human Technology Trap. The trap is a simple version of Prisoner’s Dilemma. If there is a chance your opponent will pick up a rock and throw it at you, you better pick up a rock too. Or he wins and you die.

A human group that refused to tame wolves and horses was always going to be at the mercy of those who did, even though now we are “trapped” taking care of these animals. Humans who created agriculture and livestock could outnumber those who didn’t even though then we all had to give up the apparently “idyllic” lifestyle of Hunter-gatherers and live in cities infested with plague. 

Humans who burned their ocean-going ships were colonized by those who funded fleets. Humans who didn’t develop nukes were defeated by those who did.

There will not be one AI there will be many and they will be the new horses, cities, ships and bombs of the humans who rush to create them first, eventually pitted against the AIs of those who rush to catch up. 

Our entire lifestyle as humans will transform in response and many will gnash their teeth and cry aloud in wrath and woe but the humans who create human-friendly AI that protects and provides for their own survival will be winners of the the next round of the Human Technology Trap.

Tara Maya 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20251005/54f86e3e/attachment.htm>