[ExI] The 'Don't Look Up' Thinking That Could Doom Us With AI

[BillK]

Max Tegmark  April 25, 2023
Tegmark is a professor doing AI research at the Massachusetts
Institute of Technology

<https://time.com/6273743/thinking-that-could-doom-us-with-ai/>

Quotes:
Sadly, I now feel that we’re living the movie “Don’t look up” for
another existential threat: unaligned superintelligence. We may soon
have to share our planet with more intelligent “minds” that care less
about us than we cared about mammoths.

If you’d summarize the conventional past wisdom on how to avoid an
intelligence explosion in a “Don’t-do-list” for powerful AI, it might
start like this:

☐ Don’t teach it to code: this facilitates recursive self-improvement
☐ Don’t connect it to the internet: let it learn only the minimum
needed to help us, not how to manipulate us or gain power
☐ Don’t give it a public API: prevent nefarious actors from using it
within their code
☐ Don’t start an arms race: this incentivizes everyone to prioritize
development speed over safety

Industry has collectively proven itself incapable to self-regulate, by
violating all of these rules.
--------------------

Unfortunately, the AI race won't slow down. The fear that others might
speed on and win the race to AI domination will keep up the pressure.
The order of Admiral Farragut at the Civil War Battle of Mobile Bay
“Damn the torpedoes, full speed ahead,” could lead to disaster in the
AI race.

BillK