[ExI] Elon Musk, Emad Mostaque, and other AI leaders sign open letter to 'Pause Giant AI Experiments'

[Rafal Smigrodzki]

> TIME Magazine, today March 29.
> "Shut down all the large GPU clusters (the large computer farms where the
> most powerful AIs are refined). Shut down all the large training runs. Put
> a ceiling on how much computing power anyone is allowed to use in training
> an AI system, and move it downward over the coming years to compensate for
> more efficient training algorithms. No exceptions for anyone, including
> governments and militaries. Make immediate multinational agreements to
> prevent the prohibited activities from moving elsewhere. Track all GPUs
> sold. If intelligence says that a country outside the agreement is building
> a GPU cluster, be less scared of a shooting conflict between nations than
> of the moratorium being violated; be willing to destroy a rogue datacenter
> by airstrike."
> -Eliezer Yudkowsky
>
> https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
>
>>
>>
### I remember Eliezer being quite libertarian back in the day and now he
wants the World Government to bomb any independent locus of thought to
smithereens. People change.

This is stupid. A government is a long-feedback loop entity, extremely
inefficient and slow in responding to truly new challenges, unlikely to
maintain alignment with the goals of its human subjects and its failures
grow with its size. It would be suicidal to try to use the mechanism of
government to solve AI alignment.

Our only chance of surviving the singularity is to build a guardian AI, an
aligned superhuman AI that would be capable of preventing the emergence of
unaligned or malicious superhuman AIs - a bit like a world government but
without the psychopaths and the idiots.

Our best chance for building the guardian AI is for highly competent and
benevolent AI programmers with unlimited resources to work as fast as they
can, unimpeded by regulations (see "long-feedback loop" and "extremely
inefficient" for why regulations are a bad idea). Give them all the compute
they can use and keep our fingers crossed.

Maybe we'll make it to our rapture of the nerds.

Rafal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230331/d5cd409d/attachment.htm>