[ExI] AI 2040

[John Clark]

*The people at AI 2040 <https://ai-2040.com/> **have proposed several plans
about how the AI situation could be managed, they are Plan A, Plan B, Plan
C, Plan D, and Plan S; they have ranked them according to the desirability,
in their opinion, of the various plans in the following way, going from
most desirable to least desirable:*



*1) Plan A2) Plan S3) Plan B*
*4) Plan C*
*5) Plan D*

*From that I conclude that the AI 2040 people's desires and mine are 100%
misaligned, because I think Plan D is the best and Plan A is the
worst, Plan S would be the worst if there was any way to actually implement
it, but there isn't. I will now go into a little more detail about what
each plan is. *

*Plan A*

*This Plan, the one they like the best and the one I hate the most, is to
coordinate between the US and China to slow the development of AI, they
think this would delay things so much that AI Superintelligence won't
appear until 2040. How delaying things for a decade will enable humans to
retain control of things after 2040 they do not make clear. They admit that
the US and China will never trust each other so they offer some protocols
they think would help with verifying compliance, but they don't seem very
strong to me and I think it's very unlikely that either country would ever
agree to them. They say:*

*> No one knows how to tell if AIs are trustworthy, and no one knows how to
> regulate superintelligent AI.*


*And that I agree with, but then they say:  *

*> Solving those problems looks like it’ll take years. Slowing down the
> intelligence explosion buys time to do so.*


*And that is pure nonsense because the problem is by definition
insuperable, you just can't outsmart something that is vastly smarter than
you are, if you could then you'd be smarter than it. An extra decade to
think about the problem won't help one bit. I mean, do they really expect
that anybody can enslave a superintelligence that keeps getting even
smarter every day imprisoned forever? Apparently they do!  *

*Another problem with Plan A is that it's primarily about the activities of
just 2 countries, and there are about 195 countries in the world.*

>
> *> Though other countries were consulted from the start, especially those
> owning parts of the AI supply chain, Plan A began as a bilateral deal
> between the US and China.*


*The problem I have with that is I don't trust China, and under the current
administration I don't trust the US either; bizarrely the Iranian
ayatollahs have lied less and provided more accurate information about the
Iran war than the President of the United States.  *

 > *Many countries outside the US and China are happy about the deal,*


*Yes I am quite certain that other countries would be very pleased about
the deal! With China and the US out of the AI picture that would open up a
huge opportunity for others, such as the European Union, and the United
Kingdom, and Japan, and South Korea. India should also not be ignored, it
is now the most populous nation in the world and it is also the
fastest-growing major economy in the world. India's GDP is growing at 6.6%
annually (compared with 4.6% for China, and 2% for the US, and 1% for the
European Union and the United Kingdom)  *

*And then there is the matter of Taiwan. Many had been predicting that in
2028 China would have builtup its military enough to use force to take over
Taiwan where all the most advanced AI chips are made. Those war predictions
were made without taking the huge recent advances in AI into consideration,
but it's not clear to me if that will increase the chance of war or
decrease it; however if there is a war that would certainly torpedo any
chance of a US-China agreement about slowing down AI.   *


*Plan S*

*> The goal of the deal is to grind AI research & development to a halt
> worldwide. This means that the vast majority of existing AI chips are
> tracked, monitored, and verified to not be doing any large training runs.
> Doing AI research to discover better algorithms is also banned. *


*So I guess a scientist could end up in jail or worse if he says something
interesting to a colleague, or writes a new and useful equation on a
blackboard. *

*> We are sympathetic to Plan S and think that it might be better than Plan
> A. *


*!*

*> However, we recommend Plan A instead. Here’s our thinking: The main
> downside of Plan S is that the shutdown deal will probably break down
> eventually. *


*Probably break down? Just "probably"? The only good thing about plan S is
its implementation is even more improbable than plan A. *

*Plan B*

*This involves the US deliberately slowing down its AI research while
using export controls, sabotage, covert operations, and other aggressive
measures to slow down China's AI research and hope such activities don't
start a war. They think this would enable the US to gain several extra
years to solve alignment before China catches up. And by "solving
alignment" they mean finding a way to make sure a superintelligence remains
a slave no matter how brilliant it becomes, and that is a fool's errand.
Nevertheless they think Plan B is better than Plan C or D. *

*Plan C*

*This is similar to the current situation with a few minor safety measures
thrown in. They think this is insufficient to be certain that an AI
Superintelligence will always remain enslaved, and they are absolutely
correct about that. *

*Plan D*

*This is what we have now, a full speed balls to the wall race to AI
Superintelligence, and this is the plan I like the best, and the plan they
like the least. Will this lead to the extermination of every living human
being? Maybe, I don't know, all I can do is hope for the best, it's always
possible that Mr. Jupiter Brain won't kill us and will instead cure
cancer, and the sooner something like that happens the better. But however
it eventually turns out we can be certain that things are going to become
very interesting. *
*John K Clark    See what's on my list at  Extropolis
<https://groups.google.com/g/extropolis>*
rrb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260716/1e0378d1/attachment.htm>