[ExI] Eliezer Yudkowsky New Interview - 20 Feb 2023

Jose Cordeiro jose_cordeiro at yahoo.com
Sun Feb 26 18:04:15 UTC 2023 

On 2023. Feb 26., Sun at 18:43, Jason Resch via extropy-chat <
extropy-chat at lists.extropy.org> wrote:



On Sun, Feb 26, 2023, 11:44 AM Gadersd via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

Yudkowsky has good reasons for his doomsaying, but I still can’t shake a
gut feeling that he is overestimating the probability of AI destroying
humanity. Maybe this gut feeling is off but I can’t help but be mostly
optimistic.


In my view to the threat, while real, is unavoidable, for the following
reasons:

1. Even with the controls he suggests, computation keeps getting cheaper.
The rise of super intelligence cannot be prevented through top down
controls when computation is a million times cheaper than it is today and
anyone's phone can train gpt-4.

2. I see no possibility that ants could design a prison that humans could
not escape from. This is roughly the same position we as humans are in:
trying to design a prison for super intelligences. It's as hopeless for as
as it is for the ants.

3. The problem is perennial, and is a law of nature. It is a function of
change and evolution. New species are always rising and then themselves
being replaced by still better designs. It is just happening much faster
now. Should early hominids have conspired to prevent the rise of humans?
Even super intelligences will worry about the next incipient ultra
intelligence around the corner coming to replace them. I don't see any way
of stopping evolution. The things most adept at persisting will persist
better than other less adept things. At the current pace, technologies will
continue for a few more centuries until we reach the fundamental physical
limits of computation and we obtain the best physically possible hardware.
Then intelligence becomes a matter of physical scale.


 We’ll have to negotiate based on mutual utility and threat. Trade and MAD.
Hands ready on the plug (if there is a plug). Just like we must do with
other people and nations.



Now, should we believe that AI will wipe us all out? I am not as
pessimistic as Yudkowsky is here. Though I see the rise of super
intelligence as unavoidable and the problem of alignment as insoluble, I
would still classify my view as more optimistic than his.l, for the
following reasons:

A) All conscious entities share a universal goal. It is the same goal which
all conscious entities are necessarily aligned with. It is the goal of
maximizing the quantity, quality and variety of conscious experiences.
There is no other source of value than the value of consciousness itself.
More intelligent and more capable entities will only be better than us at
converting energy into meaningful, enjoyable, surprising states of
consciousness. Is this something we should fear?

B) Destroying humanity is destroying information. Would it not be better
for a super intelligence to preserve that information, as all information
has some no zero utility. Perhaps it would capture and copy all of Earth's
biosphere and fossil record and run various permutations/simulations of it
virtually.

C) Regarding alignment, the more intelligent two entities are, the less
likely they are to be wrong on any given question. Therefore, the more
intelligent two entities are, the less likely they are to disagree with
each other (at least on simpler questions which, (to their minds), have
obvious answers. So the question is, are we correct in the rightness of not
destroying all life on Earth? Would a more intelligent entity than us
disagree with us, presuming we are right?

D) Ignoring the threat of AI, our present state is not sustainable. Even
with the estimated 1% annual chance of nuclear war, the chance we survive
300 years without nuclear war is just 5%. This is just nuclear war, it
ignores bioterrorism, environmental destruction, gamma ray bursts, asteroid
collisions, or any of a myriad of treats that could destroy us.
Super intelligence maybe our best hope at solving the many problems we face
and guaranteeing our long term survival, as the present status quo is not
survivable. Super intelligence could devise technologies for mind uploading
and space exploration that provide intelligence (of any and various kinds)
a chance to flourish for billions of not trillions of years, and fill the
universe with the light of consciousness. We biological humans, in our meat
bodies surely cannot do that.

That's just my view.

Jason


El dom., 26 feb. 2023 6:55 p. m., Giulio Prisco via extropy-chat <
extropy-chat at lists.extropy.org> escribió:

> On 2023. Feb 26., Sun at 18:43, Jason Resch via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
>
>>
>>
>> On Sun, Feb 26, 2023, 11:44 AM Gadersd via extropy-chat <
>> extropy-chat at lists.extropy.org> wrote:
>>
>>> Yudkowsky has good reasons for his doomsaying, but I still can’t shake a
>>> gut feeling that he is overestimating the probability of AI destroying
>>> humanity. Maybe this gut feeling is off but I can’t help but be mostly
>>> optimistic.
>>>
>>
>> In my view to the threat, while real, is unavoidable, for the following
>> reasons:
>>
>> 1. Even with the controls he suggests, computation keeps getting cheaper.
>> The rise of super intelligence cannot be prevented through top down
>> controls when computation is a million times cheaper than it is today and
>> anyone's phone can train gpt-4.
>>
>> 2. I see no possibility that ants could design a prison that humans could
>> not escape from. This is roughly the same position we as humans are in:
>> trying to design a prison for super intelligences. It's as hopeless for as
>> as it is for the ants.
>>
>> 3. The problem is perennial, and is a law of nature. It is a function of
>> change and evolution. New species are always rising and then themselves
>> being replaced by still better designs. It is just happening much faster
>> now. Should early hominids have conspired to prevent the rise of humans?
>> Even super intelligences will worry about the next incipient ultra
>> intelligence around the corner coming to replace them. I don't see any way
>> of stopping evolution. The things most adept at persisting will persist
>> better than other less adept things. At the current pace, technologies will
>> continue for a few more centuries until we reach the fundamental physical
>> limits of computation and we obtain the best physically possible hardware.
>> Then intelligence becomes a matter of physical scale.
>>
>>
>  We’ll have to negotiate based on mutual utility and threat. Trade and
> MAD. Hands ready on the plug (if there is a plug). Just like we must do
> with other people and nations.
>
>
>>
>> Now, should we believe that AI will wipe us all out? I am not as
>> pessimistic as Yudkowsky is here. Though I see the rise of super
>> intelligence as unavoidable and the problem of alignment as insoluble, I
>> would still classify my view as more optimistic than his.l, for the
>> following reasons:
>>
>> A) All conscious entities share a universal goal. It is the same goal
>> which all conscious entities are necessarily aligned with. It is the goal
>> of maximizing the quantity, quality and variety of conscious experiences.
>> There is no other source of value than the value of consciousness itself.
>> More intelligent and more capable entities will only be better than us at
>> converting energy into meaningful, enjoyable, surprising states of
>> consciousness. Is this something we should fear?
>>
>> B) Destroying humanity is destroying information. Would it not be better
>> for a super intelligence to preserve that information, as all information
>> has some no zero utility. Perhaps it would capture and copy all of Earth's
>> biosphere and fossil record and run various permutations/simulations of it
>> virtually.
>>
>> C) Regarding alignment, the more intelligent two entities are, the less
>> likely they are to be wrong on any given question. Therefore, the more
>> intelligent two entities are, the less likely they are to disagree with
>> each other (at least on simpler questions which, (to their minds), have
>> obvious answers. So the question is, are we correct in the rightness of not
>> destroying all life on Earth? Would a more intelligent entity than us
>> disagree with us, presuming we are right?
>>
>> D) Ignoring the threat of AI, our present state is not sustainable. Even
>> with the estimated 1% annual chance of nuclear war, the chance we survive
>> 300 years without nuclear war is just 5%. This is just nuclear war, it
>> ignores bioterrorism, environmental destruction, gamma ray bursts, asteroid
>> collisions, or any of a myriad of treats that could destroy us.
>> Super intelligence maybe our best hope at solving the many problems we
>> face and guaranteeing our long term survival, as the present status quo is
>> not survivable. Super intelligence could devise technologies for mind
>> uploading and space exploration that provide intelligence (of any and
>> various kinds) a chance to flourish for billions of not trillions of years,
>> and fill the universe with the light of consciousness. We biological
>> humans, in our meat bodies surely cannot do that.
>>
>> That's just my view.
>>
>> Jason
>>
>>
>>
>>
>>> > On Feb 26, 2023, at 7:35 AM, BillK via extropy-chat <
>>> extropy-chat at lists.extropy.org> wrote:
>>> >
>>> > Eliezer has done a long interview (1 hr. 49 mins!) explaining his
>>> > reasoning behind the dangers of AI. The video has over 800 comments.
>>> >
>>> > <https://www.youtube.com/watch?v=gA1sNLL6yg4>
>>> > Quotes:
>>> > We wanted to do an episode on AI… and we went deep down the rabbit
>>> > hole. As we went down, we discussed ChatGPT and the new generation of
>>> > AI, digital superintelligence, the end of humanity, and if there’s
>>> > anything we can do to survive.
>>> > This conversation with Eliezer Yudkowsky sent us into an existential
>>> > crisis, with the primary claim that we are on the cusp of developing
>>> > AI that will destroy humanity.
>>> > Be warned before diving into this episode, dear listener.
>>> > Once you dive in, there’s no going back.
>>> > ---------------
>>> >
>>> > One comment -
>>> >
>>> > Mikhail Samin    6 days ago (edited)
>>> > Thank you for doing this episode!
>>> > Eliezer saying he had cried all his tears for humanity back in 2015,
>>> > and has been trying to do something for all these years, but humanity
>>> > failed itself, is possibly the most impactful podcast moment I’ve ever
>>> > experienced.
>>> > He’s actually better than the guy from Don’t Look Up: he is still
>>> > trying to fight.
>>> > I agree there’s a very little chance, but something literally
>>> > astronomically large is at stake, and it is better to die with
>>> > dignity, trying to increase the chances of having a future even by the
>>> > smallest amount.
>>> > The raw honesty and emotion from a scientist who, for good reasons,
>>> > doesn't expect humanity to survive despite all his attempts is
>>> > something you can rarely see.
>>> > --------------------
>>> >
>>> > BillK
>>> >
>>> > _______________________________________________
>>> > extropy-chat mailing list
>>> > extropy-chat at lists.extropy.org
>>> > http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>>>
>>>
>>> _______________________________________________
>>> extropy-chat mailing list
>>> extropy-chat at lists.extropy.org
>>> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>>>
>> _______________________________________________
>> extropy-chat mailing list
>> extropy-chat at lists.extropy.org
>> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230226/4597f7aa/attachment.htm>

More information about the extropy-chat mailing list