[ExI] Wired article on AI risk

[Kelly Anderson]

On Wed, May 30, 2012 at 2:40 PM, Stefano Vaj <stefano.vaj at gmail.com> wrote:
> On 30 May 2012 01:18, Kelly Anderson <kellycoinguy at gmail.com> wrote:
>>
>> Let's not get into the quantum processors, that will just make my
>> brain hurt. Of course, it's the case because quantum computers exist
>> only in theory.
>
> Seth Lloyd would argue that the universe would be a quantum computer...

Last I checked, we weren't building universes...

>> But it is not true that they can all run the same programs because
>> some programs require more memory.
>
> Indeed. One important qualification of computational equivalence is that
> memory is unlimited (but floppy disks have been cheap for a long time...)
> and that time is irrelevant.

It's a pretty big deal in practice of course. If you ran a perfect
brain simulation on a very slow computer (compared to native brain
running speeds), I wonder if it really would experience consciousness
due to the time lags... It's an interesting thing to ponder. You
certainly couldn't feed sensory information into it at real time
speeds.

>> Also, if the requirement of the
>> program is that it run in real time (or the plane will crash, for
>> example) then this fact is only helpful in theory, not in practice.
>
> Sure. Speaking of "intelligence", however, human beings on a
> relativistic-speed starship would still be considered intelligent even by
> those living at a relately much faster pace, wouldn't they?

Not functionally... Are you intelligent if I can't observe you
thinking, or communicate with you in any way? I mean yes, they are
intelligent, but it's not a functional kind of intelligence in our
time line.

>> Ok. I actually get that. Sure, absolutely right. It is difficult
>> emotionally to give up free will. Maybe this will come to make the
>> sting of losing free will less sharp.
>
> We maintain however a "kind of" free will, because our reactions cannot be
> calculated unless one runs the humain being concerned through all the
> necessary steps. :-)

I've been wondering about the term "nondeterministic" today... In
computer science, a nondeterministic algorithm is an algorithm that
can exhibit different behaviors on different runs. I think that for
all practical purposes, the human brain is nondeterministic. The mere
fact that the program has "run" once, changes the brain. Like, would I
marry my first wife again? Absolutely not. That feels like "free
will", and certainly we work under the illusion of having free will,
and no, we don't have the ability to determine what the brain will do
without running the program, but, all that being said, it's just
processing. Free will is this kind of magical thing... that in some
sense probably doesn't really exist. It's chaos for sure.

>> We share DNA as a species, and my Darwin-Dawkins-given right to
>> selfishness for my genes extends, in some measure, to all humans.
>
> Mmhhh. In fact, a species is a competitive pool, since I have no direct
> sexual competitors beyond its boundaries. But my genes do not care much if
> my species survives if they do not.

So, you are telling me that if a group of Gigantopithecus comes into
the hunting area of two competing groups of Homo Erectus, that the
Homo Erectus groups would not temporarily put aside their differences
to eliminate the Gigantopithecus group? Come on, you can't really
believe that. It's the competition for the niche that is the key here.
I'm assuming that Gigantopithecus and Erectus would use more or less
the same resources (food, water, etc.) and thus would be a competitor
for the niche.

If you don't buy that, then try to imagine one group of the Erectus
making some kind of pact with the Gigantopithecus troop to go get the
other Erectus group. That is very hard for me to imagine.

So yes, I understand what you are saying about gene selection within
species, but there is stuff that happens across species that compete
for the same niche that make me more loyal to my species than to a
random other species. Humans wiped out every other dominant primate
species world wide (exception Florensiensus lived isolated til about
10,000 years ago) wherever they met.

The enemy of my enemy is my friend.

>> > And while children share 50% of the genetic endowment of each
>> > parent, already grand-children share only 25%, and so on. Accordingly,
>> > in
>> > terms "caring for one's DNA", AGIs increasing by 10% the life of one's
>> > children at the price of the immediate extinction of the rest of
>> > humankind,
>> > and the extinction of the offspring itself of the invidual concerned but
>> > only after a few generation, should be heartily welcome by the said
>> > proles.
>>
>> I'm not sure of that.. maybe you need to hone your marketing message...
>> LOL
>
>
> I am not saying that this would be a popular ideology. On the contrary, I am
> saying that most people do not think along those lines.

Ah, I thought you were saying the opposite.

-Kelly