[ExI] Why care about AI friendliness? (was Re: singularity summit on foxnews)

[Mike Dougherty]

On 9/14/07, Stefano Vaj <stefano.vaj at gmail.com> wrote:
> This is one of the interesting points. Does one mean "your children"
> in the literal sense? Or perhaps children who happen to be as
> genetically close as possible (say, of your tribe, race or country)?
> Or future generations of the humankind irrespective of any direct,
> albeit vague, genetic connection? And what about "children of the
> mind", as AIs could not too unreasonably be qualified?

What/who can you identify as your children?  I meant it in the sense
of those younger than the "I'll be dead before AI matters" group,
presumably with enough connection to actually care about their
welfare.  "children of the mind"?  How do you propose we make those?

> That is, as long as you care to emulate features which used to belong
> to your biological self...  Which in turn may be necessary even for
> purely artificial AIs if we are to consider them as  "friendly" or
> "unfriendly" in any more meaningful way than the very complex
> terrestrial climate system may be. Otherwise, the distinction between
> uploaded or emulated humans and purely artificial AIs gets blurred
> very quickly...

This is why I made the distinction of an AI developed independently of
the human hardware model  (if hardware is a valid term for meat)  I
feel the AI that evolves from software will be fundamentally different
from AI that is modelled after brain function.  If the processes of
the human brain are simulated in a computer, should the resulting
identity be part of humanity?  Should the definition of humanity be
expanded to include this state of being?  Suppose a degenerative
disease leaves you only two choices: death and destructive uploading -
would you choose the upload even if it meant you were 'reclassified'
according to your definition of the personhood of a simulated-human
AI?  For this reason, I am compelled to extend the definition of
humanity to instances of human processes on alternate substrates as
well as self-aware derivative works of humanity.  I'm sure a more
legallese and ironclad license will need to be agreed upon.  Hopefully
we can manage an inclusion policy that prevents Homo sapiens sapiens
from becoming an insignificant minority or a vestigial novelty.