[ExI] From Arms Race to Joint Venture

[Adrian Tymes]

On Tue, Oct 16, 2018 at 5:29 AM Zero Powers <zero.powers at gmail.com> wrote:
> The AI alignment, or "friendly AI," problem is not soluble by us. We cannot keep a God-like intelligence confined to a box, and we cannot impose upon it our values, even assuming that there are any universal human values beyond Asimov's 3 laws.
>
> All we can do is design it, build it, feed it data and watch it grow.

Or, you know, become it.  Which would have a good chance of "imposing"
upon it something resembling some human's values, at least.

> Though I see no solution to the God-in-a-box problem, there are some steps I think we as a species should take immediately: First and foremost is global collaboration and coordination.

You complain about impossibilities, then you say the first step is
something that is impossible in practice (regardless of theoretical
setups)?  It is at least possible to envision setting up an AI with
motivations that are compatible with continued human existence.

> The problem of bringing all the players in this space together and convincing them to trust one another enough to freely, openly and fully share their research, resources and trade secrets, is a huge one. Sharing trade secrets with competitors is contrary to human nature. But what's at stake here is not merely trillions of dollars, it's the continued existence and well-being of organic life.

Irrelevant.  As you note, some of them think they can conquer the
world.  Some of them are entirely willing to gamble humanity's future
existence in order to potentially conquer it.  Put bluntly: that they
could destroy the world is a risk they are willing to take.

You'll need some other approach than pointing out that they might
destroy the world, to convince them to not potentially destroy the
world.  They don't care.

> We cannot afford to take a partisan approach to this challenge.

And yet that seems to be exactly what will happen.  So, if we start
with that thing that we "cannot afford" as a starting point, how do we
keep humanity alive?

Keep in mind that "keep humanity alive" is now a partisan position.
There are several factions that would prefer to see the vast majority
of humanity - all who will not submit to their particular ideology -
wiped out.