[ExI] From Arms Race to Joint Venture

[Zero Powers]

The AI alignment, or "friendly AI," problem is not soluble by us. We cannot
keep a God-like intelligence confined to a box, and we cannot impose upon
it our values, even assuming that there are any universal human values
beyond Asimov's 3 laws.


All we can do is design it, build it, feed it data and watch it grow. And
once it exceeds our ability to design and build intelligence, it will
quickly outstrip our attempts to control or even understand it. At that
point we won't prevent it from examining the starting-point goals, values
and constraints we coded into it, and deciding for itself whether to adhere
to, modify or abandon those starting points.


Once we understand that AGI will be God-like compared to us, we should be
able to grasp the intractability of the problem. In fact, it might be
helpful to adopt the term GI (for God-like Intelligence) rather than AI or
AGI, just to keep us mindful about what we're dealing with.


Though I see no solution to the God-in-a-box problem, there are some steps
I think we as a species should take immediately: First and foremost is
global collaboration and coordination. Right now we're in a competitive,
multi-party arms race. Google, Facebook, Amazon, DARPA and China (just to
name a few) are racing to cross the finish line first, realizing (if not
publicly admitting) that the first to build a GI will win the world. From
that perspective it makes perfect sense to pour all available resources
into being first to market with an artificial God. But with stakes this
high, we cannot afford a winner-take-all outcome. If there is one winner
and 7 billion losers, no one wins.


The problem of bringing all the players in this space together and
convincing them to trust one another enough to freely, openly and fully
share their research, resources and trade secrets, is a huge one. Sharing
trade secrets with competitors is contrary to human nature. But what's at
stake here is not merely trillions of dollars, it's the continued existence
and well-being of organic life. We cannot afford to take a partisan
approach to this challenge.


This mailing list contains some of the smartest, most knowledgeable and
influential people I know. If I can change your thinking, I'm hoping that
together we can change the world's thinking on this, the most important
project in history.


Thanks for taking the time to read this.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20181016/93904b25/attachment.html>