[ExI] 'Friendly' AI won't make any difference

[Flexman, Connor]

>
>
> ​You want the robot to put the ketchup in the bottle before it puts the
> cap on, and if it falls into a infinite maze contemplating how best to put
> the ketchup in the bottle the cap will never be put on.​
>
> I maintain that any AI, or any mind of any sort, that has a fixed
> unalterable goal is doomed to failure.
>
>  John K Clark
>

Your point about infinite contemplation or infinite attempts to prove
Goldbach's conjecture is important in designing AI to not fall into this
failure mode, but I don't think it actually merits a condemnation that any
such AI is doomed to failure. The ketchup model itself just emphasizes that
absolute maximizers aren't always the best tool for the job, and that
shortcuts like quantilizers
<https://intelligence.org/files/QuantilizersSaferAlternative.pdf> might
work better even with a fixed unalterable goal.

However, this hides the deeper issue, which is that completing a goal
itself requires that you NOT get stuck in an infinite loop; thus, if you
build any good AI with a goal, it by necessity will not have this issue.
One simple way to engineer this might be to just program in hourly checks
to make sure the AI is taking the best expected path and hasn't been doing
the same thing for an extended period. Just as humans frequently forego a
completely depth-first approach when working with multiple paths to achieve
a goal, so would you want AIs to implement a strategy closer to A* to
combine breadth and depth so that it wouldn't keep "mindlessly" crunching
through multiples of 2 when dealing with Goldbach's conjecture. In fact,
humans can be helpful models when dealing with most of these pathological
behaviors: every time it seems like an AI might make a bad decision, ask
yourself how a human might reason so as to avoid that. You want your
decision theory to actually satisfy your desiderata, not to merely look
mathematical but quickly fall into stupid loops. If you are endlessly
contemplating how to best put the ketchup in, we label that as "neurosis":
an optimal strategy does not endlessly press explore and never exploit the
best available option.

These simple models of Goldbach's conjecture or Ketchup illustrate that one
must take steps to avoid implementing a thought process that doesn't catch
endless useless behaviors, but your concern about a fixed unalterable goal
also partially breaks down when considering that most goals will be much
more complex. If you build an AI that has a simple goal like proving a
theorem, then obviously it will soon satisfy the goal or not and you
probably will want to change its goal, so an unalterable one is silly.
However, for more difficult goals like maximizing money or aggregate
utility, your utility function can probably be essentially unalterable
because its path complexity ensures that it won't be satisfied in the near
term or even get to the point where more work won't drastically help. If an
AI is trying to maximize some high-level goal like making money, it will
almost definitely model instrumental *and alterable* goals like "gain
knowledge of the stock market" and "research possible inventions".
Importantly, though, each of these instrumental goals would come along with
a tag saying "for a while, until we re-evaluate whether it is still
beneficial". If an AI quickly consumes all the knowledge about the stock
market, it won't keep pressing refresh to make sure it DEFINITELY has ALL
of it—it should realize that info there has diminishing returns, and now it
should actually begin trading so as to make money. You shouldn't lose sight
of your goals, and any functioning AI certainly won't lose sight of its
goals. If putting more thought into ketchup funneling is unlikely to
produce more gain than simply funneling the ketchup before someone knocks
it off the table, the ketchup should always get funneled (exponential
temporal discounting helps here).

Connor Flexman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20160228/6d4e1a1e/attachment.html>