[extropy-chat] "3 Laws Unsafe" by the Singularity Institute

[Adrian Tymes]

--- Mike Lorrey <mlorrey at yahoo.com> wrote:
> Not necessarily. Computer languages change
> definitions of commands with
> regularity. Not complete changes, but incremental
> additions, just as
> occurs in legal dictionaries.

Not entirely true.  (And beside the point, since as
you mention, the Laws are in malleable human language,
not computer language.)  There are languages that most
definitely do not change their core commands from
generation to generation.  For instance, "and" means
"and" in the boolean sense, or slightly more complex,
"sprintf" means "put this formatted string into this
variable".  Thus has it always been, and the language
makers would be foolish to change it.

That said, there are libraries that change their
functions with each generation.  This is the cause of
much software problems in Windows.  The solution is
simply not to use them.  It might make the software
development more difficult (and thus more expensive,
which is why it's usually not done in practice, unless
stability really will sell more copies of the
software)
but it can be done.

> Furthermore, each new
> generation of
> computer processors themselves add new commands or
> alter old commands. 

Add new commands: so?  The old programs won't use
them.

Alter old commands: not if they want to be backwards
compatible, they don't.  (And if you're "upgrading" to
a non-backwards-compatible processor without
rewriting, or at least thoroughly retesting, the
software for the new processor, you're inviting
disaster whether you have complex AI or simple
spreadsheet software.)

> The greater weakness of this process is that the key
> changes really
> occur at the machine language level. Machine
> language is itself
> 'readable' by a very limited subset of the human
> population. How do we
> actually know that a compiler is interpreting our
> programmed code the
> way we want it to?

Again, this is perhaps delving into too much detail
since the Laws are in English (which, being malleable,
is not fundamentally translatable into machine code
except as a snapshot), but there's a deeper problem
here.  It is theoretically possible to "hijack" a
compiler and OS such that no tools run on a computer
running that OS will detect the hijacking (except
perhaps loading the disk with the hijacked OS on
another computer, but then you have to trust that
computer's OS), and that any attempts to recompile the
OS will be recognized and add in the extra code.  (One
could recompile elsewhere, but again, one would need
to have a trusted OS and compiler first.)  Of course,
the extra code could also have additional malicious
functions.

In practice, this is not an issue, since we have OSes
and compilers that have been thoroughly enough vetted
that the chances of remaining hijacking are
inconsequentially small.

> We see, it seems, news items
> almost every day of
> intentional or 'unintentional' programming back
> doors being exploited
> in current day applications and operating systems by
> malicious
> programmers.

Said back doors having been caused by laziness.  For
example, a lot of the Windows errors have to do with
buffer overflows, a class of error which in theory
should be fixable by the operating system: keep track
of what memory is allocated to what programs, granting
more as requested but keeping track of that too, and
give an error to any program that tries to write
outside of its allocated memory space.  But Microsoft
can't be arsed to put in that protection, possibly
since it would force them to clean up so much
badly-written code they've accumulated over the years
that they'd rather live with the sloppy product (which
mostly hurts other people - their customers).

> The term 'robot', of course, would only apply to
> robots. An artificial
> intelligence could decide that it is no longer an
> AI, once it has
> advanced beyond a certain point in intelligence, and
> has instead become
> a God, thus redefining itself out of the Laws of
> Robotics.

Your other arguments on the terms ring true, but this
one can be easily circumvented.  Instead of referring
to "a robot" in the third person, the robots could
simply refer to themselves.  "This unit will not harm
a human being (modulo definitions of 'harm' and 'human
being'), either directly or indirectly," for example.
This continues to apply even if the robot manufactures
other robots under its control (which would count as
indirect harm, if not direct harm, if they harmed
someone), and even if the robot ceases to think of
itself as a robot.

Overall, though, your logic is correct - indeed, one
could argue that many of Asimov's stories were about
the numerous ways these laws could be circumvented or
otherwise cause complications.  (In particular, I
remember one story about an expensive, mostly
autonomous robot, whose Third Law sensitivity was
cranked up and Second Law sensitivity was cranked
down, until an unintentionally self-destructive order
caused it to enter an endless loop.  I also remember
formation of a "Zeroth Law" that placed the good of
all of humanity above the well-being of any given
individual, justified as a way of serving the First
Law to the widest extent possible, and the resultant -
and senseless on first glance, making sense only after
analysis - negative acts towards human beings that
were thus allowed.  This was especially true for AIs
who had reason to believe that one of the greatest
possible harms to humanity would be their personal
deactivation, thus justifying preemptive reprisals
against those who might seek said deactivation.
Non-violent means where possible, such as getting
someone fired from an influential position rather than
killing that person, but still...)