[ExI] keynes vs hayek again, was: RE: 3d printers for sale

[Anders Sandberg]

On 28/08/2012 07:18, spike wrote:
> Car-drones are a big threat.  It might be hard to combat such a weapon.

Current society is very vulnerable to autonomous car-bombs (or my 
earlier sketch of terrorist drones). That doesn't mean it will remain 
vulnerable if they start to occur. It is a bit like computer security: 
it was nonexistent until needed. But continuing that analogy, if the 
incentives or popular solutions turn out to be bad, then we might get a 
suboptimal situation, like in computer security (in fact, bad computer 
security together with robot-like devices makes all such devices 
potentially malicious).

What would a *good* solution to the car-bomb problem be? Transparency is 
only good enough if it prevents bad things before they happen, so we 
need very proactive monitoring. Since a car already is a deadly device 
(just get it to drive fast to crash) it is not clear that it could see a 
hack attack coming: for transparency to fully work we need to have 
solved the computer security problem. Something like capability approach 
might be more promising: there are built in and *hardwired* safety rules 
(like "don't hit pedestrians", "if loaded by more than X kg of 
something, running with no driver, and approaching a federal building, 
allow a surveillance sweep in a sepaarate garage"), and to participate 
in the traffic system a car has to prove cryptographically that it 
follows them ("trusted commuting"). Loads of implementation and 
introduction problems of course.

An insurance friend of mine pointed out that one could estimate the 
probability of a terrorist attack on US soil in two ways: either try to 
run a complex model based on your theory of terrorism, known data, Monte 
Carlo simulation etc., or consider the question "What is the maximum 
number of real terrorist attacks that the US government/people would 
accept per year before they implemented a solution?" The latter method 
is crude, but actually helps bound the question in a constructive way. I 
am pretty confident that a similar question for car drones will produce 
a fairly low number, and that the "market" will find a way of solving 
the problem even if we cannot imagine how.

The scary part is that we apparently have decided that the number of 
cyberattacks we accept per year is pretty high: the "equilibrium" might 
not be rational.



-- 
Anders Sandberg,
Future of Humanity Institute
Philosophy Faculty of Oxford University