[ExI] Who is covering corruption in AI?
anders at aleph.se
Thu Nov 1 19:41:45 UTC 2012
On 01/11/2012 18:20, Charlie Stross wrote:
> There is a body of fiction exploring this subject. Notably "The Quantum Thief" (and sequellae -- it's going to be a trilogy) by Hannu Rajaniemi[*], and, ahem, "Rule 34" by yours truly. Also "REAMDE" by Neal Stephenson treads on the fringes of that territory.
> [*] First SF novel I've read where (a) quantum cryptography is a vital plot point, and (b) the author understands WTF he's talking about. (Hannu acquired a PhD in string theory before he got into writing SF.)
Second that. Just finished The Fractal Prince, very good (some nicely
described quantum shenanigans at the start; the quantum boxing approach
to keeping something in was new to me, despite having written about AI
boxing myself). I have just started with Rule 34.
As for the AI thief, I think it is an underappreciated societal risk.
Right now identity theft and stealing from credit cards still require
some hands-on work, and this limits their effectiveness and speed.
Suppose someone found a way of automating the process of extracting the
money? That means that the stealing could be done on computer
timescales, and scaled up to the total number of stolen credit card
numbers (presumably a fairly significant fraction of all of them, given
the statistical distribution of data losses). Also, automation means
that rare skills can be encoded in easily copyable scripts, something
that turned hacking from a skilled activity to the domain of
script-kiddies and scammers. A rational thief would milk a lot of people
for a small amount, retiring wealthy to some suitably obscure
jurisdiction with the right kind of banks. An irrational thief would
milk for a lot more, likely getting caught but also doing a lot more
damage to the credibility of our credit system - and that would be
rather bad overall. And if the exploit exists and a script-like form and
you are a thief, you will want to use it as early as possible before
either the exploit is closed or another, less rational thief uses it. So
you get a race to drain as many accounts as you can.
The fact that right now people do not see this scenario as very
plausible doesn't comfort me. Because false security is just the setting
in which nasty exploits can get maximum effect. Yes, most of the systems
involved in identity, credit and banking are security-minded and
accountable... but that doesn't stop a steady stream of minor crime, and
looking at computer security we know that sufficiently devious exploits
get through a lot of apparent armour.
Future of Humanity Institute
Oxford Martin School
Faculty of Philosophy
More information about the extropy-chat