[ExI] 23andme again

[Eugen Leitl]

On Fri, Jun 14, 2013 at 03:14:40PM -0400, Anders Sandberg wrote:
> On 2013-06-13 07:42, Eugen Leitl wrote:

> In order to figure things out, you need to be able to acquire
> information, process it and know how to notice interesting things.

And in the intelligence business, without anyone being the wiser.
 
> Right now we are somewhat obsessed with the big data idea where the
> data all resides in a central location, having been acquired
> piecemeal or wholesale over time by some organisation. It is
> effective, but there are other ways of gathering data like
> distributed searches and P2P. Currently they are inefficient, yet we
> cannot be certain they are not going to become big. The Internet of
> Things looks like it is going to store lots locally rather than in
> the cloud.

Absolutely. But nobody is going to offer the spooks full
access to their network infrastructure and cluster
facilities, or end users letting them run code on
their routers and their mobile devices -- not voluntarily,
and not without noticing it, at least occasionally.

In fact, it is possible to make the job of data and
metadata collection very hard without inconveniencing
the end user very much.
 
> The processing depends on what algorithm you runs. Right now we are
> getting good at the algorithms that run well on clusters, but again
> that might just be a low-hanging fruit. There are plenty of

If you want to cross-correlate data on billions of entities,
you will need a lot of memory and a fast interconnect.
There are use cases where you can process data on the
edge (in spatial processing), and it is being done.
That kind of processing is relatively simple, however.

> interesting distributed algorithms, and it is not too unlikely that
> people might have options for running algorithms in some of the
> clouds.

Clouds consist of data centers full of hardware, running
virtualized applications. Spooks have their own data centers
and their own cloud, as well as classical supercomputer
clusters.
 
> I suspect that the key thing is people asking the right kind of new
> question. Much of the big data infrastructures today are set up to
> answer general questions or anticipated future questions, yet they
> might well enable new user categories asking unprecedented
> questions. 23andMe was not intended to support genealogy. Flickr was
> not intended to build ontologies. Google was not intended for
> finding unsecured surveillance cameras.
> 
> So the real thing to watch out for is people who come up with clever
> questions. We have built an infrastructure that makes it easier to
> answer them. A Logic Named Joe is coming our way.

A Botnet Named Joe, more likely.