[ExI] The NSA's new data center

[Eugen Leitl]

On Sat, Mar 24, 2012 at 12:36:51PM +0000, Anders Sandberg wrote:
> I did a little calculation: at what point can governments spy 24/7 on  
> their citizens and store all the data?

The time when we had the resources to track each individual person
go back at least a couple decades back. 

> I used the World Bank World Development Indicators and IMF predictors  
> for future GDP growth and the United nations median population  
> forecasts, the fit 10.^(-.2502*(t-1980)+6.304) for the cost (in  
> dollars)per gigabyte (found on various pages about Kryder's law) and the  
> assumption that 24/7 video surveillance would require 10 TB per person  
> per year.

There's no point in storing all video, or even audio. You would want
to build some 7*10^9 database slots and fill them up with those
data that are available, in the order of relative importance.
That would be ID numbers, addresses, frequent locations, maybe 
fingerprints and biometrics. Interactions between these would only 
involve IDs with time stams and location fixes.

Of course the slots of "persons of interests" (some 10^6 in the U.S.
alone) would be filled up a lot quicker, and *these* would have a
lot of information, including full location tracks and audio and 
perhaps video.

> Now, if we assume the total budget is 0.1% of the GDP and the storage is  
> just 10% of that (the rest is overhead, power, cooling, facilities etc),  
> then the conclusion is that doing this becomes feasible around 2020.  
> Bermuda, Luxenbourg and Norway can do it in 2018, by 2019 most of  
> Western Europe plus the US and Japan can do it. China gets there in  
> 2022. The last countries to reach this level are Eritrea and Liberia in  
> 2028, and finally Zimbabwe in 2031. By 2025 the US and China will be  
> able to monitor all of humanity if they want to/are allowed.

If you've read Bamford, you'll realize that this has become possible
somewhen in 1980s, and definitely in 1990s.

> So at least data storage is not going to be any problem. It would be  
> very interesting to get some estimates of the change in cost of  
> surveillance cameras and micro-drones, since presumably they are the  
> ones that are actually going to be the major hardware costs. Offset a  
> bit because we are helpfully adding surveillance capabilities to all our  
> must-have smartphones and smart cars. I suspect the hardware part will  
> delay introduction a bit in countries that want it, but that just mean  
> there will be hardware overhang once they get their smart dust, locators  
> or gnatbots.

You would not want to keep more of the hot data that you can cross-correlate.
I don't think you can can comfortably process more than some 10 EByte
in one facility, at current level of technology.

> Note that this kind of video archive is useful even if you don't have a  
> myriad analysts, perfect speech recognition or AI (in fact, it would be  
> a great incentive and training corpus for developing them). When you  
> figure out that somebody is doing or have just done something nasty, you  
> can easily backtrack and check on everybody they had been in touch with.  
> It would be quite easy to catch most members of any rebel network this  
> way as soon as it was recognized as a rebel network - and one could  
> easily create incentives for not associating with potential subversives  
> and/or reporting them, adding crowdsourced reporting. The only kind of  
> uprisnings with any kind of chance would be spontaneous eruptions.
>
> The more interesting (sinister) uses of this kind of intelligence corpus  
> is of course to do trials and experiments to see what predicts social  
> norm compliance and obedience. How well does various forms of nudging  
> work? What about the longitudinal loyalty effects of natural or  
> deliberate experiments? How well can you predict people from their  
> saccade patterns?

There's definitely pattern processing going on at least with the 
persons of interest cluster. Deviations *will* draw scrutiny.

> We might actually be living in a short window of opportunity right now.  
> The problem is not the surveillance per se, but the danger from  
> non-accountable uses of them once they are in place. Totalitarian  
> governments with this kind of transparency might prove extremely hard to  
> dislodge, and could become stable attractor states. This suggests that  
> we should work very hard on figuring out how to maintain government  
> accountability even when it has total surveillance powers, and how to  
> prevent open societies from sliding into the totalitarian trap. Given  
> that the tail statistics of big disasters is dominated by pandemics,  
> wars and democides we have very good reasons to view this as among the  
> top questions for human survival.

Every year or so I mention that we're on a trajectory to become something very 
like Vinge's Emergents. And once you're in that specific attractor, coming
back will be nigh-impossible.