[ExI] The NSA's new data center

[Tomasz Rola]

On Sat, 31 Mar 2012, Eugen Leitl wrote:

> On Sat, Mar 31, 2012 at 09:45:07PM +0200, Tomasz Rola wrote:
> 
> > First, while cost of mere storage (a.k.a. price per gigabyte) is dropping, 
> > there is more to having extralarge database than just stacking haddrives 
> > on each other. I think maintainance cost is going to kill any such project 
> 
> In practice, the dominant costs are energy, and then the raw hardware
> costs, in case of storage, mostly disks. 
> 
> > very quickly. The biggest publicly aknowledged databases nowadays range 
> > from petabyte to somewhere around 10PB (hard to tell where exactly this 
> 
> Database sizes are irrelevant, you're running a highly specialized system,
> not stock Oracle. 
> 
> It's perfectly possible to keep ~PByte/rack online, and I would put
> the limit at 10^4 racks/facility. So in practice you're looking at 
> ~10 EByte/facility. It's probably more ~1 EByte/facility.
> 
> Many other of your assumptions are equally questionable, I will not however
> address them in detail.

Eugen, I like to learn, even on my own mistakes. If you have more to say 
about it, feel free to shoot.

Keep in mind I am not talking about what is going to be possible. It will 
be or it will be not. I believe only in shop inventory and numbers. There 
is exabyte tape library being sold by Oracle and there is about 20Tbps in 
transatlantic cables combined, with plans to extend this by maybe 300% 
*which are plans*:

http://en.wikipedia.org/wiki/Transatlantic_communications_cable

http://en.wikipedia.org/wiki/Exabyte

One exabyte library, which has to be maintained (as you said: energy, 
housing, but also active checking for errors in data made by bit rot). It 
will at best store surv data for 100 thousand heads (based on Anders' 
estimate of 10TB/year/head which is equivalent to 2Mbps a-v stream, if I 
am right). Two such libraries if you want to have any significant error 
protection.

Data transmission from sensors in the field to any kind of storage you 
want, because you do want to store this data somewhere? It is easy to 
connect cities with hi-speed net, it is easy to create metropolitan 
network, but it is not easy to deliver hi speed to every place on the map:

http://en.wikipedia.org/wiki/Last_mile

The last mile problem remains a problem, no matter if you want to deliver 
data down to enduser or up from enduser to central hub. There is going to 
be central hub, either global one or many locals. The problem remains.

Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola at bigfoot.com             **