[ExI] The NSA's new data center
rtomek at ceti.pl
Sun Apr 1 02:01:25 UTC 2012
On Sat, 31 Mar 2012, BillK wrote:
> On Sat, Mar 31, 2012 at 8:57 PM, Eugen Leitl wrote:
> > Database sizes are irrelevant, you're running a highly specialized system,
> > not stock Oracle.
> > It's perfectly possible to keep ~PByte/rack online, and I would put
> > the limit at 10^4 racks/facility. So in practice you're looking at
> > ~10 EByte/facility. It's probably more ~1 EByte/facility.
> kilobyte (kB) 10^3
> megabyte (MB) 10^6
> gigabyte (GB) 10^9
> terabyte (TB) 10^12
> petabyte (PB) 10^15
> exabyte (EB) 10^18
> zettabyte (ZB) 10^21
> yottabyte (YB) 10^24
> The original article says 'as a 2007 Department of Defense report puts
> it, the Pentagon is attempting to expand its worldwide communications
> network, known as the Global Information Grid, to handle yottabytes
> (1024 bytes) of data.'
> Wikipedia says:
> In January 2012, Cray began construction of the Blue Waters
> Supercomputer, which will have a capacity of 500 petabytes making it
> the largest storage array ever, if realized.
I think there needs to be made a distinction between:
- storage array - based on multiple hard drives - good for example, to
process grandiose matrixes... lets say square matrixes having 10^8 rows
and columns of double precision floats and you want to multiply two of
them... a trivial algorithm wouldn't fit into any supercomputer memory,
AFAIK (K-computer has about 1,3PB of RAM = 80000 nodes * 16GB RAM each,
by wikipedia). I'm not sure if there is any better than trivial for this.
Anyway, by definition of matrix *, you need to go through one matrix many,
many times. One such matrix takes 8 bytes per doublefloat * (10^8)^2 =
80PB = 80000 * 1TB disk... But perhaps it could be done with tapes, I
really don't know. I mean, without burning them.
- tape library - is made of tapes, plus enclosure (possibly hermetic) plus
some robots inside - good if you have a Library of Congress and you want
to count word occurance frequency. You only need to go throu the whole
data once and you are done.
> So for yottabytes the NSA must either be thinking years in the future,
> or planning many separate but linked data centres.
Ehem, something like, say, 1000 DCs each having 1000EB capacity - and
considering 1EB = 10^6 * 1TB, I wonder if world's combined tape production
capacity can deliver this much - and how quickly. For perspective, before
2011 flood in Thailand took about 1/3 of capacity, hard drive production
was about +650 million pieces annually. So, about one data center in two
years, assuming they all have 1TB drives and suck whole world's
production for themselves. All in all, a hell of rare elements ore is
needed. I mean, I guess.
** A C programmer asked whether computer had Buddha's nature. **
** As the answer, master did "rm -rif" on the programmer's home **
** directory. And then the C programmer became enlightened... **
** Tomasz Rola mailto:tomasz_rola at bigfoot.com **
More information about the extropy-chat