[ExI] The Actual Visionary of the Future

Kelly Anderson kellycoinguy at gmail.com
Mon Oct 28 19:51:22 UTC 2013


On Mon, Oct 28, 2013 at 4:37 AM, Eugen Leitl <eugen at leitl.org> wrote:

> On Sun, Oct 27, 2013 at 01:50:23PM -0600, Kelly Anderson wrote:
>
> > That's simply not true Eugen. You're better than that.
>
> That was obviously hyperbole, to make a point. He is, however,
> prone to see exponentials where there are none.
>

I think his data is well researched. Whether all of the curves extend into
the future, and just how far they will extend is guess work.


> > I believe MORE things are exponential than Ray does, and even I don't
> > believe everything is exponential. That being said, lots of things are,
> > like the savings in your bank account.
>
> Exponential growth of compound interest is a textbook case
> where your numerical model of physical layer processes and reality
> increasingly diverge, requiring periodic, painful readjustments.
>

I have never heard of a case where a bank simply refused to pay interest
because there was just too much money in the account. So what are you
referring to here?


> > > People
> > > forget that hard drives stopped doubling, at least for a short while.
> > >
> >
> > Because of a flood in Thailand. Nobody has said there wouldn't be bumps
> in
>
> Thailand was not the reason.
>
> We're stuck at 4 TB because they ran into limits of a particular
> technology.
>

I'm baffled by your use of the word "stuck" here. We just got to 4 Tbytes
not that long ago. We always get "stuck" by this definition. I have
attached my spreadsheet of hard drive prices that I have been maintaining
for a few years, but initially got elsewhere. I welcome comments.


> In case of platters full of spinning rust the snag is temporary,
> as there are two successor technologies about to enter the
> marketplace (HAMR and BPM, not new but close to becoming
> mature enough for practical applications) so there's probably
> another order of magnitude still to go before end of the line.
> That makes it 40 TB.
>

That hardly seems like "stuck" to me. Knowing how we're going to get the
next order of magnitude is good enough for me.

Aside from that, there are things out there that promise to give the next
order of magnitude after that... such as:
https://en.wikipedia.org/wiki/Racetrack_memory
Which is clearly not ready for prime time, but is a good idea of the sorts
of things that might happen when brilliant people are tasked with an
objective.

Coincidentally, NOR flash has recently also entered
> scaling limits.
>
> The time for surface scaling is running out. The only
> alternative is 3d volume integration. We do not have anything
> in the pipeline to arrive in time, so there will be a gap.
> The only technology to interpolate would be Langmuir-Blodgett
> serial layer deposition, with according 2d liquid mosaic
> self-assembly/alignment. I'm not aware of this technology
> to be ready for deployment. Next after that is 3d crystal
> self-assembly from solution. This is even further away.
>

That's ok, we have time to get this stuff right before falling off the
curve.


> > the road, just that there was an overall trend.
> >
> >
> > > People are unaware of finer points like
> > > http://postbiota.org/pipermail/tt/2013-October/014179.html
> >
> >
> > Ok, I read that, and what it said in a nut shell is "fuck this is hard".
>
> Yes, this is the nature of limits. Instead of constant doubling
> times the last few show longer and longer steps. As I told you,
> we're no longer at 18 months but at 3 years doubling time this
> moment. The next doubling times will be longer. This means
> that linear semilog plot is no longer linear. No more Moore for you.
>

And yet, it is still doubling rapidly. The end result is the same, just at
a slightly different time scale. And there is no guarantee that we won't
make a hop with a new technology and get back on any given curve. It can
happen.


>
> > Not, I expect it to come to a screeching halt.
>
> Why do you expect that? Look at the price tag of the
> zEnterprise 196. Obviously, a somewhat higher margin
> than on a 50 USD ARM SoC.
>

Sorry, you've lost me here. I don't know what these things are.


> > If they read your posts here Eugen, they might decide not to thaw you
> out.
> > Who needs a pessimist in a utopia... :-)
>
> Utopia? I'm afraid I have another piece of bad news for you.
> Very bad news, I'm afraid...
>

Anyone looking at 2013 from the time frame of 1913 would clearly call this
utopia, at least from the technological standpoint. Also from the number of
people operating under democracy, decreased violence and a number of other
points. Not that it is utopia in every way.


> > > > I know you MUST believe that computers will continue to get faster,
> even
> > > if
> > > > they don't quite keep up the doubling pace. Right?
> > >
> > > What does faster mean? The only reliable way is a benchmark.
> > > The only really relevant benchmark is the one that runs your
> > > problem. As such faster never kept up with Moore's law.
> > > Moore's law is about affordable transistors, and that one
> > > gives you a canvas for your potential. So your gains are less
> > > than Moore, and sometimes a lot less. For me personally,
> > > the bottleneck in classical (GPGPU is classical) computers
> > > is worst-case memory bandwidth. Which is remarkably sucky,
> > > if you look it up.
> > >
> >
> > The problem I care about the most is computer vision. We are now
>
> Computer vision is very easy, actually, and quite well understood.
>

You are clearly stark raving MAD. There is no computer on earth that can
tell a cat from a dog reliably at this point.


> The low number of layers and connectivity (fanout), all local at
> that, a retina needs are within the envelope of silicon fabrication.
>

The retina is not what I'm talking about. I'm discussing image
understanding. "That is a picture of a dog in front of a house. The house
has a victorian architecture. The 1957 Cadillac next to the house would
indicate that the picture was most likely taken between 1956 and 1976."


> > approaching automated vehicles becoming a reality. I thought it would
> > happen in 2014 since 2004. It may be delayed a year or two by bureaucrats
> > and lawyers, but the technology should be cheap enough for luxury cars to
>
> I'm afraid luxury something is going to be a very, very small market
> in the coming decades.
>

Stop. This is just irritating and unhelpful.


> I agree that autonomous cars are mostly a very good thing, unless
> you happen to be a trucker, or a car maker.


I'm not sure how autonomous cars are bad for car makers. I do get why they
are bad for truckers.


> Whatever Germany
> earns on car making is about enough to pay for the fossil fuel
> imports.
>

You're confusing me again.


> > have highway cruise control (including steering) by 2014 or 2015. So my
> > venture into guessing the future was pretty close, using Ray's technique.
> >
> >
> > > Ok. So, now your transistor budget no long doubles in
> > > constant time, but that time keeps increasing. It's roughly
> > > three years by end of this year, no longer 18 months.
> > > Physical feature size limits are close behind, and your
> > > Si real state is a 400 mm pizza, max. WSI gives you a
> > > factor of two by making yield quantitative, but it wrecks
> > > havoc to your computation model, because grain size starts
> > > being tiny (less than mm^2), and asks for asynchronous
> > > shared-nothing, and did I mention fine-grained? So no
> > > TBytes of RAM for your LUT. The next step is FPGA, as in
> > > runtime reconfigurable. That *might* give you another
> > > factor of 2, or maybe even 4. Stacking is off-Moore, but
> > > it will do something, particularly giving cache-like
> > > access to your RAM, as long as it's few 10 MBytes max.
> > >
> >
> > I've predicted that they will go to 3D. It is the only logical way to go
>
> Everybody and his dog predicted that, since early 1970s.
> The difficult is actually making it happen, just in time
> when semiconductor photolitho just runs out of steam.
> Guess what, that time is now. So, where is your 3d integration
> technology?
>
> > from here, other than maybe 2 1/2 D first...
>
> You can't have that by semiconductor photolitho. Stacking is
> off-More. What else have you got?
>
> >
> > > And then you have to go real 3d, or else there's gap.
> > >
> >
> > True, unless something completely different comes along, which may not be
> > highly likely.
>
> New technologies typically take decades of development, until
> they're sufficiently matured so that they can take on mature
> technologies that have ran into their scaling limits.
>

I totally agree that the 2d processes we are currently using are running
into limits. But we will keep making the stuff we're making now cheaper. In
my mind, that keeps us on Moore until such time as a 3D solution is worked
out that makes things faster.

The main problem in my mind isn't making stuff smaller, but in dissipating
heat so you can stack it up close to each other. That's what I mean by 2.5
D.


>
> >
> > > My guess the gap is somewhere 15-20 years long, but
> > > we've got maybe 10 years until saturation curve is pretty
> > > damn flat.
> > >
> >
> > Ok. Then we can start making larger structures. It won't speed up due to
> > decreasing transistor size, but it will be able to do useful work.
> Imagine
> > a 3d CPU 5 inches on a side. That could do some serious work. More than a
> > human brain.
>
> The human brain is a 3d integrated assembly of computational
> elements which are built from features on nm scale.
>

Sure is. And I'm convinced that we'll soon enough have similar
computational devices. Maybe we'll even figure out how to grow them. Who
knows.


> > > He implicitly implied we'll run on 100% of thin-film PV in 16 years.
> > > That was 2011, so make that 14 years. This means 4.2 TWp/year just
> > > for power in a linear model, nevermind matching synfuel capability
> > > (try doubling that, after all is sung and done -- 8 shiny TWp/year).
> > > We're not getting the linear model. In fact, we arguably sublinear,
> > > see
> > >
> http://cleantechnica.com/2013/10/14/third-quarter-2013-solar-pv-installations-reach-9-gw/
> >
> >
> > You obviously don't understand the nature of his prediction. If he says
>
> Obviously. I expect prediction to be brittle, and that the originator
> is prepared to eat some crow, in case she is wrong. I'm old-fashioned
> this way.
>

No Eugen. Let's simplify. If I predict that there will be 20 billion cats
running loose in Germany in 16 years and that the number of cats will
increase exponentially with a doubling every two years, and you come back
in two years and say "We don't yet have 2 billion cats, so the prediction
must be wrong." Then you clearly don't have an understanding of what
exponential means.


> > that the doubling in solar efficiency is 3.5 years (going from memory)
> then
> > half of the solar he envisions will be installed between July 2023 and
>
> Thank you for explaining exponential growth to me. I think I've first
> understood it before I was 10.


Clearly not so much.


> The nature of solar cells is the only
> way to double the output is to double the surface. And the according
> infrastructure in the background, simple things like 10 GUSD plants,
> electric grid upgrades, storage systems, and the like.
>
> > 2027. What's being installed now probably is sublinear, that's what an
> > exponential would predict. He didn't predict a linear model. We'll
> revisit
>
> You're not understanding me. It used to be exponential. Because it's
> very easy to double very little. Until suddenly you have to double
> quite a lot. This isn't a lily pond or a bacterial culture, this is
> infrastructure work.
>
> So Ray is already wrong, once again. The trend is no longer exponential.
>

The only thing Ray said about solar is that the cost of the panels
themselves was on an exponential curve. There is a lot more to solar costs
than that. So if you hold him to something he didn't say, then yes, we're
off the curve. If you have data showing that the panels themselves have
fallen off the curve, then I'll give you credit for being correct, and will
admit that Kurzweil is off, at least for the moment. (Unfortunately for
you, you can sometimes hop back on the curve later with a new technology.)


> > his prediction in 2027 if we're both still communicating by then.
>
> The prediction is 100% of electricity in 16 years. He then scaled
> that back by saying 20 years. That's 2021.
>
> Given that we're already off-exponential, I expect that you keep
> posting "I'm wrong" every year.
>

I will give you this. Ray made a prediction about the price of solar panels
and then extrapolated that to the price of solar generated electricity
(which includes infrastructure, inverters, batteries and the like) which is
not the same thing. I don't think Ray is right when he says things that
imply that lots of infrastructure will just pop into existence. But, how
much more infrastructure would pop into existence if solar panels were
nearly free? A little more, but the other parts are still damn pricey,
better than half of a current home installation.


> > > apply. However, in a sense it does apply. We do get some percent
> better at
> > > > extracting what's left each year. That doesn't mean we get an
> exponential
> > >
> > > No, in terms of net energy we're not getting better. We're actually
> > > getting worse.
> > >
> > > > amount of oil, since there's a limited amount of the stuff. But it
> does
> > > > mean that we get exponentially better at finding what's left (note
> that
> > >
> > > We're not getting better. We've mapped all the stuff, there are almost
> > > no unknown unknowns. And dropping EROEI and even dropping volume
> > > (not net energy, volume!) per unit of effort is pretty much the
> > > opposite of exponential. Do 40% of decay rate/well/year mean a
> > > thing to you?
> > >
> >
> > You misunderstand my point again. I know it's harder to get oil. But we
> > develop new technologies for getting at what's left.
>
> Fracking is 40 years old. Fracking is running into diminishing returns.
> So where are your new technologies, which need to be already in wide
> deployment, now?
>

Fracking is currently producing a fair amount of oil. It's produced a lot
of buzz. I am not an expert on oil production techniques, so I'm not going
to argue from ignorance.


> > > > this curve is likely much more gentle than computing, with a
> doubling of
> > > > reserves we can get at maybe every 20 or 50 years. I don't know.)
> > >
> > > There are no exponentials in infrastructure. There is an early
> > > sigmoidal that looks that way, but we've left that already.
> > >
> >
> > Infrastructure can change rapidly. How long did it take for everyone to
> get
>
> No. Infrastructure takes 30 years, frequently longer. That's a constant.
>
> > a cell phone? Smart Phones? When electric cars make financial sense (if
>
> How long did take for everybody to get their own synfuel plant?
>

I agree larger infrastructure is harder. But things can change rapidly when
they have to.


> > they ever do) then people will switch to them quickly. Large
> infrastructure
>
> Do you understand the logictics of car production? Battery manufacturing?
> Dynamics of fleet exchange? Recharging infrastructure? Including the money
> to fund it all? Do the math, it is really quite illuminating.
>

I'm sure it is a hard problem. I do know the US car fleet turns over every
16 years or so. That would indicate that if it became a real issue and we
had a real solution that it could be done in 16 years.


> > like roads and so forth will remain problematic until robotics is good
> > enough to do much more of the job.
> > > >
> > > > > This is the opposite of science.
> > > > >
> > > >
> > > > It is a part of science, the hypothesis part. LAR applied to
> computing
> > > > available per dollar in particular is a hypothesis formed in the mid
> > > 1960s.
> > > > As far as I know, we are still more or less on that track, though
> they
> > > have
> > >
> > > No, we're not. See benchmarks.
> > >
> >
> > Data please. I can't find any. I have looked.
>
> Try Stream, though it's a synthetic benchmark
>
> http://www.admin-magazine.com/HPC/Articles/Finding-Memory-Bottlenecks-with-Stream
> It would be a reasonable assumption for retina-like processing
> scaling. Deeper visual pipelines are different. Here, you need
> to access something like fetching from a large (>>10 GByte) of
> random pointers.
>

A coprocessor with direct access to the "memory" of a retina would be very
good for problems like this.


>
> > > People who hear about Amdahl's Law the first time have to stop
> worrying,
> > > and embrace nondeterminism. People who expect reliable systems at
> hardware
> > > level are gonna have a bad time.
> > >
> >
> > I disagree with that. There will be reliable hardware, or they won't be
>
> If you want to not run into Amdahl you need to embrace nondeterminism.
> Building test harnesses just got a bit harder.
>

You would need a new generation of programmers to go there, I think.


> > able to sell it. No matter how slow the previous generation was. It is
> hard
>
> Yes, there will be unreliable hardware. This is one of the problems in
> exascale: unreliable transport and unreliable components (as in: parts
> of your system keep failing at runtime, and you diagnose and remap to
> hot spares, all without breaking a stride). Beyond that, you've got
> stochastical computing elements. That's one of the joys of living at
> nanoscale.
>

But it has to be dealt with at a low level, or programmers won't stand for
it.


>
> > enough to get programmers to do multi-threading. It would be damn close
> to
> > impossible to get them to switch to a model where the answer might not be
> > right.
>
> There is no longer "exactly right" there is only "good enough".
>

Well, for now we have exactly right for computation, in the future, and for
certain algorithms that are NP complete, for example, good enough is fine.
But for credit card processing, good enough isn't good enough.


>
> > A pessimist will just hole up in his cave. I refer you to "The Croods" to
>
> I don't know what a pessimist would do. I do know that the only guy who'd
> still have water when his cars break down in the desert is a realist.
>
> The optimists always end up as bleached bones. Your call.
>

I prefer the extra years of life. Perhaps I'll spend them in a cave in the
desert.

-Kelly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20131028/106660a5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Hard Drive Costs.xlsx
Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Size: 38760 bytes
Desc: not available
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20131028/106660a5/attachment.bin>


More information about the extropy-chat mailing list