[ExI] extropy-chat Digest, Vol 81, Issue 37

[Keith Henson]

On Sat, Jun 26, 2010 at 5:00 AM,  Ryan Rawson <ryanobjc at gmail.com> wrote:

> Where does the 'few bits per second' figure come from?

Why not ask Google directly instead of asking me?  If you Google human
memory capacity in bytes the very first in the list is this one:

How Many Bytes in Human Memory?

by Ralph C. Merkle

This article first appeared in Foresight Update No. 4, October 1988.

A related article on the computational limits of the human brain is
available on the web.  http://www.merkle.com/brainLimits.html

Today it is commonplace to compare the human brain to a computer, and
the human mind to a program running on that computer. Once seen as
just a poetic metaphor, this viewpoint is now supported by most
philosophers of human consciousness and most researchers in artificial
intelligence. If we take this view literally, then just as we can ask
how many megabytes of RAM a PC has we should be able to ask how many
megabytes (or gigabytes, or terabytes, or whatever) of memory the
human brain has.

Several approximations to this number have already appeared in the
literature based on "hardware" considerations (though in the case of
the human brain perhaps the term "wetware" is more appropriate). One
estimate of 1020 bits is actually an early estimate (by Von Neumann in
The Computer and the Brain) of all the neural impulses conducted in
the brain during a lifetime. This number is almost certainly larger
than the true answer. Another method is to estimate the total number
of synapses, and then presume that each synapse can hold a few bits.
Estimates of the number of synapses have been made in the range from
1013 to 1015, with corresponding estimates of memory capacity.

A fundamental problem with these approaches is that they rely on
rather poor estimates of the raw hardware in the system. The brain is
highly redundant and not well understood: the mere fact that a great
mass of synapses exists does not imply that they are in fact all
contributing to memory capacity. This makes the work of Thomas K.
Landauer very interesting, for he has entirely avoided this hardware
guessing game by measuring the actual functional capacity of human
memory directly (See "How Much Do People Remember? Some Estimates of
the Quantity of Learned Information in Long-term Memory", in Cognitive
Science 10, 477-493, 1986).

Landauer works at Bell Communications Research--closely affiliated
with Bell Labs where the modern study of information theory was begun
by C. E. Shannon to analyze the information carrying capacity of
telephone lines (a subject of great interest to a telephone company).
Landauer naturally used these tools by viewing human memory as a novel
"telephone line" that carries information from the past to the future.
The capacity of this "telephone line" can be determined by measuring
the information that goes in and the information that comes out, and
then applying the great power of modern information theory.

Landauer reviewed and quantitatively analyzed experiments by himself
and others in which people were asked to read text, look at pictures,
and hear words, short passages of music, sentences, and nonsense
syllables. After delays ranging from minutes to days the subjects were
tested to determine how much they had retained. The tests were quite
sensitive--they did not merely ask "What do you remember?" but often
used true/false or multiple choice questions, in which even a vague
memory of the material would allow selection of the correct choice.
Often, the differential abilities of a group that had been exposed to
the material and another group that had not been exposed to the
material were used. The difference in the scores between the two
groups was used to estimate the amount actually remembered (to control
for the number of correct answers an intelligent human could guess
without ever having seen the material). Because experiments by many
different experimenters were summarized and analyzed, the results of
the analysis are fairly robust; they are insensitive to fine details
or specific conditions of one or another experiment. Finally, the
amount remembered was divided by the time allotted to memorization to
determine the number of bits remembered per second.

The remarkable result of this work was that human beings remembered
very nearly two bits per second under all the experimental conditions.
Visual, verbal, musical, or whatever--two bits per second. Continued
over a lifetime, this rate of memorization would produce somewhat over
109 bits, or a few hundred megabytes.

While this estimate is probably only accurate to within an order of
magnitude, Landauer says "We need answers at this level of accuracy to
think about such questions as: What sort of storage and retrieval
capacities will computers need to mimic human performance? What sort
of physical unit should we expect to constitute the elements of
information storage in the brain: molecular parts, synaptic junctions,
whole cells, or cell-circuits? What kinds of coding and storage
methods are reasonable to postulate for the neural support of human
capabilities? In modeling or mimicking human intelligence, what size
of memory and what efficiencies of use should we imagine we are
copying? How much would a robot need to know to match a person?"

What is interesting about Landauer's estimate is its small size.
Perhaps more interesting is the trend--from Von Neumann's early and
very high estimate, to the high estimates based on rough synapse
counts, to a better supported and more modest estimate based on
information theoretic considerations. While Landauer doesn't measure
everything (he did not measure, for example, the bit rate in learning
to ride a bicycle, nor does his estimate even consider the size of
"working memory") his estimate of memory capacity suggests that the
capabilities of the human brain are more approachable than we had
thought. While this might come as a blow to our egos, it suggests that
we could build a device with the skills and abilities of a human being
with little more hardware than we now have--if only we knew the
correct way to organize that hardware.