[ExI] Why “Everyone Dies” Gets AGI All Wrong by Ben Goertzel

[Ben Zaiboc]

> On Mon, Oct 6, 2025 at 10:07 AM <spike at rainier66.com> wrote:
>
>     /> After studying how ChatGPT works, using matrix multiplication,
>     I keep coming back to the same question I had 40 years ago: is
>     human-like intelligence substrate dependent. /
>

Yes. All intelligence is substrate-dependent, as in, dependent on a 
substrate. There's no such thing as a disembodied mind.

I don't see, though, why it wouldn't be substrate-/indifferent/, within 
certain bounds. i.e. the substrate has to have certain properties, but 
there will be a large number of physical systems that can act as 
substrates (beer-cans and string, for instance ��).

>     /> The only reason I can think of is that computer software is
>     deterministic: same input, same process, same output every time.
>     /
>

Well, for one thing that's not exactly true, for another, we wouldn't be 
talking about 'computer software' exactly. Or at least not only.

This relates to a question that has come up again and again in the past. 
"How can a bunch of 'if-thens' have feelings etc.??"

That's like asking how can a bunch of ion channels have feelings, when 
trying to understand a brain.

Or "how can you sit on a bunch of quarks?" when considering a chair.

Levels, it's all about ascending levels of organisation, or what 
sometimes gets called (misleadingly, I think) 'emergence'. H2O molecules 
aren't wet, neural spike trains aren't blue, etc.

You know that it's trivially easy to create a program that will produce 
a different result every time it's run, or results that are not possible 
to predict.

Consider that a brain is made up of membranes with ion channels, which 
are organised into tubes and bags separating different volumes of 
different solutes, which are organised into networks that dynamically 
change, that are organised into a large number of specialised brain 
areas that implement different mental modules, connected to each other 
in lots of different ways, and to the external environment via sensory 
organs and muscles and endocrine glands. How many levels is that? At 
least 5, probably more if you go into detail.

And then look at the same system from the point of view of the 
information it processes. Same principle. Simple signals (individual 
neurons firing or not), organised into more complex data structures 
(neural spikes), up a level (spike trains) up a level (patterns of 
activation of low-level neural networks, where the same pattern of spike 
trains signifies different things in different axons), up a level 
(patterns of activation in different networks), up a level (modules and 
sub-modules switching into different configurations), up a level (groups 
of modules talking to one another)...

You get the idea?

You end up with the Rubaiyat of Omar Khayyam, Beethovens 9th, wavelet 
theory, love, homesickness, awe, curiosity, etc. etc.

Zeroes and Ones in a digital information-processing machine are like 
action potentials in an axon. Go up through several levels of 
organisation and you get spreadsheets and word processors and chat-bots. 
Several more and you get various kinds of AI and systems capable of 
running a human mind.

The thing we don't know yet is how to actually organise these higher levels.

I'm thinking that current AI systems could help with that. Just as we 
use them for drug discovery and creating new enzymes, we could use them 
for 'cognition discovery'. Rapidly trying out different configurations 
of data-processing systems to see which of them show promise for complex 
cognition and consciousness. And for supporting human thought-patterns.

-- 
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20251006/eec3c48d/attachment-0001.htm>