[ExI] My theory of consciousness

[Ben Zaiboc]

On 27/09/2026 04:26, Jason Resch wrote:
> On Sat, Sep 26, 2026, 12:28 PM Ben Zaiboc via extropy-chat <extropy-chat at lists.extropy.org> wrote:
>
>     On 26/09/2026 15:22, Jason Resch wrote:
>
>     > On Fri, Sep 25, 2026, 4:38 AM Ben Zaiboc via extropy-chat <extropy-chat at lists.extropy.org> wrote:
>
>
>     >     Jason Resch wrote:
>
>     >     > from the level of perspective of a program running on a computer, there is an abstract level below that program to which changes of implementation are invisible. A program may be able to read variables in its scope, but it can't tell how those values are physically implemented, it can't read outside it's address space, it can't tell if it's running on an emulator or not. It is trapped within a particular causal structure beyond which certain information has no way to get to or otherwise influence it. This is an epistemic firewall, and it nearly allows us to define what facts can or can't be relevant to the process at the level of the process.
>
>
>
>     >     > By considering the conscious state as a computational process, we can make a similar determination of what facts are able to enter and influence that conscious state, and this sets a minimum level of detail that a brain scan must capture and that a brain emulation must emulate to preserve the same character of consciousness.
>
>
>
>     >     I don't see how the second paragraph follows from the first.
>
>
>     >     You're saying that a process can't access what underlies it, so for example we can't perceive the workings of our neurons, an addition operation has no information about whether it's being performed on an abacus or a computer, etc.
>
>
>
>     > Yes, every process has an underlying interface ( https://en.wikipedia.org/wiki/Interface_(computing) ), so from the point of view of that process, there are only certain things that can make a difference to it.
>
>
>     You're talking about /interfaces/ now. That's confusing, I thought you were talking about processes happening on different levels.
>
>
> I am. Interfaces provide a clear example of boundaries that can separate and isolate levels. Every computable process has such an interface boundary (minimally at the Turing machine that computes it, but often times it exists higher).
>
>
>
>     To clarify, take the example of signals travelling through a network of some kind. This might be underlain by a different type of process like transistors switching on and off, or ion channels selectively allowing specific ions through membranes. The network has no 'knowledge' of the transistors/ion channels. In that example, what is the relevant interface?
>
>
> You have to identify the process before you can answer that question. There may be several independent processes in your example, depending on which signals can ever reach or interact with others. Interest casualty as paramount in defining process boundaries.


Please explain what "Interest casualty as paramount in defining process boundaries" means. I assume it's at least one typo, but can't guess what you mean to say.


>  There are also different levels in the sense that some processes on your example may depend only on high level logical operations on bits, while at other levels of description there are electromagnetic fields and numbers of electrons.
>
> When you identify the process you are talking about, then you can begin to define the causal boundaries around it that can alter and affect that process at the level you've defined it, for the possible states relevant and accessible to that process that are capable of altering it's future behavior.
>
> All this is described in the first few pages of my paper.
>
>
>     >     Then you say that this sets a minimum level of detail that must be captured in a brain scan. How? I don't see how one has any bearing on the other.
>
>
>     > Here is an example in computing that I think illustrates it cleanly. Let's say you found a computer that does your taxes. You inspect this computer at various layers from its circuits and transistors to its programming code up to the high level UI it presents on screen.
>
>
>     > During your investigation you find the tax program is written in Java and runs on a Java virtual machine, which provides a well defined interface. It allows you to run this same program on very different hardware, by replacing the JVM with another suited for that different processor.
>
>
>     > The program can't tell that the hardware beneath that JVM interface has changed because there's no difference it can access that makes a a difference for it.
>
>
>     > In the theory of consciousness I defend in the paper, this inability to make a difference or possible distinction at the layer of a conscious mind process is operating at implies there can be no difference in awareness for that process, its internal behaviors at that level will be unchanged and so I argue it's consciousness must be unchanged as well.
>
>
>     > Now determining where this level is for the brain is not trivial,
>
>
>     Well, that's making the assumption that there is only one such level. That is not true. There are many, many levels.
>
>
> There are many levels in the brain. I've said that. The question is how do we determine the level that is relevant to maintaining a conscious state.
>
>
>     But never mind, you just seem to be arguing for substrate indifference for minds. In other words, presenting a defence against 'carbon chauvinism". Is this really necessary?
>
>
> No, that is an assumption.


What is? That you are arguing for substrate indifference, or that it's necessary?


>  The paper starts from computational functionalism. But while computational functionalism gives us substrate indifference, it doesn't tell us what the substitution level is. That is one of the main contributions of this paper. Please read at least the abstract to get a picture for what this paper provides beyond standard functionalism.


I can't access the abstract. Can you provide a source that doesn't use Google? Or post it here?

When you say 'substitution level', do you mean the level of detail that needs to be modelled?


>      You seem to be expending a lot of words and time on something that's pretty obvious to anyone with any common sense.
>
>
> Have you taken a look at the paper?
>
>     >  but I give a procedure in the paper for how it could begin. My hunch is it would be somewhere above where the reliable computational units/components of the brain start to begin.
>
>     >     Knowing that a mind can't directly access the workings of the neurons that give rise to it doesn't give us any information about how exact an emulation of the neurons must be in order to recreate the mind.
>
>     > But that is part of it. We identify ALL the things a mind is unable to access from its formal description as a process.
>
>
>     I doubt that that is possible. We can identify a lot of things, but surely not ALL of them.
>
>
>     >  All those things sit outside anything that makes a difference to what that process knows, and hence are details that can't affect the mind state. 
>
>
>     There are  things that the mind knows nothing about that DO affect its state (actually most things that change a mind state are going to be things that the mind knows nothing about, I expect). 
>
>
> A substance that changes one's state of awareness in a way noticable to the mind affects what that kind knows (in the sense of that mind knowing a difference in its consciousness). It doesn't mean the mind will know what it is that causes that effect, however, as you rightly indicate.


I think we have a confusion of terminology here.

When you say a mind 'knows' about something, you seem to mean that some change has occurred in the mind, not that there is knowledge about what has caused the change. Is that right?

If so, it might be a good idea to drop the term 'know', as that is what seems to be confusing.

So going with that, as it seems likely, you're saying that a thing that makes a difference to a mind state, makes a difference to a mind state. And that figuring out what such things are, helps us to determine the level of detail of emulation necessary.

So once we figure out the lowest level of the things that cause changes to a mind, we'll know what level of detail to model. Obviously.

This doesn't help us to know what that level is, though.

...

>     >     We are just as blind to the structure of neural circuitry as we are to the activation of individual synapses, 
>
>     >
>
>     > These are empirical problems to be solved to be sure. But note we presently also lack a theoretical basis for deciding which details matter for consciousness.


How is that an extra thing?

We just have to assume that a normally-functioning human mind is conscious, or is capable of it. I don't see what else you can do. That's all we do with other people, and if we can't tell for sure if other biological humans are conscious or not (leaving aside logical absurdities such as philosophical zombies), why would we be able to tell for sure about uploaded humans?

By definition, 'consciousness' is a subjective thing. All we can do is leave it there, not worry about it and focus on the things we actually can find out about, such as behaviour (by which I don't just mean motor outputs, but behaviour of neural networks etc., real or emulated. Basically, anything we can actually measure).


>      Even if we had a complete understanding of neurons and synapses, we still need a theory of consciousness to guide us in determining whether we have to simulate neurons down to the molecular scale or not, or just their coarse spiking behavior, or perhaps we can even take shortcuts and get away with just simulating the high level input and output behavior of cortical columns.


No we don't. A 'theory of consciousness' can't add anything useful unless it's actually a theory of how brains work, in which case we should call it that to avoid confusing people. That's what will guide us. Talking about consciousness is pointless and distracting. You might as well try to create a theory of 'free will' and try to use that to guide you.


>     If we had a complete understanding of neurons and synapses, I don't see that anything else would be necessary. We'd have a set of information that describes a system in great detail, we'd have (presumably) a way of emulating that system to an arbitrary level of detail, so we just start with a coarse level of detail in the emulation, and keep increasing it until the system behaves as expected (i.e. produces the correct outputs for a set of specified inputs). 
>
>
> Yes but correct behavior is always indexed to some level of fidelity. Where and how do you draw the line at defining the level you are checking again for correctness?
>
> The range of possible behavioral outputs will always be strictly less than the range of internal states the mind can be in. So what you must index on is the fidelity of internal states rather than just the outputs.


("index on" = concentrate on?)

Yes, so long as by 'internal states' you don't mean subjective experiences, because you can never measure those.
We can, however, look at how biological brains behave. What patterns of neural activity occur under certain circumstances, and compare them with the patterns of simulated neural activity occurring in the upload under the same circumstances. Of course there will be the question of how many tests do you perform before you're satisfied, what variety of circumstances do you consider, etc. I expect that will be worked out as we go along, and a standard set of tests would be arrived at.


>     Much like was done with the fruit fly connectome recently. If it didn't behave like a fruit fly, we would have known that it didn't work.
>
>
> A lot of shortcuts were made with that emulation. I would have a lot of doubts whether its conscious state of this emulation was a good approximation of the original fruit fly's awareness.


You can have doubts about the conscious states of anything or anybody, till the cows come home. You'll never satisfy them as long as you keep concentrating on 'consciousness', as long as we can't define it in objective, testable terms. Which is, as far as I can see, always.

If shortcuts were taken in the fruit fly emulation, and it still behaves like a fruit fly, that's a good thing. It suggests that the same kind of shortcuts could be taken with human brain emulations, with similarly good results.


>  I don't know what a 'theory of consciousness' would even look like. It sounds like the sort of thing that leads down a rabbit hole. Surely a theory of how minds are produced would be more relevant. Leave 'consciousness' out of it, I say. Nobody really knows what the word means anyway. I suspect it's something that can't really be defined.
>
>
> To me, my having a mind implies being conscious.


Hmm, maybe.


> So when you say make a theory of minds but don't bother with a theory of consciousness, that system doesn't parse meaningfully to me.
>
> A complete theory of consciousness would tell us which sorts of systems qualify as minds.


No, a theory of minds would (more accurately, a theory of how brains work to produce minds).
Having a mind implies being conscious to me as well (at least for humans), but so what? I'm not even sure what the word actually means (I normally use it to mean the same thing as 'awake'), I'm just using it because you are. Who knows, consciousness could feel like a completely different thing to each person. Does my consciousness feel the same as yours? We will never know, and it doesn't matter.


>     The thing to concentrate on is behaviour, not some elusive (probably forever) theory of consciousness. If we can get a brain emulation to behave the same as the original brain, we've succeeded. It's that simple.
>
>
> Internal behavioral fidelity is the anchor I use in the process my paper describes for identifying the level of fidelity required to reproduce a mind. 


Fine.


> But I think this involves more than just checking motor neuron output. 


So do I.


> I think it would involve tracking distinctions available to areas in the brain including the visual cortex, language centers, and many typically unconscious processes as well.


Yes, as long as we can measure and record those things.


> Essentially all physical processes consist of a directed acyclic graph (DAG) of causal events and information flows through time. Determining the boundaries of differences available to a process involves identifying a subgraph of these events, by tracing causes backwards until a boundary is reached. 


I have no idea what that means, but surely when you say "all physical processes consist of a directed acyclic graph ...", you mean "all physical processes can be represented by a directed acyclic graph ...". After all, the physical processes are the real things we are talking about, the graph thingys are just one way of trying to understand them.


>     > We might accidentally ressurect someone whose consciousness is not quite the same as it was before.
>
>
>
>
>
>     Accidentally?
>
>
>
>     Taking 'consciousness' to mean 'mind', I can't see that anyone who'd been uploaded would have, or want to have, exactly the same mind-state as before uploading. Just the knowledge that you've been uploaded would make a big difference, and there wouldn't be much point to uploading someone without their knowledge. I expect they'd soon find out, though, and then they'd be different. Personally, If I woke up one day to find I had been uploaded, it would make a /huge/ difference to me!
>
>
>
> Here I'm talking about preserving the same qualities of experience (qualia), to the same level of fidelity and richness they had before, rather than any particular set of memories.


How on earth could you tell that you were preserving the same qualia? They are subjective, not available to any kind of measurement and recording. You're effectively saying there's a way of telling if my experience of eating a bacon sandwich is the same as yours. If you mean the individual could tell if their experience of eating a bacon sandwich pre- and post- uploading was the same, how could they tell, except by relying on their memories? So preserving memories would be essential, then you could ask the question, and if the answer was "no, actually, this bacon sandwich tastes of cheese", what does that tell us? That the taste routines need fine-tuning, or that the upload has failed? Or the memories have been corrupted, but the basic mind is ok? Or what, exactly?

Certainly we'd know that something was wrong, but how serious it was would have to be determined by more tests. "What's your name?" ... "Napoleon Bonaparte" would be a red flag. Or a confirmation that their sense of humour was intact.

I'd expect that testing if an upload has worked or not would be a lot easier than this. Failure would be more likely to result in things like catatonia, symptoms of brain damage, severe memory loss, radical changes in personality, sensory defects, and so-on. If the subject responds normally to a range of normal stimuli, remembers who they are, and is capable of initiating things on their own, properly controlling their virtual or robotic body, etc., I'd say that the upload was likely a success. Problems like seeing the wrong colours, tasting cheese when bacon is expected, etc., will just be matters of fine-tuning the system.
Leave questions like "but are they really the same person?" to the philosophers (who will never agree on an answer anyway).
-- 
Ben