More theory of mind >
More ontology (being) >

Towards a Theory of Qualia

The nature of sentience

Consciousness, experience, sentience, call it what you will, its nature has been approached in different ways by different cultures and at different times. Its location has been attributed to the blood, the stomach, the heart and suchlike bodily locations besides the brain. For most of Western history it has been the domain of religion and metaphysics. Through the nineteenth century the focus moved on the one hand towards brain function and neurology, on the other towards an underpinning by the unconscious or subconscious mind. Meanwhile, several hundred years BC and roughly contemporary with the Greek philosopher Plato's realm of ideas, Eastern sages had remarked that it was by its nature delusory. What exactly was being deluded was never made clear – one recent mystic has described it aptly as "the enjoyer of the self" but, as ever, left its nature a metaphysical mystery. Through the twentieth century these more scientific approaches from East and West began to coalesce.

Consciousness was long thought to be a property of the physical brain. But you cannot simply look at a neuron, or a network of connections, and say what it is experiencing. Consciousness does not lie in the physical structure of the brain. The rise of neuroscience and the discovery of the complex electrochemical signalling that goes on, led to its identification with consciousness. But even if you look at the electrochemical signals and their patternings in the minutest detail, you are none the wiser, a signal pattern in one area looks much like a signal pattern in any other. Consciousness does not lie in the pattern of signals within the brain. The emergence of information science brought a new abstraction from the physical brain, focusing this time not on metaphysical or even physical notions but on the information being processed. What matters is the semantic information carried by those patterns. But even there, most of them carry unconscious information. It is only certain patterns in certain areas, which carry highly integrated semantic meanings, that manifest as conscious experience. One twenty-first century writer has said that "Consciousness is what information feels like when it has reached a certain level of complexity."

In all this, philosophers have sought to identify the distinguishing characteristics of sentience. On the one hand, everybody by definition has their inner experience. But on the other hand, that is where it stays. We may assume that it correlates exactly to certain patterns of electrochemical signalling within the brain and we are beginning to gain some loose idea of which signals correlate with which meanings, but the inner experience itself remains elusive. It is hard to do good science on something which is inaccessible to other scientists. Indeed, philosophers call this the "hard problem" in the theory of mind.

Because of this, some scientists (and philosophers) with positivist leanings deny the validity of such inner experience as a subject of scientific study, even to the point of denying the fact of its existence. A sentient being who denies the quality which defines them as sentient (as opposed to being a mere robotic information processor) is not being rational. One cannot argue with such blunt and vehement, dare one say fundamentalist rejection, only disagree profoundly.

Some writers have likened consciousness to a kind of magical projection of the brain patterns onto a cinema screen (called a "Cartesian theatre", after the philosopher René Descartes). In computing terms, which are closer to AI than are movies, one might describe the modern psychological view of the self as an object model, the Self Object Model or SOM. Web developers will immediately know what I mean by this. Most web browsers construct what is called the Document Object Model (DOM) of the page they are viewing. The DOM is then "rendered" into visual elements so that it can be viewed in a "window". Similarly to the Cartesian theatre, one may regard qualia as the rendered output from the SOM and the conscious mind as the window in which they are displayed.

Whether cinema or browser, the problem remains the same. The delivery system has no conception of what the final view will "look like" to a human viewer. The film director or web developer only knew because they were human, their experience had to come from outside the finished system. Neither the film reel nor the SOM, the projector nor the renderer, has any knowledge of what it will feel like to the conscious self.

The executive dashboard

It seems to me that an executive dashboard is an apt analogy for the conscious mind. These dashboards are software products which run on the personal computers of senior managers in large organisations. They receive business data from the various systems on the corporate network, process it to extract the salient highlights, and then display these on the screen. Thus, just like the dashboard of a car, at a glance the busy executive can see how the business is running, and any problem areas which may be occurring.

This appears to be exactly what the conscious area of the brain is doing. Most of the business of monitoring and controlling an active body goes on behind the scenes, deep within the network (in this case, a network of neurons). Only the most salient information is passed to the conscious mind, where it is perceived as qualia.

A particular feature of the dashboard is the analytics it runs on the data. These will include models of how the corporate processes are structured, how they relate to each other, how they should be performing and how they are performing. This looks remarkably like the virtual model of the self which the human brain is known to maintain.

But there is one thing which is in neither the business model nor the information to be displayed, and that is the layout and styling of the display itself, even the very fact that it is a diplayed image and not a log file or something. The equivalent in the conscious mind is the perceived or experiential quality of the information stream passing through it.

The place of qualia

In the quest to define that essence of sentience, philosophers talk of the quale (pronounced "kwar-lay") of a given experience – the redness of red, for example. Its quality is, famously, unaccountable to modern science: experiment gives us the energy, the wavelength, the frequency, the wave equation, the associated brain signal when we see or remember a patch of red, but not what it actually feels like to experience that redness. Thus, qualia are by definition those aspects of cognitive processes which cannot be measured or communicated in any way whatsoever. They are wholly personal and private. Conscious experience comprises an endless stream of such qualia (the plural of quale), but it is embarrassing how little we really know about them. If you still find the idea of qualia somewhat specious and mystical, you might like to check out The Layered Mind, which tries to set them in a rational context.

Some materialists have tried to brush the whole thing off as an "emergent property" of the information. Emergent properties are of course common in the physical world. For example if we lump a load of fundamental particles together they will have an average relative motion to each other, which we call their temperature. Thus, temperature is an emergent property of lumps made up from smaller stuff. Similarly, consciousness is seen as an emergent property of an information flow which has achieved a certain level of complexity. But emergent properties are objective, they can be characterised and measured by scientists in the usual way. How do you know that yesterday's qualia are the same as today's, or that mine are the same as yours? Is my experience of redness fundamentally different from yours? What about a crow or a large octopus, both creatures acknowledged by science to display the signs of consciousness, how do they experience redness? Answers to such questions are not, in general, falsifiable. Or at least, not through the scientific method. Yet science is founded on the assumption that if you make the same experiment yesterday and today, or if you and I both make it independently, then the results will always be the same. No such assumption is assured with qualia. Thus, qualia and conscious experience cannot be mere emergent properties, in this sense, of a clever computer.

One might think from all this that qualia can have no impact on the physical world. After all, if science cannot characterise them or tell one from another then what use are they as a rational concept? The brain signals contain all the information that can be known. The Cartesian theatre was thought up as a way to illustrate this view; the images on the screen have no influence whatever on the film itself or the projector and screen showing it. But, despite being taken seriously by many philosophers, this picture is trivially false. If qualia had no part to play in the physical world then the words you are reading right now would never have been typed. My qualia have grossly affected this document and the physical display you see before you.

This phenomenon is soewhat analogous to an executive dashboard being provided with control buttons, so that the user can make decisions on say priority calls in a difficult situation and communicate those decisions to the workforce (this analogy cannot be pushed too far, for example there is no executive humunculus within the brain!).

If you disagree with this understanding, as some materialists claim to, then you have simply not understood the definition of what a quale is: it is just a private element of subjective conscious experience which is inaccessible to science. Suffice to say that to refute conscious experience at all, as behaviourists such as B.F. Skinner once did, marks you out as a mid-twentieth century anachronism. And if you don't call such an element a quale then you must think up something else to call it. I cannot emphasise this point strongly enough. Once you accept the fact of subjective conscious experience, then of necessity you accept qualia or something differing only in name. Any denial is just defensive and untenable sophistry, typically built around a distorted usage of the word. It is like denying that the data generated by an executve dashboard have any kind of user interface at all. To be fair, some of the more zealous proponents of qualia have some pretty distorted theories about them. But you should not refute an idea just because someone else has expressed it badly. Somehow, despite being inaccessible to science, qualia can and do pervade the nature of human society. If that is not to be a self-defeating concept, then how can it possibly work?

First of all, it is not necessarily an impossible position. Many physical phenomena of interest, from black holes to parallel universes, are inherently inaccessible to science and are understandable only through their effects on the world around them. Similarly, we learn something about qualia from their physical effects. For example, it is through their physical effects on other people that we observe that they are sentient beings like ourselves. One the principle has sunk in, I am sure there is more to learn. For example, that is probably why you are reading this at all.

But, other than the scientist's model-predict-observe, is there any other rational method by which we can get closer to the truth? That is really what this essay is about. What follows are a series of observations and ideas which may or may not begin to shed light on the Hard Problem.

Symbolic or semantic information

Mediating between the objective brain activity and the subjective experience is the semantic information carried by the brain activity. For example if the eyes see a red object, then the visual cortex passes the appropriate signal to the conscious area. This area recognizes that the signal represents redness, i.e. that it carries the semantic meaning of redness, and produces the sense or "quale" of redness in the conscious mind. When I perceive or recall redness, there is an exact correlation between the associated signal pattern, its semantic meaning of redness and the quale which I actually experience.

Consider a butterfly which evolves symbolic "eye spots" on its wings through a blind process of natural selection. Their effect is to trigger, in the brain of a small predatory creature, the signal – the symbolic information – that it is looking at the face a much larger predator. In this sense, we can say that the eye spots themselves are symbols for the eyes of a large predator. The key point is that they are lying, their symbolic meaning is untrue. Further, there is no reason to suppose that any of these creatures has any experience of consciousness (they might have, but that is a different question). The untruth has evolved through the blind processes of natural law and is wholly a part of the physical world.

This level of symbolic or semantic information resides entirely in the observable realm of science, of physics, biology and information theory. Many theories of mind fail to recognise this and present all sorts of weird and wonderful arguments that metaphysics must be involved. But no, whatever we dream of in our philosophising must have an objective signal pattern to carry it through and one can isolate no conceivable thought from that rigour (the subject of that thought is of course a different matter). A favourite example of this is provided by memory. It is often suggested that we might carry our memories beyond the death of the brain, or even that we might be able to "remember" snatches of a future that has yet to happen to us. But memories are held in the brain. Destroy or disturb that part of your brain and you destroy or disturb the memory. Stimulate it and you recall the memory. This is so well known as barely to need remarking on. Any theory of memory must perforce be a physiological theory of the brain. If anything survives beyond death or reaches back from the future, it is certainly not the brain patterns on which memory rides.

Continuity of qualia

But there is one aspect of such thoughts that we have isolated from the brain itself. The subjective quality of a thought is not directly encoded in the thought itself. The hard problem is rooted in the fact that there seems no way that we can reach into another ordinary person's mind to discover how they experience the world. Is the quality of redness which they experience the same as mine? Their exact brain signal is a little different from anybody else's because everybody's brain is wired up uniquely. Even if we learn to extract details of the semantic pattern carried by those signals, as we are slowly beginning to do, there is no way of knowing how that signal appears to the subjective consciousness.

The problem appears even within a single individual. Although I have many memories of redness and blueness, were I to wake up tomorrow with my red and blue qualia swapped round, not only would my immediate sight swap them round but so too would all my memories: sight of a red object would invoke the same quale as my memories of redness. Today would be just like every day I can remember and I would carry on, blithely unaware of the trick just pulled on me.

I'd like to suggest here that the barrier is not totally impregnable, some cracks do exist and we can catch glimpses of what may lie beyond.

I hold the view, common enough in the East for the last two and a half centuries but still struggling for ascendance in the West, that there is no atomic "self" inside the brain, just an endless river of experiences. The inner "me" is neither more nor less than that flow. A semantic model of "self" has proved useful to the brain as a kind of coathangar for all those semantic correlates of experience, as that actor who does the experiencing. But is that assumption of an actor anything more than a convenient fiction?

In so far as "I" exist beyond a blob of grey jelly and its busy patterning of symbols, all I can actually perceive is a stream of qualia. Certain characteristics of this stream may be noted.

First, there is a continuity over time. My experience of redness as a child was the same as it is now. Of course, the underlying signal pattern is the same, but it is instructive to observe that this is reflected in the constancy of the associated quale. Everybody reports this same continuity. Of course, changes in brain function can change the information and hence the qualia, but for any individual at any given moment the same basic information always produces the same quale.

Descartes had a similar problem in deducing the existence of the outside world from his stream of consciousness. He posited a malignant demon who had created him, complete with false memories of a lifetime, only seconds earlier. Although there is no solid logical proof of its absurdity, he nevertheless argued that it was so extremely unlikely as to be not worth taking seriously. Thermodynamicists make a similar argument over the possibility that a drop of ink, once mixed into a glass of water, might spontaneously reform. I offer the same argument for qualia: that some malignant demon might be swapping my red and blue perceptions in the night, so that next day I not only experience them the other way round but also experience my memories of them the other way round as well, is just too outlandish to take seriously. In any theory of qualia we should at least assume the fundamental principle that what was so yesterday is still so today and will still be so tomorrow.

Another problem lies in the reliability of distinctions between one quale and another. In synaesthesia, the sufferer's brain sends signals to the wrong place and what should be say a thought or a sound might gain visual qualities. Seeing letters, number or words in particular colours is well enough known. More bizarre are recently documented cases such as a lady who "sees" any calendar as a numbered wheel projecting from her chest and, when figuring out say what day of the month last Thursday was, here eyeballs can be tracked looking to that particular part of the wheel, from where she then reads out the answer. Such cases obviously present anomalies in brain function. The key observation for us is that the qualia follow the logic of the customary brain signals with absolute precision. The sufferer is well aware that an auditory or mental stimulus has produced a visual response, there is no confusion in their minds about that, and the qualia too are free of such confusions. This demonstrates a stronger consistency among qualia in that each aspect of sense and thought has its own unique characteristics which may be identified by their qualia with as much precision as any imaginable brain scanner would ever be able to match. This establishes another fundamental principle, that wherever certain qualia were once distinct, they will always be so.

These two observations between them establish that several of the core principles of logic apply to qualia – we can study qualia rationally, build rational theories about them, and not have the rug pulled from under our feet by some sudden inconsistency. Some of those theories may even be falsifiable. This marks a first breach of the idea that qualia are utterly unapproachable to science and reason.

There is also the question of continuity between individuals. How do I know that, seeing the same red as you do and having the same associated brain pattern recorded to indicate that we share the same symbol for red, we necessarily share the same experience of redness? Other than another appeal to the malignant demon argument, there may be something stronger here. Physically, the human brain is built in distinct left and right halves, linked by a mighty highway called the corpus callosum. Occasionally, whether by accident or design (the operation has been performed as a desperate last-ditch defence against severe epilepsy), the corpus callosum is severed, creating an almost complete separation between the two halves. Such a split-brain individual can show some extraordinary characteristics. One half, usually the left, contains the specialist centres for language and can communicate with the people around. The other half must remain silent but, like a deaf mute, can still get by after a fashion with basic hand gestures, body language and the like. Each may develop a distinct personality. For example when getting dressed, one half may wish to wear red, the other blue. Scenes reminiscent of the film Fight Club have been observed, where one arm struggles against the other to don the garment of choice. Typically, one garment is eventually put on and the other added over it. Similar conflict has been seen where one half wants a cup of tea, the other coffee. There have also been cases where the corpus callosum has merely been damaged or only partly severed and the split-personality effects similarly limited. There is clearly a continuity of medical conditions from normal connection to complete disconnection. What is the significance of all this for qualia? I would suggest that the key here is continuity. It goes against the previous arguments to suggest that the qualia in one half might suddenly break the continuity of a lifetime, just because it could no longer communicate with the other half as effectively as before. In a partially-connected brain there has never been any suggestion that the qualia of experience diverge between the two halves and if such a thesis were to be proposed some tortuous and intricate embellishment would be needed to explain how a partially-connected brain might reconcile the inevitable clashes of different qualia from each half. No, Occam's razor puts forward the thesis that there is a continuity of qualia between the two halves, the two personalities, no matter how separated they become.

From here it is but a short and natural step to the proposition that qualia are, in essence, consistent across all individuals of a given biological species. Perhaps one day the first-hand testimony from a recipient of a half-brain transplant will provide the evidential base to prove the case one way or the other. But for now, the burden of proof would seem to lie with those who would seek a convincing case for any differences.

That may not be as impossible as I perhaps paint it. Take for example people blind from birth. They have spare brain capacity, normally used for visual processing. This capacity tends not to be wasted but gets hijacked and used for language instead. So the brain signals associated with, say, hearing someone say "Hello" are different between blind and sighted people. Does this mean that they necessarily experience different qualia? On the other hand, the same meaning is attached to both experiences, so does this instead mean that they both necessarily experience the same quale? Given that the brain signals differ but the meanings do not, what does it mean to suggest that the qualia might or might not be consistent between the two individuals? Consistent with what? There is work to be done here.

Qualia space

The inaccessibility of qualia to direct scientific measurement may not be the showstopper it seems. Other areas of science have their no-go areas and have evolved ingenious techniques to tease out what might be going on in there. None is more obscure and challenging than the wave aspect of matter and energy. In the standard view of quantum mechanics, the Copenhagen interpretation, the nature of the wave is explicitly treated as inaccessible: one may calculate possibilities and from these make a best guess at what an experiment might find, but it is in the nature of the beast that the quantum does not make up its mind what exactly it is up to until you measure it. Nevertheless, we can discover certain properties, most of which turn out to be utterly weird. A quantum may pass through two adjacent slits at the same moment yet arrive in only one place, we can send a particle's momentum down one path and its spin down another, we can entangle two quanta and use this to transmit information faster than the speed of light, a cat may be both alive and dead at the same time, and so on and so on. Perhaps we can go some way towards such sideways approaches to qualia space and deduce a little more about it. And we must be prepared for it to turn out to be as weird as the world of the quantum.

Probably the first thing to understand about qualia space is that its characteristics need have little or nothing to do with the relationships between the qualia themselves, any more than the street you live in is related to what you ate for breakfast.

The four-dimensional spacetime on which Einstein built Relativity is called Minkowski space. One can picture it as a frozen landscape in which time points in one direction and space lies at right angles to it. Imagine taking a reel of film and unrolling it across the landscape, carefully aligning it with time, creating a timeline. The human mind has been found to proceed in a series of jumps and pauses in time, much like a film being taken, so the analogy is an apt one. Let the film represent your life, the timeline running from birth to death. Each film is multi-dimensional, in that each human sense or capacity is an independent dimension in its own right. Now, scatter a multitude of films across Minkowski space, each on its own timeline. Your and those of your fellow humans are in glorious surround-sound widescreen high-definition colour. Those of your dog, horse or parrot are still in colour but perhaps shot in 35 mm. Remember those hamsters, the sparrows pecking at the bird table? Silent 12 mm monochrome, the Keystone Cops of consciousness. The finest threads might represent an awareness of little more complexity than the Morse code.

Looking at qualia space, we see immediately the continuity of qualia along each timeline and between timelines of the same grade of film. But everything else about qualia space is blank, invisible, like the mist rising from the river in a Taoist landscape. Is there anything else we can discover about it? Probably the first thing is that the assumption that qualia inhabit a Minkowski space is unjustified, that was just a picture to help grasp the idea. The space separating the timelines might be anything or nothing. The picture is drawn here only because the physical world inhabits a (relativistic) Minkowski space. Indeed, everything else about the physical world – the structures formed by the qualia, the frame rate, the fact that some films are higher quality than others, even the division into distinct timelines – are all overlaid on qualia space, none is an intrinsic property of it. For example, does time pass in qualia space, or is it set as solidly as Minkowski spacetime and the passage of time is a mere interrelationship between the qualia themselves? At present there seems no answer.

I would suggest that we can at least summarise a small number of observations about qualia space. It exists in some form or other and obeys logical laws which can only be investigated to a very limited extent. It is multidimensional beyond our direct experience – even if you happen to have the electric senses and motor systems of an electric eel, or of a shrimp which sees in a dozen primary colours. While these and other aspects of its nature remain obscure, perhaps innately so, it is evident that its laws are sufficiently rich that, in common with the laws of physics, upon them may be overlaid every thought or state which might ever conceivably occur in a sentient mind.

Another property, discussed earlier, is the ability of qualia space to interact with the physical world. Given the strict correlation between brain signal, semantic information and quale of experience, it has been suggested that the requirement to obey the laws of physics must drive the correlation one way with the physical signal, such as reading this essay, causing both the information and the quale. The conscious mind is then seen as a kind of film screen, passively displaying as qualia the information projected onto it by the active projector of the brain. But a moment's thought reveals the incompleteness of this approach. The reason that I am physically typing this essay and you are scanning it with your senses is because, and only because, qualia exist. In this light, the proposition that qualia can also affect events in the physical world becomes self-evident. Thus, we can see that the interaction between qualia, semantic information and the material world is very much an omnidirectional phenomenon. The correlation between universes is obviously not complete; we are confident that not all material objects are conscious or carry semantic information, and that not all possible information structures have yet been expressed via physical or mental media. The proposition naturally follows that not all qualia have yet been instantiated in experience or symbols. Indeed, the story of animal evolution has been one of constant instantiation of new kinds of qualia as senses and brains have evolved and there is no reason to think that this process has ended. Qualia space is bigger than we have yet seen.

The origin of the qualiverse is as mysterious as the origin of the physical universe. Did it suddenly spring into existence to house the correlate of the first self-model to became sufficiently complex and sophisticated? Or had it been there all the time, as a kind of blank canvas awaiting its first artist? Can it exist in any sense without a physical substrate to carry the semantic correlates?

Conclusion

I have tried to make my case as evidence-based as may be. Some of the evidence may be found only by considering one's own internal experience and is not susceptible to the tools of the laboratory. But then, it is a fundamental characteristic of the scientific method that every tool must be manipulated and examined by a conscious observer. Indeed, the role of the observer is a foundational aspect of both Relativity and Quantum theories. The entire edifice of modern science is a tertiary structure, built upon a secondary belief in an external world, in turn built upon a primary inner world of experience. Thus, where my evidence here must perforce come from that same inner experience, it shares the exact same foundation stone as does any rational discipline, including science. And in exactly the same way, any scientist can look into their own stream and verify for themselves what I find in mine. This, again, is where science starts. Whether one regards the present analysis as an apologist justification of metaphysics or its shrinking away under the glare of rational science is all the same to me, in fact I regard the distinction as meaningless if not positively harmful: the "science vs religion" debate must surely be one of the most sterile and vacuous of modern times.

Where might all this take us? I have suggested that in some fashion there exists a rationally ordered "universe of qualia" going beyond not only the haphazard collection of individual creatures who make up sentient life on Earth but also the spacetime they inhabit. Like the physical universe, most of it is empty. Each of us may in this sense be regarded as the intersection of our own experience with qualia space. I have explored one way of structuring the whole of existence as The Three Levels of Being, where I treat logical constructs as a third such universe. I am hopeful that modern discoveries across many disciplines may soon be bringing real progress within our grasp. In particular, today's computer systems are already nibbling away at artificial intelligence (although what is touted as AI by the vendor may not yet be what a philosopher means by it). In robotics, models of the self as a physical entity are beginning to take embryonic shape. It is only a matter of time before the information processed by such systems gains sufficient sophistication to become aware, true artificial minds. Beyond that lies the ability of these systems to improve their cognitive abilities, to become "superintelligences". They might simulate or virtualise lesser minds to tackle individual tasks, communicating "telepathically" within their parent supermind. Such a twinkling, ever-changing network across the dimensions would shine out in qualia space like a firework display. To them, we will become little more than those slender snail-like trails of Morse code, but perhaps they will still be kind enough to tell us more. Or perhaps not.

One might even entertain the notion that qualia space is, in and of itself, the ultimate supermind. But I doubt that possibility will ever illuminate opinion as to whether rolling phrases such as, "In Whom we live and move and have our being", are insightful truths or metaphysical mumbo-jumbo. Human nature never changes.

Updated 21 Jun 2022