Error message

Warning: Use of undefined constant openphil_social - assumed 'openphil_social' (this will throw an Error in a future version of PHP) in openphil_social_block_block_view() (line 90 of /var/www/html/openphil/live/sites/all/modules/custom/openphil_social_block/openphil_social_block.module).

A Software Agent Illustrating Some Features of an Illusionist Account of Consciousness

Published: October 2017, Updated: November 2017

by Luke Muehlhauser1

One common critique of functionalist and illusionist theories of consciousness2 is that, while some of them may be “on the right track,” they are not elaborated in enough detail to provide compelling accounts of the key explananda of human consciousness,3 such as the details of our phenomenal judgments, the properties of our sensory qualia, and the apparent unity of conscious experience.4

In this report, I briefly describe a preliminary attempt to build a software agent which critics might think is at least somewhat responsive to this critique.5 This software agent, written by Buck Shlegeris, aims to instantiate some cognitive processes that have been suggested, by David Chalmers and others, as potentially playing roles in an illusionist account of consciousness. In doing so, the agent also seems to exhibit simplified versions of some explananda of human consciousness. In particular, the agent judges some aspects of its sensory data to be ineffable, judges that it is impossible for an agent to be mistaken about its own experiences,6 and judges inverted spectra to be possible.

I don’t think this software agent offers a compelling reply to the critique of functionalism and illusionism mentioned above, and I don’t think it is “close” to being a moral patient (given my moral intuitions). However, I speculate that the agent could be extended with additional processes and architectural details that would result in a succession of software agents that exhibit the explananda of human consciousness with increasing thoroughness and precision.7 Perhaps after substantial elaboration, it would become difficult for consciousness researchers to describe features of human consciousness which are not exhibited (at least in simplified form) by the software agent, leading to some doubt about whether there is anything more to human consciousness than what is exhibited by the software agent (regardless of how different the human brain and the software agent are at the “implementation level,” e.g. whether a certain high-level cognitive function is implemented using a neural network vs. more traditional programming methods).

However, I have also learned from this project that this line of work is likely to require more effort and investment (and thus is probably lower in expected return on investment) than I had initially hoped, for reasons I explain below.

How the agent works

The explanation below is very succinct and may be difficult to follow, especially for those not already familiar with the works cited below and in the footnotes. Those interested in the details of how the agent works are encouraged to consult the source code.

The agent is implemented as a Python program that can process two types of text commands: either an instruction that the agent has “experienced” a color, or a question for the agent to respond to.

Each color is identified by its number, 0-255, such that (say) ‘20’ corresponds to my quale of ‘red,’ ‘21’ corresponds to my quale of something very close to but not quite ‘red,’ and so on.8 Upon being “experienced,” each color is stored in the agent’s memory in the order it was experienced (color1, color2, etc.).

To respond flexibly to questions, the agent makes use of the Z3 theorem prover. Upon receiving any question, all axioms (representing the agent’s knowledge) are passed to Z3, which serves as the agent’s general reasoning system. Z3 then returns a “judgment” in response to the query. These “phenomenal judgments” are meant to instantiate, in simplified form, some (but far from all) explananda of human consciousness.

First, consider judgments about colors — a familiar kind of phenomenal judgment in humans. The software agent also makes judgments about colors. Specifically, the agent judges that each color it has experienced has some absolute ‘value’ (red experiences are intrinsically red and not, say, blue), but (like a human) it doesn’t know how to say what that value is, other than to (e.g.) say whether a color is more similar to one color or another (e.g. red is more similar to orange than it is to blue). This is because the agent’s reasoning system doesn’t have access to the absolute values (0-255) of the colors it has seen (even though they are stored in memory), and it also doesn’t “know” anything about how its reasoning system works or why it doesn’t have access to that information. Instead, it only has access to information about the magnitude of the differences between the colors it has seen. Thus, when asked “Is the 1st color you saw the same as the 6th color you saw?” the agent will reply “yes” if the difference is 0, and otherwise it will reply “no.”9 And when asked “Is the 1st color you saw more similar to the 2nd color you saw, or the 3rd color you saw?” the agent is again able to reply correctly. But when asked “Is the 4th color you saw ‘20’?” it will respond “I don’t know,” because the reasoning system doesn’t have access to that information. This is somewhat analogous to Chalmers’ suggestion that ineffabilty is an inevitable consequence of information loss during cognitive processing, and our lack of direct cognitive access to the facts about that process of information loss.10

This agent design naturally leads to another phenomenal judgment observed in humans, namely the intuitive possibility of an inverted spectrum, e.g. a situation “in which strawberries and ripe tomatoes produce visual experiences of the sort that are actually produced by grass and cucumbers, grass and cucumbers produce experiences of the sort that are actually produced by strawberries and ripe tomatoes, and so on.” For our purposes, we imagine that the agent has spoken to other agents, and thus knows that other agents also talk about having color experiences, knows that they seem to believe the same things about how e.g. red is more similar to orange than to blue, and knows that they also don’t seem to have access to information about the ‘absolute value’ of their color experiences. In that situation, the agent concludes that inverted (or rotated) spectra are possible.11

Finally, another phenomenal judgment familiar to humans is the judgment that while one can be mistaken about the world, one cannot be mistaken about what one has experienced. In the software agent, this same judgment is produced via a mechanism suggested by Kammerer (2016), which Frankish (2016b) summarized this way:

[According to Kammerer’s theory,] introspection is informed by an innate and modular theory of mind and epistemology, which states that (a) we acquire perceptual information via mental states — experiences — whose properties determine how the world appears to us, and (b) experiences can be fallacious, a fallacious experience of A being one in which we are mentally affected in the same way as when we have a veridical experience of A, except that A is not present.

Given this theory, Kammerer notes, it is incoherent to suppose that we could have a fallacious experience [i.e. an illusory experience] of an experience, E. For that would involve being mentally affected in the same way as when we have a veridical experience of E, without E being present. But when we are having a veridical experience of E, we are having E (otherwise the experience wouldn’t be veridical). So, if we are mentally affected in the same way as when we are having a veridical experience of E, then we are having E. So E is both present and not present, which is contradictory…

For details on how this mechanism is implemented in the software agent, see the code.

Some lessons learned from this project

In my 2017 Report on Consciousness and Moral Patienthood, I listed a more ambitious version of the present project as a project that seemed especially promising (to me) for helping to clarify the likely distribution of phenomenal consciousness (and thus, on many theories, of moral patienthood).12 I still think work along the lines begun here could be helpful, but my estimate of the return on investment from such work has decreased, mostly (but not entirely) because my estimate of the cost of doing this kind of work has increased. In particular:

  • Implementing the proposed mechanisms (e.g. from Chalmers and Kammerer) requires a large amount of “baggage” in the code (e.g. for using a theorem prover) that doesn’t illuminate anything about consciousness, but is required for the code to be set up so as to implement the proposed mechanism. This “baggage” requires substantial programming work, and also makes it more cubersome to write (and read) a full explanation of how the program implements the proposed mechanisms.
  • Before the project began, I guessed that in perhaps 20% of cases, the exercise of finding a way to program a suggested mechanism would lead to some interesting clarification about how good a proposal the mechanism was, e.g. because the proposed mechanism would turn out to be incoherent in a subtle way, or because we would discover a much simpler mechanism that provided just as good an explanation of the targeted explanandum. However, based on the details of our experience implementing a small number of mechanisms, I’ve lowered my estimate of how often the exercise of finding a way to code a proposed mechanism of consciousness will lead to an interesting clarification.
  • A project like this would benefit greatly from career consciousness scholars who are more steeped in the literature, the thought experiments, the arguments, the nuances, etc. than either Buck or I are.
  • I don’t think a program which implements three (or even five) mechanisms will be enough to learn or demonstrate the main thing I’d hoped to learn/demonstrate, namely that (as I write above) “the agent could be extended with additional processes and architectural details that would result in a succession of software agents that exhibit the explananda of human consciousness with increasing thoroughness and precision [such that] perhaps after substantial elaboration, it would become difficult for consciousness researchers to describe features of human consciousness which are not exhibited (at least in simplified form) by the software agent, leading to some doubt about whether there is anything more to human consciousness than what is exhibited by the software agent…”
  • Even if we took the time to implement (say) 10 proposed mechanisms for various features of consciousness, it’s now clear to me that a compelling explanation of those mechanisms (as implemented in the software agent) would be so long that very few people would read it.

For these reasons and more, we don’t intend to pursue this line of work further ourselves. We would, however, be interested to see others make a more serious effort along these lines, and we would consider providing funding for such work if the right sort of team expressed interest.

Appendix: Notes to users of the agent’s code

This appendix is written by Buck Shlegeris, who wrote the code of the software agent, which is available on Github here.

In this appendix, I explain some of the decisions I made in the course of the project, and explain some of the difficulties we encountered.

I wrote the code in Python because it’s popular, easy to read, and has lots of library support. The main library we use is the Python bindings for Z3, which is a popular theorem prover.

Almost all of the complexity of this implementation is in the first order logic axioms that we pass to Z3. The rest of the code is mostly a very simple object oriented sketch of the architecture of an agent.

Implementing proposed mechanisms of conscious experience in Z3 was difficult. Expressing yourself in first order logic is always clunky, and Z3 often couldn’t prove the theorems we wanted unless we expressed them in very specific ways. I suspect that a programmer with more experience in theorem provers would find this less challenging.

Also, there were many ideas that we wanted to express but which first order logic can’t handle. I’ll mention three examples.

First, it would have been easier to express human-like intuitions about inverted spectra if the theorem prover could reason about communication between agents, e.g. if it could prove something like “No matter what question system A and system B ask each other, they won’t be able to figure out whether their qualia are the same or not.” This can’t be expressed in first order logic, but I believe it can be expressed in modal logic. Perhaps this kind of project would work better in a modal logic theorem prover.

Second, it’s not very easy to express the fuzziness of beliefs using first order logic. A lot of our intuitions about consciousness feel fuzzy and unclear. In first order logic (FOL), we’re not able to express the idea that some beliefs are more intuitive than others. We’re not able to say that you believe one thing by default, but could be convinced to believe another. For example, I think that the typical human experience of the inverted spectrum thought experiment is that you’ve never thought about inverted spectrum before and you’d casually assumed that everyone else sees colors the same way as you do, but then someone explains the thought experiment to you, and you realize that actually your beliefs are consistent with it. This kind of belief-by-default which is defeatable by explicit argument is not compatible with first order logic.

Logicians have developed a host of logical systems that try to add the ability to express concepts that humans find intuitively meaningful and that FOL isn’t able to represent. I’m skeptical of using the resulting logical systems as a tool to get closer to human decision-making abilities, because I think that human logical reasoning is a complicated set of potentially flawed heuristics on top of something like probabilistic reasoning, and so I don’t think that trying to extend FOL itself is likely to yield anything that mirrors human reasoning in a particular deep or trustworthy way. However, it’s plausible that some of these logics might be useful tools for doing the kind of shallow modelling that we attempted in this project. Some plausibly relevant logics are default logic and fuzzy logic, potentially combined into fuzzy default logic.

Third, I can’t directly express claims about the deductive processes that an agent uses. For example, Armstrong (1968) is a theory about a deductive process that humans might have; namely, that in certain conditions, we reason from “I don’t perceive that X is Y” to “I perceive that X is not Y.” To express this, we might need to use a logic that has features of default logic or modal logic.

In general, Z3 is optimized for projects which require the expression of relatively complicated problems in relatively simple logics, whereas for this project we wanted to express relatively simple problems in relatively complicated logics. Perhaps a theorem prover based on something like graph search over proofs would be a better fit for this type of project.


Aleksander (2017)Source (archive)
Armstrong (1968)Source (archive)
Bayne (2010)Source (archive)
Bennett & Hill (2014)Source (archive)
Bjorner (2017)Source (archive)
Brook & Raymont (2017)Source
Buck ShlegerisSource (archive)
Byrne (2015)Source
Chalmers (1990)Source (archive)
Chalmers (1996)Source (archive)
Chalmers (2017a)Source (archive)
Chalmers (2017b)Source (archive)
Chalmers (2017c)Source (archive)
Clark (1993)Source (archive)
Cold Spring Harbor Laboratory (2001)Source (archive)
Drescher (2006)Source (archive)
Feynman (1988)Source (archive)
Frankish (2016a)Source (archive)
Frankish (2016b)Source (archive)
Gamez (2008)Source (archive)
Graziano (2016)Source (archive)
Herzog et al. (2007)Source (archive)
Kammerer (2016)Source (archive)
Loosemore (2012)Source (archive)
Marinsek & Gazzaniga (2016)Source (archive)
Molyneux (2012)Source (archive)
O’Regan (2011)Source (archive)
Reggia (2013)Source (archive)
Rey (1983)Source (archive)
Rey (1995)Source (archive)
Rey (2016)Source (archive)
Shlegeris (2017)Source (archive)
Tomasik (2014)Source (archive)
Weisberg (2014)Source (archive)
White (1991)Source (archive)
  • 1. Buck Shlegeris wrote the software agent, and the appendix containing notes for programmers who might want to review the agent’s code. I (Luke) suggested the specific theories to attempt implementing, and wrote the rest of the report. The main body of the report is written in my voice because it discusses the origins and evolution of my personal views and intuitions, which don’t necessarily match Buck’s views and intuitions.

  • 2. For a quick introduction to some functionalist theories of consciousness, see Weisberg (2014), chs. 6-8. On illusionist theories of consciousness in particular, see Appendix F of my earlier report on consciousness and moral patienthood, and also Frankish (2016a).

  • 3. I make this critique of functionalist theories of consciousness in Appendix B of my report on consciousness and moral patienthood. Below I quote some additional examples of (roughly) this objection being made.

    During a February 2017 “Ask Me Anything” session on, David Chalmers made similar complaints about illusionist theories of consciousness. Below, I link to each comment from which I quote. I have also reformatted Chalmers’ replies a bit, for clarity:

    [1] I think [Dennett’s] basic view, that consciousness is an illusion, is a really important one to pursue. If I have any complaint it would be that he hasn’t pursued it strongly and deeply enough in recent years.

    [2] Regarding the paradox of phenomenal judgment: I agree the key is finding a functional explanations of why we make judgments such as “I am conscious”, “consciousness is mysterious”, “there’s a hard problem of consciousness over and above the easy problems”, and so on. I tried to give the beginnings of such an explanation at a couple of points in The Conscious Mind, but it wasn’t well-developed… Illusionists like Dennett, Humphrey, Graziano, Drescher, and others have also tried giving elements of such a story, but usually also in a very sketchy way that doesn’t seem fully adequate to the behavior that needs to be explained. Still I think there is a real research program here that philosophers and scientists of all stripes ought to be able to buy into… It’s an under-researched area at the moment and I hope it gets a lot more attention in the coming years. I’m hoping to return soon to this area myself.

    In response to Frankish (2016a)’s defense of illusionism about consciousness, Marinsek & Gazzaniga (2016) write:

    One major limitation of [illusionism as described by Frankish] is that it does not offer any mechanisms for how the illusion of phenomenal feelings works. As anyone who has seen a magic trick knows, it’s quite easy to say that the trick is an illusion and not the result of magical forces. It is much, much harder to explain how the illusion was created. Illusionism can be a useful theory if mechanisms are put forth that explain how the brain creates an illusion of phenomenal feelings…

    …phenomenal consciousness may not be the product of one grand illusion. Instead, phenomenal consciousness may be the result of multiple ‘modular illusions’. That is, different phenomenal feelings may arise from the limitations or distortions of different cognitive modules or networks… Illusionism therefore may not have to account for one grand illusion, but for many ‘modular illusions’ that each have their own neural mechanisms.

  • 4. On phenomenal judgments, see e.g. Chalmers (1996), pp. 184-186 and 288-292; Molyneux (2012); Graziano (2016).

    On the properties of our sensory qualia, see e.g. Clark (1993); O’Regan (2011).

    On the apparent unity of conscious experience, see e.g. Bayne (2010); Bennett & Hill (2014); Brook & Raymont (2017).

  • 5. Other motivations for this project included (1) the intuition that to understand something better, it is often helpful to try to build it, and (2) a desire to test the intuition that many theories of consciousness seem as though they’d be satisfied by a relatively simple computer program.

    On (1): A commonly-quoted version of this dictum was found on Richard Feynman’s blackboard upon his death: “What I cannot create, I do not understand.” Below are some examples of scholars expressing similar sentiments in the context of consciousness studies.

    Molyneux (2012):

    …Instead of attempting to solve what appears unsolvable, an alternative reaction is to investigate why the problem seems so hard. In this way, Minsky (1965) hoped, we might at least explain why we are confused. Since a good way to explain something is often to build it, a good way to understand our confusion [about consciousness] may be to build a robot that thinks the way we do… I hope to show how, by attempting to build a smart self-reflective machine with intelligence comparable to our own, a robot with its own hard problem, one that resembles the problem of consciousness, may emerge.

    Graziano (2016):

    One useful way to introduce [Graziano’s theory of consciousness] is through the hypothetical challenge of building a robot that asserts it is subjectively aware of an object and describes its awareness in the same ways that we do.

    Reggia (2013):

    What would be the value of studying consciousness with machines? There are two main answers to this question. First and foremost would be to improve our scientific understanding of the nature of consciousness. At the present time, our understanding of how a physical system such as the human brain can support the subjective experiences that are at the core of a conscious mind are largely pre-scientific… Individuals working on artificial consciousness… [observe] that computer models of specific aspects of consciousness (simulated consciousness) may prove to be useful in advancing our understanding of conscious information processing, just as computer models are useful in many other fields.

    On (2): George Rey and others have suggested that it would be fairly simple to write a computer program that seems to satisfy many popular theories of consciousness. See e.g. Rey (1983):

    …it seems to me to be entirely feasible… to render an existing computing machine intentional by providing it with a program that would include the following:

    1. The alphabet, formation, and transformation rules for quantified modal logic (the system’s “language of thought”).

    2. The axioms for your favorite inductive logic and/or abductive system of hypotheses, with a “reasonable” function for selecting among them on the basis of given input.

    3. The axioms of your favorite decision theory, and some set of basic preferences.

    4. Mechanical inputs, via sensory transducers, for Clauses 2 and 3.

    5. Mechanical connections that permit the machine to realize its outputs (e.g., its “most preferred” basic act descriptions).

    Any computer that functioned according to such a program would, I submit, realize significant Rational Regularities, complete with intensionality. Notice, for example, that it would be entirely appropriate — and probably unavoidable — for us to explain and predict its behavior and internal states on the basis of those regularities. It would be entirely reasonable, that is to say, for us to adopt toward it what Dennett (1971) has called the “intentional stance.”

    …However clever a machine programmed with Clauses 1-5 might become, counting thereby as a thinking thing, surely it would not also count thereby as conscious. The program is just far too trivial. Moreover, we are already familiar with systems satisfying at least Clauses 1-5 that we also emphatically deny are conscious: there are all those unconscious neurotic systems postulated in so many of us by Freud, and all those surprisingly intelligent, but still unconscious, subsystems for perception and language postulated in us by contemporary cognitive psychology. (Some evidence of the cognitive richness of unconscious processing is provided by the interesting review of such material in Nisbett & Wilson, 1977, but especially by such psycholinguistic experiments as that by Lackner & Garrett, 1973, in which subliminal linguistic material provided to one ear biased subjects in their understanding of ambiguous sentences provided to the other ear.) In all of these cases we are, I submit, quite reasonably led to ascribe beliefs, preferences, and sometimes highly elaborate thought processes to a system on the basis of the Rational Regularities, despite the fact that the systems involved are often not the least bit “conscious” of any such mental activity at all. It is impossible to imagine these psychological theories getting anywhere without the ascription of unconscious content — and it is equally difficult to imagine any animals getting anywhere without the exploitation of it. Whatever consciousness will turn out to be, it will pretty certainly need to be distinguished from the thought processes we ascribe on the basis of the rational regularities.

    How easily this point can be forgotten, neglected, or missed altogether is evidenced by the sorts of proposals about the nature of consciousness one finds in some of the recent psychobiological literature. The following seem to be representative:

    Consciousness is usually defined by the ability: (1) to appreciate sensory information; (2) to react critically to it with thoughts or movements; (3) to permit the accumulation of memory traces. (Moruzzi, 1966)

    Perceptions, memories, anticipatory organization, a combination of these factors into learning — all imply rudimentary consciousness. (Knapp, 1976)

    Modern views… regard human conscious activity as consisting of a number of major components. These include the reception and processing (recoding) of information, with the selection of its most important elements and retention of the experience thus gained in the memory; enunciation of the task or formulation of an intention, with the preservation of the corresponding modes of activity, the creation of a pattern or model of the required action, and production of the appropriate program (plan) to control the selection of necessary actions; and finally the comparison of the results of the action with the original intention … with correction of the mistakes made. (Luria, 1978)

    Consciousness is a process in which information about multiple individual modalities of sensation and perception is combined into a unified, multidimensional representation of the state of the system and its environment and is integrated with information about memories and the needs of the organism, generating emotional reactions and programs of behavior to adjust the organism to its environment. (John, 1976)

    What I find astonishing about such proposals is that they are all more-or-less satisfiable by almost any information-processing system, for precisely what modern computational machinery is designed to do is to receive, process, unify, and retain information; create (or “call”) patterns, models, and subroutines to control its activity; and, by all means, to compare the results of its action with its original intention in order to adjust its behavior to its environment. This latter process is exactly what the “feedback” that Wiener (1954) built into his homing rocket was for! Certainly, most of the descriptions in these proposals are satisfied by any recent game-playing program (see, e.g., Berliner, 1980). And if it’s genuine “modalities,” “thoughts,” “intentions,” “perceptions,” or “representations” that are wanted, then, as I’ve argued, supplementing the program with Clauses 1-5 will suffice, but without rendering anything a whit more conscious.

    White (1991), ch. 6, summed up Rey’s point like so:

    …a survey of recent characterizations of consciousness by philosophers and psychologists reveals that most or all characterizations would be satisfied by information-processing devices that either exist now or would be trivial extensions of devices that exist.

    See also Rey (1995, 2016) and the “small network argument” of Herzog et al. (2007).

  • 6. The software agent would better-illustrate an illusionist approach if we could say that it is clear that it “mistakenly judges that it is impossible for an agent to be mistaken about its own experiences,” but we decided not to put in the extra work required for the agent to clearly satisfy that criterion.

  • 7. In this sense, this project is similar in motivation to many other “machine consciousness” research projects (Cold Spring Harbor Laboratories 2001; Gamez 2008; Reggia 2013; Aleksander 2017). Arguably, the major distinguishing characteristic of the present project is merely its particular (illusionism-motivated) selection of theories to attempt to instantiate.

  • 8. This number is analogous to the gensym name is Drescher’s “qualia as gensyms” account (Drescher 2006, ch. 2).

  • 9. Technically, the agent’s “yes” reply is “necessarily true,” and its “no” reply is “necessarily false.” The other possible replies from Z3 are equivalent to “Both that statement and its negation are possible” (which I translate as “I don’t know”) and “The axioms I was passed contain a contradiction.” Note that for the agent, 0-255 acts as a color wheel, in the sense that the difference between 255 and 0 is 1 rather than 256.

  • 10. Chalmers (1990):

    Very briefly, here is what I believe to be the correct account of why we think we are conscious, and why it seems like a mystery. The basic notion is that of pattern processing. This is one of the things that the brain does best. It can take raw physical data, usually from the environment but even from the brain itself, and extract patterns from these. In particular, it can discriminate on the basis of patterns. The original patterns are in the environment, but they are transformed on their path through neural circuits, until they are represented as quite different patterns in the cerebral cortex. This process can also be represented as information flow (not surprisingly), from the environment into the brain. The key point is that once the information flow has reached the central processing portions for the brain, further brain function is not sensitive to the original raw data, but only to the pattern (to the information!) which is embodied in the neural structure.

    Consider color perception, for instance. Originally, a spectral envelope of light-wavelengths impinges upon our eyes. Immediately, some distinctions are collapsed, and some pattern is processed. Three different kinds of cones abstract out information about how much light is present in various overlapping wavelength-ranges. This information travels down the optic nerve (as a physical pattern, of course), where it gets further transformed by neural processing into an abstraction about how much intensity is present on what we call the red-green, yellow-blue, and achromatic scales. What happens after this is poorly-understood, but there is no doubt that by the time the central processing region is reached, the pattern is very much transformed, and the information that remains is only an abstraction of certain aspects of the original data.

    Anyway, here is why color perception seems strange. In terms of further processing, we are sensitive not to the original data, not even directly to the physical structure of the neural system, but only to the patterns which the system embodies, to the information it contains. It is a matter of access. When our linguistic system (to be homuncular about things) wants to make verbal reports, it cannot get access to the original data; it does not even have direct access to neural structure. It is sensitive only to pattern. Thus, we know that we can make distinctions between certain wavelength distributions, but we do not know how we do it. We’ve lost access to the original wavelengths - we certainly cannot say “yes, that patch is saturated with 500-600 nm reflections”. And we do not have access to our neural structure, so we cannot say “yes, that’s a 50 Hz spiking frequency”. It is a distinction that we are able to make, but only on the basis of pattern. We can merely say “Yes, that looks different from that.” When asked “How are they different?”, all we can say is “Well, that one’s red, and that one’s green”. We have access to nothing more - we can simply make raw distinctions based on pattern - and it seems very strange.

    So this is why conscious experience seems strange. We are able to make distinctions, but we have direct access neither to the sources of those distinctions, or to how we make the distinctions. The distinctions are based purely on the information that is processed. Incidentally, it seems that the more abstract the information-processing - that is, the more that distinctions are collapsed, and information recoded - the stranger the conscious experience seems. Shape- perception, for instance, strikes us as relatively non-strange; the visual system is extremely good at preserving shape information through its neural pathways. Color and taste are strange indeed, and the processing of both seems to involve a considerable amount of recoding.

    The story for “internal perception” is exactly the same. When we reflect on our thoughts, information makes its way from one part of the brain to another, and perhaps eventually to our speech center. It is to only certain abstract features of brain structure that the process is sensitive. (One might imagine that if somehow reflection could be sensitive to every last detail of brain structure, it would seem very different.) Again, we can perceive only via pattern, via information. The brute, seemingly non-concrete distinctions thus entailed are extremely difficult for us to understand, and to articulate. That is why consciousness seems strange, and that is why the debate over the Mind-Body Problem has raged for thousands of years.

    Chalmers revised this account somewhat in Chalmers (1996), pp. 184-186 and 288-292.

    Another version of the basic idea is Drescher’s “qualia as gensyms” account (Drescher 2006, ch. 2).

    A related idea, cast in terms of neural networks, can be found in Loosemore (2012), which Tomasik (2014) summarizes like this:

    Loosemore presents what I consider a biologically plausible sketch of connectionist concept networks in which a concept’s meaning is assessed based on related concepts. For instance, “chair” activates “legs”, “back”, “seat”, “sitting”, “furniture”, etc. (p. 294). As we imagine lower-level concepts, the associations that get activated become more primitive. At the most primitive level, we could ask for the meaning of something like “red”. Since our “red” concept node connects directly to sensory inputs, we can’t decompose “red” into further understandable concepts. Instead, we “bottom out” and declare “red” to be basic and ineffable. But our concept-analysis machinery still claims that “red” is something — namely, some additional property of experience. This leads us to believe in qualia as “extra” properties that aren’t reducible.

  • 11. To elicit this judgment, we can ask the agent a question of the form “For all 2 agents and one hue, is it true that the the experience of agent 1 and that hue is the same as the experience of agent 2 and that hue?” The reasoning system replies: “Both that and its negation are possible.” For more detail on why this judgment is returned, see the code. Note that for technical reasons the agent doesn’t “believe” all the statements made above (which I gave for providing the right intuition), but instead deduces the possibility of inverted or rotated spectra from the mere belief that other agents experience the same relations between different colors as itself.

  • 12. In my earlier report, I mentioned the present project in section 5.1:

    …I’d like to work with a more experienced programmer to sketch a toy program that I think might be conscious if elaborated, coded fully, and run. Then, I’d like to adjust the details of its programming so that it more closely matches my own first-person data and the data gained from others’ self-reports of conscious experience (e.g. in experiments and in brain damage cases)… We have begun to collaborate with a programmer on such a project, but we’re not sure how much effort we will put into it at this time.

    I also outlined what a more ambitious version of this project might look like, in section 6.2.4.