I really appreciate how Mark Changizi approaches the subject of why we think the way we do. In a posting (here) he looks at colour.

I noticed that whenever the ‘hard question’ is discussed, the example that is used to illustrate its nature is colour. How do we explain colour and how do we tell if others see the same colours as we do? Changizi has an answer.

One of the reasons may be that the world can seem arbitrarily labeled in color, as if a painter dabbed over everything in order to make it beautiful… and that naturally makes us wonder what a different artist might do…. It’s an unfortunate intuition, one that seeps its way not only into the minds of laymen, but into our “enhancement” products and even the hallowed halls of philosophy. In trying to explain what’s wrong with the intuition, let me begin with a thought experiment concerning a product that gives the wearer “shape enhancement” vision…. But few of us would be interested in using them for everyday wear. We want to see the world roughly as it is, not geometrically warped for no reason… Why should it be acceptable to warp colors but not shapes? I’ll suggest here that it’s not acceptable - that once we appreciate the meaning of color it becomes apparent that we shouldn’t arbitrarily engage in color distortion…. colors are just as steeped in meaning as are shapes, pitches, and all the other non-invertible dimensions of our experience. I’ve argued in my research and in my book The Vision Revolution that our primate-variety color vision is optimized for sensing the spectral signals on skin when we blush, flush, blanch and signal other emotions. Our peculiar variety of color vision is just the needed peculiarity to sense oxygenation and concentration modulations in the blood under the skin, the physiological dimensions undergirding the colors we signal. …But I don’t believe that the fundamental appeal of color is due to this arbitrary-splashes basis at all. Instead, it seems more likely that our love of color comes from the meaning of color, namely, that color vision for us primates is a deeply human and emotional sense. Color is evocative and aesthetic because its subject-matter concerns the most evocative states of the most important objects in our lives: other people. That’s why we find color so captivating. It’s not because color floats above the world ungrounded, but, rather, because it is so deeply rooted in our psyche.

And, have you noticed that there is sometimes an assumption that our perception of colour is less mechanical than other senses. ScienceDaily (here) has a report from U of Rochester, Color Perception Is Not In The Eye Of The Beholder: It’s In The Brain. Williams and Hofer found large differences in people’s retinas but very small differences in their perception of colour. I am not sure that their results should have surprised them. There is a well known phenomena called colour consistency which ensures that the perceived color of objects remains relatively constant under varying illumination conditions.

But really, our development puts a lot of biological cost into vision – eyes, the crossover of the optic nerve, a lot of processing prior to the optical cortex and then the size and complexity of the optic cortex. Can anyone think that this system is constructed by just slapping it together so that each person’s sight is a question of chance? Your red can be my green is nonsense. What is important is what the colour means to us.

For the majority of us, the characteristics of things stay within their sensory categories. We do not see numbers with colours; we do not feel sounds; we do not taste shapes. People who have these strange mixing of the senses, synesthetes, are not that rare. For a long time, the idea of synesthesia was not accepted. Now, it has been shown over and over to be a real way of perceiving, individual to the person but stable and automatic.

Recently a few people have been found that see colour in response to various emotions. Ramachandran studied a man with this type of synesthesia. He saw a blue aura around a person which changes shade when particular emotions were sensed. The researchers placed a target person against a white background and drew a black line on the background around the person. The subject saw the blue aura in the space between the person and the black line. Blue or orange letters were projected onto the background and the subjects was timed in his identification of the letters. Blue letters inside the black line were identified slower than orange letters or blue letters outside the line. Blue letters against blue aura were more difficult to see. The researchers will be looking for cross-activation of V4 colour perception area and other areas (they hypothesis insular cortex, activated during subjective experience of emotion).

A Spanish group under Milan studied healers. One healer in particular, with a very good reputation, was clearly a face-colour synesthete seeing the aura of people, a touch-mirror synesthete who experience touch and pain when seeing it happen to someone else, and also a schizotypy personality tending towards slight paranoia and delusions. The researchers believed that the synesthesia encouraged the healers to believe in their own ability to heal other people, and their confidence caused a significant placebo effect in the healed people. However they believe that synesthesia is not an extrasensory power but a subjective and adorned perception of reality.

Oh well, educated people used to believe that meteorites and ball lightening were silly tales of the ignorant and superstitious. Now we are finding that near death experiences, ghosts and auras may have reasonable explanations and are not faked but simply not understood.

There is an interesting blog post (here) by W Davies. It was written on April fools day and so may possibly be tongue in cheek – if so – too, too subtle. He is a materialist and his question is “where is the cat?” when he looks at his cat.

I can make a picture of a cat in my head; I can close my mind and think of it. So I’m perceiving this image of a cat. Where is the image? Where is the cat? … my initial answer is that the cat is simply a 1:1 correlate of certain neurological activity in the brain. That is, if you open up my head you won’t see a picture of a cat, but you’d see something that’s the equivalent of it, sort of like the dots and dashes of Morse code are not English characters, but they are equivalents of them. From a materialistic perspective, you’d theoretically be able to interpret the activity in my brain through some technology, and recreate the image of the cat that I am picturing on a screen.

He then recounts the video ‘mind reading’ experiments and concludes that there is a coded picture of his cat in his brain. But he still has a problem.

So when I was asked “where” my mental image of the cat is, that’s why I responded in this way — the image is located in the brain – it’s just in a different format. But really, I’m not satisfied with that answer. Because in my mind I can see (well maybe not see, but certainly perceive) the cat; not the equivalent neural ‘code’, but the actual cat. I know where the neural code is, but I don’t know where the cat is.

Actually there are at least three cats: the reality-cat, the cognitive-cat, and the consciousness-cat. Its like the territory, the map and the traveler. All we are aware of is the consciousness-cat, the other two are invisible. We can only infer them (and we do infer them reliably). Consciousness is the awareness of a constructed model of reality. We are not aware of the cat, we are aware of our model of the world which includes the cat. One thing our conscious model does not include is the construction of itself. We not not model the modeling process. Modeling the modeling gives an infinite regression of Quakers on oats boxes. So why does the model seem so real? That is its purpose – it is a model – it’s supposed to feel real – it’s evolved to feel real – it would be useless if it was not believable.

If you abandon materialism, you still have the same problem. Where is the magic mind-stuff that has the picture of the cat, how did it the cat get there, why does it seem real? The answers are similar too. They amount to the idea that we need to be aware of a model of the cat in a model of the world and believe it. This does not side-step any questions but just makes them harder.

The brain expends a great deal of energy to create the model and make it available to many processes in the brain. It does a good job of make that model useful and believably ‘real’. We have to infer how this is done and progress is being made in understanding it. Have a bit of patience. Don’t panic.

Synesthesia is a condition where attributes associated with one sense (say colour with sight) can be experienced in another inappropriate sense (say colour with the perception of musical notes). There are many kinds, and rare ones are still being discovered. There is no longer any question that these are ‘real’ perceptions and not hoaxes. Synesthesia seems to have its roots at the sensory level and is a bottom-up rather than top-down phenomenon. There is evidence for heightened sensory activity levels and of additional connectivity between sensory modalities. A lack of normal ‘pruning’ is one of the possible causes.

It is no longer a question that the condition is inherited – it is. But not the specific type of synesthesia. Rather the genetic tendency is for any one or more of 60 odd varieties. Brang and Ramachandran (see citation) discuss the possible reasons for this condition not to be eliminated during evolution. Perhaps it has no disadvantage; perhaps it is a side-effect of a useful gene(s); perhaps it is the extreme of a normal distribution that includes us all.

Another possible explanation is that synesthesia simply represents the tail end of a normal distribution of cross-modality interactions present in the general population. Partial evidence supporting this idea comes as sensory deprivation and deafferentation (i.e., loss of sensory input through the destruction of sensory nerve fibers) can lead to synesthetic-like experiences. For example, after early visual deprivation due to retinitis pigmentosa, touch stimuli can produce visual phosphenes, and after loss of tactile sensation from a thalamic lesion, sounds can elicit touch sensations . More remarkably, arm amputees experience touch in the phantom limb merely by watching another person’s hand being touched. Long-standing evidence has also demonstrated that hallucinogenic drugs can cause synesthesia-like experiences, suggesting the neural mechanism is present in all or many individuals but is merely suppressed. However, no research has yet established the relationship between these acquired forms to the genetic variant and whether the same neural mechanism is responsible for both.

And perhaps, synesthesia is actually advantageous. What are some possible plus points?

1. Synesthesia may assist creativity and metaphor – it is more frequent in creative people and is a little similar to metaphor.

2. It may assists memory – there is some evidence from savants.

3. There is enhanced sensory processing – such as finer discrimination of colours

These demonstrations of enhanced processing of sensory information suggest a provocative evolutionary hypothesis for synesthesia: synesthetic experiences may serve as cognitive and perceptual anchors to aid in the detection, processing, and retention of critical stimuli in the world; in terms of memory benefits, these links are akin to a method of loci association. In addition to facilitating processes in individual sensory modalities, synesthetes also show increased communication between the senses unrelated to their synesthetic experiences, suggesting that benefits from synesthesia generalize to other modalities as well, supporting their ability to process multisensory information. Furthermore, others have argued that synesthesia is the direct result of enhanced communication between the senses as a logical outgrowth of the cross-modality interactions present in all individuals.

The puzzle of how genetically, how physiologically, and why it is that synesthesia arises will be very illuminating to the questions of how qualia are bound to objects and why we have the vivid conscious experience that we have.

Brang, D., & Ramachandran, V. (2011). Survival of the Synesthesia Gene: Why Do People Hear Colors and Taste Words? PLoS Biology, 9 (11) DOI: 10.1371/journal.pbio.1001205

ScienceDaily has an item (here) reporting a paper by Damasio’s group, Seeing Touch is Correlated with Content-Specific Activity in Primary Somatosensory Cortex. They examined the touch equivalent of the mind’s eye.

“When asked to imagine the difference between touching a cold, slick piece of metal and the warm fur of a kitten, most people admit that they can literally ‘feel’ the two sensations in their ‘mind’s touch,’ ” said Meyer, the lead author of the study. “The same happened to our subjects when we showed them video clips of hands touching varied objects,” he said. “Our results show that ‘feeling with the mind’s touch’ activates the same parts of the brain that would respond to actual touch.”

Human brains capture and store physical sensations, and then replay them when prompted by viewing the corresponding visual image. “When you hold a thought in your mind about a particular object, that is not just mental fluff. It is rather a detailed memory file that is being revived in your brain,” Antonio Damasio said.

Here is the abstract:

There is increasing evidence to suggest that primary sensory cortices can become active in the absence of external stimulation in their respective modalities. This occurs, for example, when stimuli processed via one sensory modality imply features characteristic of a different modality; for instance, visual stimuli that imply touch have been observed to activate the primary somatosensory cortex (SI). In the present study, we addressed the question of whether such cross-modal activations are content specific. To this end, we investigated neural activity in the primary somatosensory cortex of subjects who observed human hands engaged in the haptic exploration of different everyday objects. Using multivariate pattern analysis of functional magnetic resonance imaging data, we were able to predict, based exclusively on the activity pattern in SI, which of several objects a subject saw being explored. Along with previous studies that found similar evidence for other modalities, our results suggest that primary sensory cortices represent information relevant for their modality even when this information enters the brain via a different sensory system.

We think of synaesthesia as an unusual sensory effect – the senses getting ‘mixed up’. But it may be more accurate to think of it as a ‘mix up’ in the binding of qualia to concepts. D. Nikolic, U. Jurgens, N. Rothen, B. Meier and A. Mroczko published a paper in Cortex, Swimming-style synesthesia (2011) that shows a clear concept component. I have not found free access to this paper but here is the abstract:

The traditional and predominant understanding of synesthesia is that a sensory input in one modality (inducer) elicits sensory experiences in another modality (concurrent). Recent evidence suggests an important role of semantic representations of inducers. We report here the cases of two synesthetes, experienced swimmers, for whom each swimming style evokes another synesthetic color. Importantly, synesthesia is evoked also in the absence of direct sensory stimulation, i.e. the proprioceptive inputs during swimming. To evoke synesthetic colors, it is sufficient to evoke the concept of a given swimming style e.g., by showing a photograph of a swimming person. A color-consistency test and a Stroop-type test indicated that the synesthesia is genuine. These findings imply that synesthetic inducers do not operate at a sensory level but instead, at the semantic level at which concepts are evoked. Hence, the inducers are not defined by the modality-dependent sensations but by the “ideas” activated by these sensations.

It would be interesting to find out if this effect is operating just at a semantic level or whether, as I suspect, it acts at a more general conceptual level. Can it happen with concepts that do not have associated name-words?

Terhune, Cardena and Lindgren published a paper, Disruption of synaesthesia by posthypnotic suggestion: an ERP study, and this paper was discussed by Vaughan Bell in the blog Mind Hacks and in the Guardian newspaper. (here)

Abstract:

This study examined whether the behavioral and electrophysiological correlates of synaesthetic response conflict could be disrupted by posthypnotic suggestion. We recorded event-related brain potentials while a highly suggestible face-color synaesthete and matched controls viewed congruently and incongruently colored faces in a color-naming task. The synaesthete, but not the controls, displayed slower response times, and greater P1 and sustained N400 ERP components over frontal-midline electrodes for incongruent than congruent faces. The behavioral and N400 markers of response conflict, but not the P1, were abolished following a posthypnotic suggestion for the termination of the participant’s synaesthesia and reinstated following the cancellation of the suggestion. These findings demonstrate that the conscious experience of synaesthesia can be temporarily abolished by cognitive control.

In Bell’s discussion, he points out that this is very unexpected “it is equally new to science because no one had suspected that synaesthesia could be reversed.” The synaesthesic effect was being measured by the addition time it took to identify targets that were coloured differently than the colour that the synaesthesic gives them and by the neurological signs of conflict between the two colours (the Stroop effect). Hypnosis can reverse this by eliminating the synaesthesic colour. How?

This trait (hypnotisability) is usually described as “suggestibility” but it is nothing to do with gullibility or being easily led. People susceptible to hypnosis are not more naive, trusting or credulous than anyone else, but they do have the capacity to allow seemingly involuntary changes to their mind and body. The key phrase here is that they “have the capacity to allow” because hypnosis cannot be used to force someone against their will…

When a suggestion is successful, the experience of it seeming to “happen on its own” is key and this is exactly what neuroscientists have been working with – by suggesting temporary changes to the mind that we wouldn’t necessarily be able to trigger on our own. In the case of the two experiments that managed to temporarily “switch off” the Stroop effect in highly hypnotisable people, the suggestion was that the words appeared as “meaningless symbols”. This avoided a clash between the colour and the word because the text suddenly appeared to be gibberish…

Neuroscientists Amir Raz and Jason Buhle suggest hypnosis is really when we allow suggestions to take over from our normally self-directed control of attention that deals with mental self-management, allowing science an exciting tool to “get under the hood” of the conscious mind.

If you find hypnosis intriguing then you will find Bell’s article very interesting, as I did.

Vision is so important to humans that it is difficult to imagine how we can produce a conscious model of the world without it. And what is done with the third of the cortex that is involved in vision when it is idle. Kupers and others (see citation) have been comparing fMRI scans using congenitally blind, blind that were once sighted, sighted and blindfolded sighted individuals.

How do individuals who never had any visual experience since birth form a conscious representation of a world that they have never seen? How do their brains behave? What happens to vision-devoted brain structures in individuals who are born deprived of sight or who lose vision at different ages? To what extent is visual experience truly necessary for the brain to develop its functional architecture? What does the study of blind individuals teach us about the functional organization of the sighted brain in physiological conditions?

It is known that the cortex has capacity for plasticity and reorganization when input from a sense is lost. Other senses will use the spare cortical areas. Studies have shown that there are changes in the grey matter, the white matter under it, and the cell metabolism during this reorganization. In the blind, the occipital cortex (visual cortex) becomes involved in other senses and in a variety of cognitive functions including: lexical, semantic, phonological, attention, verbal memory, working memory.

A part of the cortex (extrastriate ventrotemporal cortex) is concerned with recognizing objects, a function that is very important to acquiring knowledge of the external world. Different categories of object give specific activity patterns in this region, termed object-form-topology. This processing relies heavily on vision. But blindfolded people who recognize an object from feel show very similar patterns to those that occur when they use sight. The patterns are supramodal – they do not depend on any particular sense.

The findings in the congenitally blind subjects are important also because they indicate that the development of topographically organized, category-related representations in the extrastriate visual cortex does not require visual experience. Experience with objects acquired through other sensory modalities appears to be sufficient to support the development of these patterns. Thus, at least to some extent, the visual cortex does not require vision to develop its functional architecture that makes it possible to acquire knowledge of the external world.

So the ventral ‘what’ pathway can process without vision. What about the dorsal ‘where’ pathway? Is spatial processing possible without vision? Yes, the dorsal pathway can use senses other than sight and does not require visual experience to develop. We process motion per se.

Both optic and tactile motion provide information about object form, position, orientation, consistency and movement, and also about the position and movement of the self in the environment.

And when they looked at mirror neurons, they found the same condition. Vision is not necessary for the development of a functional efficient mirror neuron system. This suggests that abstract representation of actions is also not tied to any particular sense.

The main hypothesis that we have put forward here is that the development of consciousness in the absence of vision is made possible through the supramodal nature of functional cortical organization. The more abstract representation of the concepts of objects, space, motion, gestures, and actions – in one term, awareness of the external world – is associated with regional brain activation patterns that are essentially similar in sighted and congenitally blind individuals. The morphological and/or functional differences that exist between the sighted and the blind brain are the consequence of the cross-modal plastic reorganization that mostly affects that part of the cortex that is not multimodal in nature.

What about the experience that results from the reorganization in the blind? It appears that the type of qualia is connected to the source of the input not the region that processes it.

The results of these TMS studies constitute the first direct demonstration that the subjective experience of activity in the visual cortex after sensory remapping is tactile, not visual. These findings provide new insights into the long-established scientific debate on cortical dominance or deference. What is the experience of a subject in whom areas of cortex receive input from sensory sources not normally projecting to those areas? Our studies suggest that the qualitative character of the subject’s experience is not determined by the area of cortex that is active (cortical dominance), but by the source of input to it (cortical deference). Our results are in line with evidence that sensory cortical areas receive input from multiple sensory modalities early in development.

Kupers, R., Pietrini, P., Ricciardi, E., & Ptito, M. (2011). The Nature of Consciousness in the Visually Deprived Brain Frontiers in Psychology, 2 DOI: 10.3389/fpsyg.2011.00019

I have often wondered about how we recognize faces. We are so very good at recognition and so bad at describing faces in words. One person will say big nose, freckles, oval shape – and this is correct – but it is of no help to someone else in forming an image of the face. The way faces are recognized rarely rises to consciousness and therefore to useful for verbal description.

The current theory is that we build up by experience an ‘average face’. We then compare faces we encounter to this average face. It can be thought of as a mathematical ‘space’, a multi-dimensional face-space. The distance from the average corresponds to the amount of difference the face is from the average and the direction from the average face corresponds to the way/s the face differs from the average. So there is a center with arrows going out various distances in various directions to each known face.

This makes certain things clearer.

First, this is probably the reason that a person who has encountered only a few people from another racial group, has difficulty identifying people of that group. People are left stammering that all Chinese people look alike, knowing how stupid this sounds. They do not have a good average face for the unfamiliar group and therefore have difficulty establishing the differences between any face and the average.

Second, we use the average face as a sign post for beauty. The closer a face is to our average, the more attractive it is.

Third, the closer a face is to the average, the faster we recognize that it is a face. But knowing that a object is a face, the farther it is from average, the faster it is recognized as a particular face. These results depend on the density of faces in the face-space: high near the average and low far from the average.

Fourth, caricatures are recognized as the target although they are actually very, very different from the target face. But a good caricature is on the near exact direction from the average but just a much farther distance away then the target face. It is like following our imaginary arrow to the particular faces and then carrying on in the same direction for some distance.

Fifth, another interesting effect is archetypes. We can think of face-space having places (other than the average) of unusual high density of encountered faces surrounded by low density areas. Some particular face near the center of such a clump could come to stand for this type of face.

Sixth, this explains why we can sometimes hardly notice such prominent changes as new glasses, loss of a mustache etc. These are just not dimensions/directions in the face-space so they are not used to recognize faces.

I have found another Edge answer that is very interesting (here). Donald Hoffman, author of Visual Intelligence, describes a metaphor for sensory qualia – a computer desktop.

Our perceptions are neither true nor false. Instead, our perceptions of space and time and objects, the fragrance of a rose, the tartness of a lemon, are all a part of our “sensory desktop,” which functions much like a computer desktop.

I have encountered people who judge our senses by how accurate they are. They are not happy with the lack of a one-to-one mapping between wave length of light and the perception of colour. The illusions that fool us are treated as mistakes. This all is interpreted as sloppiness in biological systems. But really, the purpose of our perceptions is not accuracy but usefulness.

Graphical desktops for personal computers have existed for about three decades. Yet they are now such an integral part of daily life that we might easily overlook a useful concept that they embody. A graphical desktop is a guide to adaptive behavior. Computers are notoriously complex devices, more complex than most of us care to learn. The colors, shapes and locations of icons on a desktop shield us from the computer’s complexity, and yet they allow us to harness its power by appropriately informing our behaviors, such as mouse movements and button clicks, that open, delete and otherwise manipulate files. In this way, a graphical desktop is a guide to adaptive behavior.

Graphical desktops thus make it easier to grasp the nontrivial difference between utility and truth. Utility drives evolution by natural selection. Grasping the distinction between utility and truth is therefore critical to understanding a major force that shapes our bodies, minds and sensory experiences.

… We must take our sensory experiences seriously, but not literally. This is one place where the concept of a sensory desktop is helpful. We take the icons on a graphical desktop seriously; we won’t, for instance, carelessly drag an icon to the trash, for fear of losing a valuable file. But we don’t take the colors, shapes or locations of the icons literally. They are not there to resemble the truth. They are there to facilitate useful behaviors.

This is useful to keep in mind then thinking about just how personal our personal conscious experience is. Of course we know that we can’t actually know whether your red is the same as my red. But we know that we have a map of our retinas in our thalamus and another in the cortex at the back of our heads. There may be others too. These maps are linked with nerves so that the same points on the maps communicate with one another. A place on the retina has a corresponding place in the thalamus map and the cortex map. This is accomplished by a combination of genetically produced developmental chemicals and ordinary experiences of the world environment. There is no reason for either identical or for significantly different results from this developmental program. Similarly we have the same chemicals in our retina to respond to colours etc. So the answer to whether your red is the same as mine is probably – not identical but extremely similar. Further, it hardly matters because the reason for the red or any other shade is to inform and guide our behaviour – the system has evolved to give us ‘adaptive behaviour’. Qualia have evolved to contrast what needs to be separated, to notice what needs to be noticed, to be attracted or alarmed as appropriate and they seem to do a good job of it.