Do We See the World or Just a Map of It?
“O mighty process... what talent can avail to penetrate a nature such as this? What tongue will it be that can enfold so great a wonder? Truly none!”(1) Thus wrote Leonardo da Vinci commenting upon the marvels of our visual sense.
We have every reason to share the Tuscan polymath's awe towards this sensory modality even though - perhaps because - we know a great deal more about the psychophysiological processes underlying vision than he even imagined. What these processes reveal about our epistemological relationship to the world - and about us more generally - is no less intriguing.
In this article, I would like to outline some basic characteristics of visual perception which expose the extent to which its seemingly effortless and mirror-like apprehension of the environment is a highly complex construction of our nervous system, shaped by a variety of factors and resulting in a representation of the environment that serves us well in negotiating our pragmatic interaction with it, but is far from representing the world as it is (or at least as we understand it to be based upon the findings of the natural sciences).
On the Symbolizing Nature of Vision
In one of his books (2), visual scientist William Uttal aptly illustrated the essential elements leading to the visual perception of the world by means of an image similar to the crude sketch shown here. The interested reader is encouraged to turn to Uttal's own insightful commentary: which I also relied upon here, but rather freely, and only up to a point, in the following initial remarks.
The image portrays an 'interpreter' whose task it is to construct a map that represents some properties of the bottom of a lake (specifying, for instance, the areas where the bottom is muddy, or sandy, weedy, rocky etc.) The lake's waters are murky, hence the interpreter does not have direct access to the information he is seeking. He must do so indirectly, by using a probe or sensor connected to a fishing line. He carries out his task by dropping the sensor at various points into the lake. If the probe hits, say, a rocky bottom, the sensor's impact imparts a vibration upon the fishing line. Such a vibration travels through the length of the line and eventually reaches the hands of the interpreter. We may presume that the contact of the sensor with a rocky bottom produces a brisk, high frequency vibration in the line, whereas the impact with a muddy area will induce a lower frequency vibration, and so on. The 'interpreter' (it should be clear now why he is thus called) therefore uses the rate of vibration as felt by his hands to infer the properties of the bottom: different vibration frequencies encode different properties of the bottom. He will then adopt a symbol for a vibration frequency that stands for 'rock', one for 'mud' etc., and will proceed to build his map of the bottom of the lake by using such symbols.
This metaphor seeks to capture the essential components and processes that underlie visual perception. The irregular bottom stands for the alleged physical reality external to the perceiver's visual system. The probe or sensor represents the organ of vision, the eye, which is in contact with the light reflected from the objects that make up the world. The contact with the light leads to a change in the physical state of the receptor cells located in the retinas of the eye; this change in turn eventually leads to the generation of a train of tiny electrical signals (the vibrations in our metaphor) that are transmitted via the optic nerve (the fishing line) to several specialized visual areas within the brain (the interpreter), where they will be analyzed. The end point of this process is the conscious visual image of the objects and events in the physical world one is looking at (the 'map' of the lake).
This metaphor helps making clear that we do not perceive the object itself (the bottom of the lake) but a symbolic representation of it (the 'map' produced by our visual system). It is difficult to grasp this intuitively. Normally, we have no trouble distinguishing a map from what it represents. But this is not the case with vision or perception in general, in part because of the apparent immediacy and naturalness of the sensations produced by our sensory organs.
For a specific illustration of the sense in which our perceptions are best understood as symbolic representations of the various features of objects and events, and not as exact reproductions of the things in themselves, consider color. One of the physical determinants of the perception of color is the wavelength of the light that reaches the receptors in the eye's retina. An object's color is the visual system's way to symbolically represents this property. Let us imagine that sunlight (which contains a mixture of all wavelengths that are visible to the human eye) reaches the painted surface of a table. The pigment of the paint will absorb some of these wavelengths, and reflect back some others. Let us further assume that the light which is reflected is mostly in the range of 500-550 nanometers. This band of wavelengths usually gives rise to the perception of green. 'Greenness' therefore is not a physical property intrinsic to the table; it is rather the construct of a visual system that over time has evolved in such a way as to produce the sensation of green when light in the appropriate wavelength range reaches it.
Just as our 'interpreter' used a symbol to stand for a rocky bottom etc., so our visual system uses the 'symbols' 'green' 'red', 'blue' etc. to differentially encode certain properties of the light. There is no intrinsic reason why a particular wavelength should produce the specific sensation of green or of any other color. In this sense, colors as symbols are as arbitrary as the symbols chosen by our map maker.
The same process occurs with other visual features of an object. For instance, remember that, according to physical science any object is constituted by atoms (and its many subatomic elements), and an atom is more than 99% empty space: yet we will perceive the surface of our table as not only 'green' but also as solid.
We Always Perceive What Is there No Longer
One somewhat startling consequence of the functioning of our perceptual apparatus is that the awareness of the environment it gives rise to always pertains to what is no longer physically present.
Consider what has to happen for us to see something. Sunlight strikes the surface of our table, and some of it is reflected. The reflected light travels from the table to our eyes; much of it is reflected back from the sclera (the 'white' of the eye), but some of it makes it through the pupil (the small opening at the center of our cornea). It then travels through the various substructures that make up the eye and eventually reaches the retina, the thin network of cells at the back of the eye which hosts among others the light-sensitive receptor cells. Some of the molecules of photopigment in the outer segment of these photoreceptors capture the particles of light (photons), and as a result undergo a series of biochemical processes that eventually change the electrical state of the photoreceptors' membranes. This in turn leads via synaptic communication to the alteration of the electrical state of the various layers of cells that make up the retina. This perturbation eventually reaches the ganglion cells, which manufacture a series of tiny electrical signals (action potentials). These signals along with the environmental information they contain leave the retina, travel through the optic nerve, and pass on their stimulation to various structures in the midbrain, where some of the information is processed. The stimulated cells therein in turn make synaptic contact mostly with the cells of area 17 of the occipital cortex, which carry out a yet more complex analysis of the sensory input. The information from there is delivered to many other centres - both visual and non-visual - within the cortex for further interpretation. The end product of this process is the conscious perception of the object or event the viewer is looking at.
This complex chain of events takes time. This means that by the time we have become conscious of an external event, the event itself no longer exists as such. If an action in response to a perception is also called for, it will take yet more time to make a decision and then send a signal to our muscles to, say, move our arms to reach for an object. We will therefore be reacting to events that are even further removed in the past.
Fortunately, this temporal mismatch is small enough to have in most cases negligible consequences for our ability to negotiate the environment. But it is significant from the conceptual point of view. Along with the symbolizing nature of our perceptual processes, its temporal dimension further reinforces the view that in a very real sense, we 'live', not in the world itself, but in a mind-created world. Making a similar point, Uttal noted that our isolation from the world is relieved only by whatever information reaches us from our sensory systems, so that 'the old canard that we do not perceive the outside world at all, but only the activity of our receptors, has a very great degree of truth to it.'(3)
We Learn To See
Since visual perception is a complex process involving a large part of our central nervous system, one should expect it to be open to a number of influences beyond the purely sensory input. Indeed, psychological research has abundantly shown that factors such as memory, emotional state, previous experience, expectations, physical environment and culture, all powerfully affect the way we perceive a scene.
Yet another factor which shapes our perception is learning. We literally learn to see through our continuous commerce with the environment.
Perceptual learning was long known to play a significant role in the early years of human sensory development. However, until the later decades of the 20th century it was generally assumed that no meaningful perceptual learning occurs past childhood, and none in adulthood.
We know better now. Recent empirical research has shown that significant perceptual learning can and does occur even in the adult years: our learning to see – or hear or smell or taste or touch – as mediated by both perceptual, attentional, and cognitive factors may extend over a long arc of our lifespan.
That adults can continue to learn to see was apparently understood in their own terms by some artists and poets well before it was even suspected by perceptual scientists. Let me give you a good example of this.
A Poet Goes to the Zoo
In the year 1902, Bohemian-Austrian poet Reiner Maria Rilke (1875-1926) went to the zoo in the Jardin des Plantes in Paris. This is what he tells us he saw (4)
His weary glance, from passing by the bars,
Has grown into a dazed and vacant stare;
It seems to him there are a thousand bars
And out beyond those bars the empty air.
The pad of his strong feet, that ceaseless sound
Of supple tread behind the iron bands,
Is like a dance of strength circling around,
While in the circle, stunned, a great will stands.
But there are times the pupils of his eyes
Dilate, the strong limbs stand alert, apart,
Tense with the flood of visions that arise
Only to sink and die within his heart.
When I first read this poem I was impressed, not just by its esthetic worth, but by the intensity, precision, and vividness of the poet's powers of observation. This is what truly 'seeing' something amounts to, I thought: the ability to fully inhabit the present as it unfolds by remaining totally focused on the object of one's vision.
I learned afterwards that Auguste Rodin, the preeminent French sculptor of his time, whom Rilke had come to visit in Paris with the intent of writing a monograph about his work, 'had urged Rilke to take himself to the Jardin des Plantes in Paris and pick one of the animals in the zoo there and study it in all its movements and moods until he knew it as thoroughly as a creature or thing could be known, and then write about it.' (5)
This power of vision was not innately given to Rilke, I then realized. It had required the promptings of a great visual artist to induce Rilke to train his visual skills. Indeed, in a later work, a semi-autobiographical novel written during his Parisian sojourn, Rilke has the protagonist of the story note that he is 'learning to see. I don't know why it is, but everything enters me more deeply and doesn't stop where it once used to. I have an interior that I never knew of...' (6)
1. Lael Wertenbaker (1984). The eye. New York: Torstar Books.
2. William Huttal (1981). A Taxonomy of Visual Process. Hillsdale, NJ.: Lawrence Erlbaum Associates.
4. Rainer M. Rilke (1918). Poems. Translation by J. Lamont. New York: Tobias and Wright.
5. Quoted in: John Banville, Study the Panther, New York Review of Books, January 10, 2013.
6. Rainer M. Rilke (1910). The Notebooks of Malte Laurids Brigge. New York: Norton Co.