How the Brain Sees: From Pixels to Perception

The human visual system is not a single organ that records images like a camera. Instead, it is a layered, distributed piece of machinery: many specialized brain areas work together, each made of intricate networks of neurons tuned to extract particular information from the visual world. Any object we look at evokes a pattern of activity that is, in principle, unique to that object. Higher brain centers then use that activity to inform us consciously about what we are seeing.

Built-in assumptions and hidden knowledge
Our brains don’t interpret visual input from scratch. Over evolutionary time and during childhood, the visual system has incorporated statistical regularities of the physical world as implicit “assumptions” or hidden knowledge. Because the world is not chaotic but largely stable in its physical properties, the brain can use these assumptions to resolve ambiguity in perception. One everyday example: when interpreting the shapes of shaded objects, our visual system assumes that light comes from above. On convex objects that bulge toward us, the upper surfaces are expected to be illuminated. This single rule helps the brain disambiguate shading patterns and recognize three-dimensional shape from two-dimensional images.

Early filtering and many visual areas
The first map the brain “reads” from the eye acts like a sorting editorial office. Redundant or useless information is discarded and certain attributes of the image are emphasized. This edited information is then distributed to roughly 30 distinct visual areas. Each area receives either a partial or full map of the visual world. Why so many areas exist is not fully understood, but they appear highly specialized: some extract color, others depth, motion, texture, or object identity. The majority of those areas are located within what’s called the “what region” of the brain.

Two major pathways from the eye
Messages from the eyeball travel along the optic nerve and immediately split into two phylogenetically different pathways: an older, more primitive one, and a newer one that is most highly developed in primates.

Older pathway: Eyes → Superior Colliculus (brain stem) → higher cortical regions such as the parietal lobe. This route functions as an early warning and orienting system. It tells you where an object is and triggers reflexive actions—like turning your head so the object falls on the fovea, the region of highest acuity.
Newer pathway: Eyes → Lateral Geniculate Nucleus (LGN, a relay station) → Primary Visual Cortex → other visual areas. This pathway supports the conscious experience of seeing—“I see this”—and the detailed analysis of what the object is.

Because of this division, the older pathway can guide behavior even without conscious awareness, while damage to the newer pathway (particularly the primary visual cortex) produces conventional blindness. Stroke affecting the posterior cerebral artery can destroy part of the primary visual cortex, causing blindness in the opposite visual field, a condition called hemianopia.

The newer pathway further divides into two streams:

The “where” pathway, terminating in the parietal lobe, assigns spatial location to objects and supports abilities like navigating uneven terrain, following moving targets, and judging distance.
The “what” pathway, terminating in the temporal lobe, handles object recognition, including color and form. Most of the 30 visual areas cluster in this region.

Losses and syndromes that reveal function
Clinical damage to specific regions makes their roles obvious. Damage in the Middle Temporal Area (MTA) causes motion blindness: vision may be normal in the static sense, but moving scenes are perceived as a succession of snapshots. Damage to V4 produces complete color blindness, distinct from congenital color blindness tied to retinal pigment abnormalities. If the “what” pathway is obliterated, objects remain visible in the sense of edges and boundaries, but nothing will be recognizable or evoke emotional associations. If the “where” pathway is damaged, patients struggle to look toward objects of interest or reach for them—sometimes producing Balint’s syndrome. In Balint’s syndrome, bilateral parietal damage produces tunnel-like attention: the patient’s gaze locks onto whatever small object falls into the fovea while ignoring other nearby objects.

Color processing begins early: cones in the retina send outputs to clusters of color-sensitive cells in primary visual cortex—“blobs” and thin strips—which project to V4; color processing becomes increasingly sophisticated along this route.

Filled-in perception and its limits
The visual system also fills in missing information, a process driven by assumptions about the world and by neural processing efficiency. We all have a physiological blind spot in each eye; normally the opposite eye supplies matching data and the brain “fills in” the missing region so we aren’t aware of it. Filling-in operates at different speeds for different perceptual attributes—color, motion, texture—and it can be either perceptual or conceptual.

Perceptual filling-in is carried out by visual neurons. Once those neurons make a decision about what should occupy a gap, that percept is fixed and not easily reversible.
Conceptual filling-in comes from memory and imagination. You can imagine the space behind your head and populate it with anything; that is conceptual completion rather than the automatic perceptual completion done by the visual system.

The filling-in process is an economy measure: evolution favors doing as little processing as necessary to get the job done. By wiring statistical regularities (for example, that table surfaces are uniform) into early visual pathways, the brain reduces processing load downstream.

Pathologies, hallucinations, and imagery
Certain disorders show what happens when normal visual input or its regulation breaks down. Charles Bonnet Syndrome is common among people with vision loss from glaucoma, cataracts, macular degeneration, or diabetic retinopathy. Damage somewhere along the visual pathway causes partial or complete blindness, and patients often experience vivid visual images. These images are typically based on conceptual completion—memory-driven—rather than perceptual completion.

Migraines offer another example. If a blood vessel goes into spasm, a temporary blind spot (scotoma) arises. If that scotoma falls on an object, the object may disappear from awareness. Instead of a void, the brain may fill the missing region with wallpaper-like texture—not a literal blank.

Imagery and perception share neural machinery, but not perfectly. When we see a cat, visual attributes from the retina travel up through the thalamus to primary visual cortex, then split into streams that extract depth and motion (allowing grasp or dodge) and object features (shape, color, texture), and finally combine with memory and emotion so we recognize “a cat.” When we imagine a cat, evidence suggests the visual machinery runs in reverse: information flows top-down from higher regions back into early visual areas, producing a virtual-reality-like simulation.

Why imagination usually stays inside the head is instructive. Early visual areas and retinal baseline activity constantly signal whether a stimulus is present; that baseline informs higher centers that no external input is hitting the retina, vetoing top-down imagery so it does not register as real perception. If those early pathways are damaged and baseline signals are reduced, top-down imagery can escape inhibition and become hallucination. This makes evolutionary sense: imagination must not substitute for reality—if imagining food satisfied hunger, organisms wouldn’t eat and would fail to survive.

Structural lesions also teach us about visual function. An arteriovenous malformation (AVM) in the back of the brain—a cluster of fused arteries and veins—can rupture and cause lethal hemorrhage. Treatments that reduce an AVM’s size can scar visual cortex and alter perception.

The interplay of bottom-up and top-down
Taken together, patient studies and basic anatomy suggest perception is the end product of a dynamic interplay between incoming sensory signals and high-level stored information. Early visual areas act as both filters and canvases: they edit and emphasize image attributes and feed many specialized areas that further analyze color, motion, depth, and identity. Higher areas supply memory, expectation, and imagination that can shape early visual activity, producing simulations that are vivid but usually constrained by baseline signals that distinguish imagined from real input.

We still lack a precise map of the interface between perception and imagination—that “where” top-down signals meet bottom-up sensory data—but studying patients with selective visual defects continues to reveal how the brain balances economy, stability, and flexibility to create our visual world.

Source : The Brain that Changes Itself: Stories of Personal Triumph from the Frontiers of Brain Science by Norman Doidge

Goodreads : https://www.goodreads.com/book/show/570172.The_Brain_that_Changes_Itself

Read the Previous Article in the Series :