Map Is Not the Territory: How Sensory Modalities and Perception Abstract Reality
- Operations ATF
- Jul 10
- 13 min read
Nia Jain
St. Xavier's College (Autonomous), Mumbai
10th July 2025
Information received through our sensory modalities do not necessarily represent reality as it is. While sensory limitations restrict the amount of information we actually absorb, cognitive processing transforms sensory information into abstract representations of realities. This article examines how gestalt organization and active inference misrepresents sensory information. It also discusses how language abstracts reality, and discusses limitations of sensory modalities briefly.
The Velvet Hand Illusion is a tactile illusion in which participants report feeling two smooth wires being rubbed on their palms. Gestalt psychologists studying this phenomenon decided to examine the gestalt principles that governed this sensation. Participants were exposed to two columns of pins spaced at varying distances and moving in the same direction. The participants reported feeling two smooth wires being rubbed, but the sensation did not persist when the distance between the pins was increased (Komura et al., 2021). On the basis of this experiment, Komura et al. (2021) concluded that the Gestalt Law of Closure and the Law of Common Fate resulted in such a sensation. The Law of Closure facilitates perception of objects as whole despite discontinuity because of parts being hidden or absent. The Law of Common Fate allows the perception of objects moving in the same directions as a group. As a result of these two laws, the participants perceived the pins as continuous as a wire instead of being interspersed (Komura et al., 2021). This experiment also demonstrates an interesting phenomenon; how information detected by our sensory modalities can be limited. Additionally, cognitive processes can also render misrepresentation of the information received by our sensory modalities (Martin et al., 2021). Alfred Korzybski’s formula, “Map is not the territory it represents” reflects the idea that what we perceive is an abstract representation of reality and not reality itself (Lavoie, 2011). These representations are mediated by top-down cognitive processes like motivations, expectations, and language (Akdoğan, 2015; Frenkel et al., 2008; Johnson et al., 2013; Lupyan et al., 2020; Martin et al., 2021). Additionally, they are also mediated by sensory mechanisms (Bruns, 2019; Murray & Shams, 2023; Urale et al., 2022). This article examines how organization of sensory information using Gestalt principles causes errors in interpretation. Additionally, it also discusses how both top- down and bottom- up processes cause misrepresentation of reality. It discusses how the error management theory causes a response bias in signal detection, and how language primes us to pay attention to certain sensory stimuli. It also discusses certain limitations of the sensory modalities.
Organization by Using Gestalt Principles
The manner in which the information received from our sensory modalities is organized can also result in misrepresentation of reality. The principles of Gestalt psychology such as law of similarity, closure, proximity and common fate are commonly used to organize sensory input (Komura et al., 2021; Valerjev & Gulan, 2013). The subconscious utilization of the law of closure can lead to an illusion in which two lines of inherently the same length are perceived to differ in length, called the Muller-Lyer illusion (Valerjev & Gulan, 2013). Using the law of closure, the arrow on the bottom in figure 1 is construed as forming a closed shape like the inner corner of a room. On the other hand, the arrow at the top in figure 1 creates an open shape like the outward corner of a room. Previous knowledge guides that an outward corner of a room is farther than the inner corner. Using depth cues, then, the line at the bottom/ top is considered to be longer. Thus, visual input is reinterpreted and misconstrued by mental organization.
Figure 1
The Muller- Lyer Illusion
Gestalt principles are also used to organize auditory information (Komura et al., 2021). The law of similarity governs that similar elements are perceived as a group. This law establishes the basis of our experience of the octave auditory illusion (Deutsch & Diana, 2004). To create the illusion, two sounds of different pitches are played in an octave, alternatively in each ear. Even though the sound plays alternatively in each ear, the listeners experience each sound separately in each ear. The brain categorizes similar sounds playing in both the ears into two groups, and then assigns each group to an ear. Thus, each ear hears only a single tone. The Velvet Hand Illusion as discussed above is also an example of how mental organization leads to a misrepresentation of sensory information (Komura et al., 2021). As a result, the abstract representations we form of our environment can differ from reality.
Korzybski differentiates observation from inference (Lavoie, 2011). He proposes four processes that ultimately lead to reporting of an event via language. First, an event or happening occurs. Next, the body reacts to the event physiologically by observing the event and obtaining information through our sensory modalities. Then occurs inference, which is governed by cognitive processes. Lastly, communication about the event occurs through language. Korzybski also holds that the language we use to describe the information our senses have received is simplistic. He argues that our sensory experiences are far more complex than what we express through our language. He compares language to a map. Just how a map is a simplistic rendition of the territory with fewer details, so is language. Language production, he suggests, is the last stage to comprehending an event. Thus, he also emphasizes on the need to distinguish a person from the adjectives used for them. For instance, a hyperactive child is not only hyperactive; he has other aspects of his personality which he might not demonstrate in a particular context (Lavoie, 2011).
Error Management Theory
Signal-detection theory (SDT) proposed by Green and Swets (1966; as cited in Lerman et al., 2010) suggests that an observer predicts the absence or presence of a stimuli based on two factors- sensitivity, which refers to the ambiguity of the stimulus and the sensory capability of the observer, and the response bias. A response bias refers to a proclivity to respond in a certain way. The amount of attention mediates sensitivity, and increased sensitivity can result in better judgement (Lerman et al., 2010). Whereas, the response bias is mediated by a cognitive process called predictive processing (PP) (Martin et al., 2021). Through PP, the brain makes various predictions about a sensory input. The Bayesian inference could be used in predictive processing. The Bayesian approach utilizes both sensory likelihood and prior probability. In case of an ambiguous stimulus where the object could be considered as a stick or a snake, the Bayesian estimate could be applied. Since human beings have encountered more sticks than snakes, the Bayesian approach would suggest its perception as a stick. However, such inferences carry inevitable uncertainties. In this case, the false identification of a snake as a stick could be a deadly error. Thus, human brains often consider the utility of an inference along with its possibility in a process called active inference. Active inference involves the Bayesian inference occurring subconsciously. Additionally, it involves recasting high-utility inferences as high probability inferences. However, such a remodelling only occurs when the sensitivity is low or when the expectation is highly inaccurate. Active inference aims to minimize costly errors and maximize rewards, which is the essential principle of the Error Management Theory (Johnson et al., 2013; Martin et al., 2021).
In an experiment conducted at Koc University in 2015, participants were exposed to multiple reference durations that they were supposed to identify as longer or shorter than a model duration (Akdoğan, 2015). They classified these durations under two bias conditions - long bias and short bias. This article discusses only the long bias condition because the results and inferences drawn from both the conditions were increasingly similar. Under the long bias condition, identifying a longer duration as short would cause them to lose eight points, whereas identifying a shorter duration as long would cause them to lose two points. These participants were a part of the penalty group. Additionally, a reward group also existed which received eight points for correctly identifying the longer duration as long, and two points for identifying the shorter duration as short. Consequently, participants in the penalty group identified more durations as longer under the long bias than did the participants in the reward group and the control group. The researchers then concluded that the participants were inclined to avoid riskier mistakes (identifying the longer duration as shorter) at the cost of making a more trivial mistake (Akdoğan, 2015). Martin et al. (2021) also propose that the primary function of active inference is allostasis, the process of regulating the body’s physiological parameters to maintain it in a state compatible with survival. This includes reducing the possibility of surprisal. Surprisal refers to the informational implausibility of a stimulus in an environment. When the body encounters a ‘surprising’ stimulus, it might respond due to activation of the autonomic nervous system. This forms the basis of perceptual models. Active inference acts by creating models of the environment that will ready the body to react in a way that can minimize risk, instead of acting based on surprisal. For instance, anxious persons rate neutral faces as more fearful than non-anxious persons (Frenkel et al., 2008). Identifying non-threatening faces as threatening has minimal costs. However, not identifying a threatening face could be fatal. Such a model minimizes risk by keeping the body ready to react to a threat if it actually occurs (Martin et al., 2021). This perceptual model also leads to lower sensitivity in anxious persons (Frenkel et al., 2008). Thus, it can be inferred that a person’s expectations about the environment can reduce the actual amount of sensory information that a person takes in to draw inferences (Martin et al., 2021).
Since expectations are utility - driven and consistent with allostasis, these expectations might update with severity of consequences; events with more serious consequences need to be avoided more (Martin et al., 2021). The Error Management theory then can be used to explain ‘one-shot learning’ in which events that occurred once but had fatal consequences are avoided passionately in the future. This can lead to long term behavioural changes such as avoiding certain foods. In extreme cases, however, these expectations can lead to pathological perceptual models such as hallucinations and post-traumatic stress disorder.
Language
Language impacts perception and thus modifies sensory experiences by increasing attention towards certain elements, and by facilitating categorical processing (Lupyan et al., 2020).
Perception through attention
Language acts as a prime which directs people’s attention to certain elements in an environment (Lupyan et al., 2020). In a study that used continuous flash suppression (CFS), a meaningful image was shown to one eye whereas a continuous flash was shown to the other eye, inducing binocular rivalry. Participants who were exposed to the word ‘pumpkin’ right before the experiment could detect the pumpkin in the meaningful image which was otherwise invisible (Lupyan et al., 2013 as cited in Lupyan et al., 2020). In another study, participants exposed to the word ‘blue’ right before the experiment identified a more typical colour as blue than a less typical example of blue (Forder & Lupyan, 2019 as cited in Lupyan, 2020). Linguistic cues can affect the interpretation of a visual scene even when the cues are unrelated to the scene (Lupyan et al., 2020). For instance, participants in a study were exposed to an ambiguous picture of a bird. The participants could not distinguish its head or tail. They were then exposed to a video showing random motion, or they read a description of physical motion. They were instructed to draw a worm in the bird’s beak, and most participants drew the worm consistent with the motion that they had read or watched (Dils et al., 2010 as cited in Lupyan et al., 2020).
Perception through categorical learning
Language provides a categorical label which includes typical characteristics of an element (Lupyan et al., 2020). Consequently, the label of the colour ‘red’ includes the characteristics of ‘red’. Similarly, a categorical label of ‘cow’ includes typical characteristics of a cow that can be used to identify a cow. These characteristics are then used to quickly identify elements in an environment; participants identified a cow better when provided typical distinguishing features than when they were presented with unique features like auditory cues to identify the cow. The availability of multiple labels allows observers to distinguish between elements more meticulously. Although it might not lead to labelling, availability of multiple labels allows a person to distinguish better. Cultures with more - colour words can promote better distinction between colours than in cultures with less - colour words. Botanists can also distinguish between several species merely by sight due to exposure to specific characteristics within distinct labels. Participants also showed an enhanced ability to distinguish between colours when exposed to more labels (Lupyan et al., 2020).
However, choosing these labels also becomes a function of predictive processing (Lupyan et al., 2020). A person deciding whether an object is red or green must remember the typical characteristics of the colour red. At the same time, they must also determine why the object is not green by contrasting its features with typical characteristics of the colour green. Additionally, this process includes referring to the person’s prior experience with the colours. Although such categorical learning might increase speed in simple identification tasks, it can lead to ‘categorical’ predictions, where objects with typical features are remembered better than objects with atypical features (Forder & Lupyan, 2019 as cited in Lupyan, 2020). Participants in a study were exposed to the word ‘green’ and then exposed to a colour swatch (Forder & Lupyan, 2019 as cited in Lupyan, 2020). After some time they were asked to remember if they had been exposed to G1 or G2- both being different shades of green. Participants identified being exposed to the colour more typical of Green even if it was not the colour they had seen. These results are consistent with the interpretation that language occurs after cognitive processing of stimuli and is not a direct report of the stimuli (Lavoie, 2011).
The cognitive processing itself transforms and organizes the sensory information (Martin et al., 2021). A lot of cognitive processing is also subconscious. The combination of the utility bias with the Bayesian inference, for instance, occurs subconsciously. The language we use after processing is also simplified, representing larger categories instead of nuances (Lavoie, 2011; Lupyan et al., 2020). This idea aligns with Korzybski’s suggestion that a map does not capture the territory in detail; it shows only a few important details and leaves out many others. He also proposes that it is because of the limited representational capacity of the map (Lavoie, 2011). Our sensory modalities are also limited in capacity (Murray & Shams, 2023; Urale et al., 2022; Valerjev & Gulan, 2013). Obtaining information from the environment is the bottom-up process involved in perception (Murray & Shams, 2023).
Bottom-up Processing
Our sensory modalities are limited in numerous ways (Murray & Shams, 2023; Urale et al., 2022; Valerjev & Gulan, 2013). The blind spot on the eye’s retina has no photoreceptors. Consequently, the brain predicts whatever exists in the blind spot (Urale et al., 2022). Cross-modality refers to the integration of information from various senses to process the information (Murray & Shams, 2023). A stimulus often activates more than one sensory modality. Information from all the senses is combined and processed to obtain the best representation of reality. However, cross - modality can at times also lead to cognitive distortions. The Ventriloquist effect is a common illusion that occurs when one sensory modality dominates over the other (Bruns, 2019). A ventriloquist typically produces the sound himself but is perceived that it is his doll that speaks. This phenomenon occurs because visual cues often dominate auditory cues and identify the source of sound to be the doll.
Conclusion
The information we receive from our senses is limited by the capacities of our sensory modalities. Factors like the blind spot, directional blind spot and errors in cross-modality already render the information we receive to be incomplete, with some parts of it predicted. Further, what we observe is organized and processed. The organization process usually follows Gestalt principles that transform the information into meaningful wholes. Although the wholes might be meaningful to us, they might not be accurate reflections of reality. Such organization can lead to illusions like the octave auditory illusion, the Muller- Lyer illusion and the velvet hand illusion. Additionally, the brain interprets the sensory information based on emotional states like fear and motivation. The brain predicts the possibility of an event in the environment using active inference in predictive processing. Active inference biases predictions in favour of utility rather than actual possibility. Thus, it aims to maximize rewards and minimize loss. The language we use to describe events also leads to misrepresentation of sensory information. It firstly draws attention to certain elements in the environment over other elements. It also facilitates categorical processing. Thus, sensory limitations and cognitive processing results in an abstract representation of reality rather than reality itself.
It reflects Korzybski’s idea that the Map is not the territory. The Map has limited representational capacity like our sensory modalities. Additionally, it only represents the elements important to the reader rather than all the details of the territory. It also represents the information in a manner understandable to the reader of the map. Parallelly, cognitive processing interprets the information in a way that it has some utility to us. Using language, we express what we observe in a manner that is understandable to us even if it simplifies the observation using categories and typical examples of the categories.
References
Akdoğan, B. (2015). The effects of payoff manipulations on temporal bisection performance. In Graduate School of Social Sciences and Humanities [Thesis]. KOÇ UNIVERSITY. Bruns, P. (2019). The Ventriloquist Illusion as a Tool to Study Multisensory Processing: an update. Frontiers in Integrative Neuroscience, 13.
Deutsch, Diana. (2004). The Octave Illusion Revisited Again. Journal of experimental psychology. Human perception and performance. 30. 355-64. 10.1037/0096-1523.30.2.355.
Fibonacci. 2007. Müller-Lyer illusion. The three horizontal lines have the same length. Wikimedia Commons.
https://upload.wikimedia.org/wikipedia/commons/f/fe/M%C3%BCller-Lyer_illusion.sv
Frenkel, T. I., Lamy, D., Algom, D., & Bar-Haim, Y. (2008). Individual differences in perceptual sensitivity and response bias in anxiety: Evidence from emotional faces. Cognition & Emotion, 23(4), 688–700. https://doi.org/10.1080/02699930802076893
Johnson, D. D., Blumstein, D. T., Fowler, J. H., & Haselton, M. G. (2013). The evolution of error: error management, cognitive constraints, and adaptive decision-making biases. Trends in Ecology & Evolution, 28(8), 474–481.
Komura, H., Nakamura, T., & Ohka, M. (2021). Investigation of tactile illusion based on gestalt
Theory. Philosophies, 6(3), 60. https://doi.org/10.3390/philosophies6030060
Lerman, D. C., Tetreault, A., Hovanetz, A., Bellaci, E., Miller, J., Karp, H., Mahmood, A., Strobel, M., Mullen, S., Keyl, A., & Toupard, A. (2010). APPLYING SIGNAL‐DETECTION THEORY TO THE STUDY OF OBSERVER ACCURACY AND BIAS IN BEHAVIORAL ASSESSMENT. Journal of Applied Behavior Analysis, 43(2), 195–213. https://doi.org/10.1901/jaba.2010.43-195
Lupyan, G., Abdel Rahman, R., Boroditsky, L., Clark, A., University of Wisconsin-Madison, Humboldt-Universität zu Berlin, University of California San Diego, & University of Sussex. (2020). Effects of language on visual perception. In Trends in Cognitive Sciences.
Martin, J. M., Solms, M., & Sterzer, P. (2021). Useful misrepresentation: perception as embodied proactive inference. Trends in Neurosciences, xx.
Murray, C. A., CA, & Shams, L. (2023). Crossmodal interactions in human learning and memory. Frontiers in Human Neuroscience, 17, 1181760.
Urale, P. W. B., Puckett, A. M., York, A., Arnold, D., & Schwarzkopf, D. S. (2022). Highly accurate retinotopic maps of the physiological blind spot in human visual cortex. Human Brain Mapping, 43(17), 5111–5125. https://doi.org/10.1002/hbm.25996
Valerjev, P., & Gulan, T. (2013). The role of context in Müller-Lyer illusion: The case of negative Müller-Lyer illusion. Annual Review of Psychology, 20, 29–36.
http://mjesec.ffzg.hr/revija.psi/vol%2020%20no%201-2%202013/Valerjev_2013_v20-1- 2.pdf
Nia Jain is part of the Global Internship Research Program (GIRP) under IJNGP.
Tags PERCEPTION | ILLUSION | MEMORY | GESTALT
Comments