key: cord-1056253-zkwj7v25
authors: Blum, Sonja
title: Failure to Perceive Change: From Neurons to Social Networks
date: 2021-01-11
journal: Hu Arenas
DOI: 10.1007/s42087-020-00180-0
sha: a57d44a269fe9101d4b1d1b9704c86c0a700474e
doc_id: 1056253
cord_uid: zkwj7v25

Humans are fundamentally bad at perceiving change which is not expected to be present based on prior experience of the organism. Change blindness and inattentional blindness are striking such examples of failure to notice changes in a visual scene. This has been observed in other sensory systems including auditory and somatosensory. In conditions of crisis, whether it is a global crisis such as the COVID-19 pandemic or a local crisis such as Hurricane Katrina, profound changes in structure and function of previously static environmental elements occur and trickle down to each neighborhood, street corner, and individual person’s life. In such conditions of flux and instability, susceptibility to micro-perceptual failures such as change and inattentional blindness may be heightened. These phenomena occur at millisecond to seconds time scale, at the level of initial sensations and first level perceptions, however are linked to multimodal associative brain-wide processes. Here we review the evidence for, and develop a line of argument that following propositions hold true in conditions of crisis: (1) Frequency of change blindness events and related failures of micro-perceptions may be higher; (2) accumulation of micro-perceptual failures of a single sensory system such as vision can impact complex cognitive processes such as reasoning, reality-based belief, and judgment occurring over longer time scales and brain-wide networks; and (3) same failures of micro-perceptions across individuals in a population may have impact on faulty consensus assessments of reality of situation and missed opportunities for solution building on group or collective level.

70% when individuals belong to different social groups (Simons & Levin, 1998) . Even if they seemingly look at the other person's face and body, they will most of the time miss that the individual asking for directions is a different person than moments ago.

Change blindness occurs when observers fail to notice changes in a visual scene, particularly when the change coincides with some manner of brief interruption in the view (Rensink et al., 1997) (Simons & Levin, 1997) (Daniel J. Simons & Levin, 1998) (Rensink, 2005) . In early classic experiments demonstrating this phenomenon, it noted during visual saccades and during an eye blink. Standard lab conditions replicating this phenomenon use short masks of 100 ms to 1 s between scenes. Visual attention is drawn to change when there is a clear "pop" or movement in the scene, a change transient, but when such transients are not attended to, do not exist, or are obscured, change is difficult to detect (Rensink, 2005) . Inattentional blindness occurs when attention is focused on a task, and a sometimes drastic change in visual scene is missed, such as a person in a gorilla suit appearing unexpectedly (Simons, 2012) .

Humans are fundamentally bad at perceiving change which is not expected to be present, and we heavily rely on predictions from prior experience to have a continuous conscious experience through filling in perceptual gaps (Dennett, 1992) and/or through amodal perception (Noë, 2002) . Our fund of knowledge and semantic memory about the world modulates perceptual processing via feedback connections (Kilner et al., 2007) (Friston & Kiebel, 2009) (Clark, 2013) . Evidence suggests that visual awareness and conscious experience reflect prediction rather than the actual state of the world (Nijhawan, 2008) , whether we consider this in a framework of Bayesian sensorimotor psychology (Rescorla, 2016) , or sampling of environment without assuming existence of an underlying representation (Noë, 2005) , or simply keeping in mind that the nervous system and the environment are intimately linked, and through all of the ongoing interactions of the body and the environment, psychological states emerge.

The visual system is bombarded by upwards of 10 8 bits of information at all times, and attention serves as a bottleneck to process information about salient events in the environment or for accomplishing a task at hand (Itti & Koch, 2001) .

It is more efficient to abstract from small number of data points if situation is predictable than devote attentional energy to continued musing over details; however, a problem arises when we are wrong about assuming predictability. Interestingly, lines of research of attention suggest that we orient to novelty, but this is not always the case as sometimes generalization obscures the novelty by attributing sameness based on prior experience and expending attentional focus elsewhere, thus missing novelty entirely. This is nowhere clearer than in the examples of change and inattentional blindness.

Explanations for occurrence of inattentional blindness have been framed as failures of attention: attention is directed to where new information or change are expected, but we do not allocate much attention to monitoring aspects of our surroundings which are not expected to change based on prior experience of the organism. Explanation for occurrence of change blindness is framed as a failure of comparison of pre-and post-change representations. Change blindness assumes temporal transformation: it is not a misperception of difference; rather, it is failure of perception of change which occurred in the past, very recent past, in the range of milliseconds to seconds, but was missed because the observation of dynamic change did not occur (the moment of change was not observed; rather, the change occurred without the viewer's continuous monitoring).

Change blindness as well as inattentional blindness has been observed not only in the visual system but also in other sensory systems including auditory and somatosensory. Some of the earliest work to consider selective attention, the foundations of inattentional and change blindness, was actually conducted using auditory stimuli (Cherry, 1953) . Dichotomous listening tasks of Cherry and colleagues presented participants with different audio-tracks simultaneously in each ear. Participants in these experiments were instructed to follow one stream, while ignoring the other. Such studies highlighted that individuals can filter out the unattended voice and fail to notice changes of speaker or language, although people did appear to remain sensitive to certain cognitively conspicuous prompts, such as their own name, in the unattended stream.

In recent past, there is evidence emerging for change and inattentional blindness on group level, suggesting that majority of individuals across groups and teams are susceptible to change and inattentional blindness for same items across the group (Tollner-Burngasser et al., 2010 )(DiVita et al., 2016 . In particular, the impact in real-world situations has become apparent in aviation, medicine, driving, combat conditions, and eyewitness testimony, where such perceptual changes can lead to catastrophic incidents (Davies & Hine, 2007) (NTSB, 1973) . They failed to notice that the autopilot had inadvertently been disconnected and, as a result, the aircraft gradually lost altitude and crashed.

Flight crew apparently remained unaware that the aircraft was descending, despite a functional altimeter and auditory alarms. The crew had become fixated on a minor fault in the indicator lamp, and apparently ignored the gradual loss of altitude until only moments before the aircraft struck the ground. It is not a radical leap to suggest that change blindness and inattentional blindness and deafness were likely factors in this accident.

We greatly overestimate our ability to detect changes in our environment, whether in real life or experimental conditions. Poor insight into perceptual limitations related to change blindness was first reported by Beck, Levin, and Angelone, who coined the term "changeblindness blindness" (Beck et al., 2007) . They found that 70-90% of participants in movie scene detection task believed they will detect changes in movie sequences that in fact none notice when tested.

Studies on military teams have demonstrated benefits of working in teams on situational awareness, provided that members can communicate freely (Redden & Blackwell, 2001) . These data raise the interesting possibility that teams with well-functioning communication are to a degree protected against the perceptual failures of change and inattentional blindness. There is emerging evidence for this (Tollner-Burngasser et al., 2010) (Greig, 2016) (Jones & Johnstone, 2017) .

In conditions of societal crises, whether it is a global crisis such as the COVID-19 pandemic or a local crisis such as Hurricane Katrina, stakes are high, just as in the cockpit or an intensive care unit, and mistakes of perception, which evidence suggests we can suffer in a majority for same stimuli, may leave us effectively blind for crucial information, which may impact perception of problems and possible solutions. We are biased to act as if conditions are still A when they have decidedly already become B. In conditions of crisis, profound changes in structure and function of previously relatively static stable environmental elements occur and trickle down to each neighborhood, street corner, and individual person's life. Furthermore, the new conditions are in a state of flux rather than a stable new normal, for some time, until a new normal is established or a return to previous conditions occurs. In such conditions of flux and instability, and in a mindset of crisis (Billings et al., 1980) , susceptibility to change blindness may be heightened (Shi & Li, 2020) .

Scenes around us, conversations, all are susceptible to the perceptual failures of change and inattentional blindness which limit our access to information that is right in front of our eyes. Similarly, possibilities for solutions and novel approaches to current climate, political, and health crises may be missed if we are missing information about current conditions by simply not seeing what is right in front of us. Although we can never perceive reality "accurately," we can be attuned to data in our environment with focus on this or that, not everything, and what we are attuned to depend on what is known to be normative and thus expected.

How can we make leaps between change and inattentional blindness, fast-perceptual processes at time scales of hundreds of milliseconds, to longer-time scale "higher order" cognitive processes? Here we explore neurophysiological and theoretical underpinnings for impact of change and inattentional blindness events on longer term cognitive processes such as reasoning, reality-based belief, and judgment.

The primary visual cortex is where the first-order information from the retina terminates. It is at this level of processing that we perceive literal aspects of visual stimuli, the lines, the edges, and the structures of forms. The fibers from the retina to the primary visual cortex travel a long path via the optic tract to the lateral geniculate body of contralateral thalamus then fanning out within the temporal lobe as "optic radiations" to terminate in the primary visual cortex within occipital lobe. The secondary zones of the occipital cortex, the increasingly higher levels of visual association cortices, differ from primary visual cortex in structure and function. The IVth cortical layer, which is plump and thick with incoming afferents in the primary cortex, is much thinner in the association areas. However, the associative cortico-cortical connection layers II and III are of much greater thickness (Luria, 1976 ) (Palomero-Gallagher & Zilles, 2018 .

Whereas in evolutionarily older primates, primary visual cortex projections are much more extensive than secondary and higher order visual association cortices, in humans it is reversed: the secondary visual cortex is larger than primary visual cortex. This reflects the function of visual system in humans in that the role of systems for reception is less prominent than the role of systems for analysis and synthesis ("coding") of incoming visual information .

On a physiological level, mechanisms dedicated to fast information processing are thought to be coded in the spikes and synaptic activity of neuronal populations. On the other hand, the mechanisms involved in processing of slow-evolving events may depend on changes in both the activity of populations of neurons as well as changes in the cellular and biological properties of the large-scale neuronal networks wrapped in white-matter tissue (Pestilli, 2019) . Change blindness as a fast information processing event is not only immediately linked to fast perceptual events occurring on millisecond to seconds time scale but also plays a role in shaping slow-evolving events, which are dependent on interplay of spikes and synaptic activity as well as large-scale neuronal network processing.

A narrow focus on the synapse has given us a "mere stick-figure conception" of how learning and the memories it engenders work (Fields, 2020) . This kind of thinking has also framed considerations of phenomena such as change blindness only within immediate perceptual timeframes, but it is worth considering the relationships of such phenomena to complex reasoning events occurring on longer time scale.

As visual stimuli come in at the level of primary visual cortex, the millisecond immediate sensations cascade to higher level of processing through the associative cortices. At level of cortico-cortical associative processing, the immediate sensations of vision coming from retina via the primary visual cortex trigger and wake up existing knowledge and memory about what we are currently seeing, integrating comparing and "making sense," playing a role in what we describe as "higher order" complex cognitive tasks of reasoning, judgment, and perception of reality.

William James wrote that "much of our thinking consists of trains of images suggested one by another, of a sort of spontaneous revery of which it seems likely enough that the higher brutes should be capable. This sort of thinking leads never the less to rational conclusions, both practical and theoretical… a sunset may call up the vessel's deck from which I saw one last summer, the companions of my voyage, my arrival into port, etc.; or it may make me think of solar myths, of Hercules' and Hector's funeral pyres, of Homer and whether he could write, of the Greek alphabet, etc. If habitual contiguities predominate, we have a prosaic mind; if rare contiguities or similarities, have free play, we call the person fanciful, poetic or witty. But the thought as a rule is of matters taken in their entirety. Having been thinking of one, we find later that we are thinking of another, to which we have been lifted along, we hardly know how" (James, 1950, p. 325) .

It is this kind of reasoning on how thoughts arise that makes one easily link the first sensations upon a primary sensory cortex to higher order thought. The immediate stimuli give rise to action of higher order association cortices, and at light speed associations are made, images, stories, and beliefs are woken up from memory, which then contribute to the content of our ideas, thought, and reasoning patterns at any given moment. It is the constellation of perceived sensations that paints the picture of reality. James further states that "whenever an object so appeals to us that we turn to it, accept it, fill our mind with it, or practically take account of it, so far it is real for us, and we believe it. Whenever on the contrary we ignore it, fail to consider it or act upon it, despise, reject it, forget it, so far it is unreal for us and disbelieved" (James, 1950, p. 295 ).

M. Merleau-Ponty argues that the basic level of perceptual experience is the gestalt (Merleau-Ponty, 1962) . Unitary sensory experiences do not exist separately from consciousness as a whole. Reasoning and immediate sensations are not separated by complex barriers but rather both immediately and at all times belong to the processing that amounts to consciousness. Merleau-Ponty states that the relationship between the world and the observing/sensing subject is one in which "an object perceived can concentrate in itself a whole scene or become the imago of a whole segment of life. Sense experience is that vital communication with the world which makes it present as a familiar setting of our life." (Merleau-Ponty, 1962, p. 52) .

In this sense, what becomes accessible to us via vision, and failures of vision, has impact on what becomes the content of our consciousness. Attention is what allows perception of particulars to be extracted from the field of unified/clumped consciousness. Nothing is a particular in its core, but rather particulars are extracted through selective attention applied to objects (Christopher, 2011, p.164) . This line of thinking would suggest that ignorance of unaccounted for items in change blindness and inattentional blindness is not because these items are not present in our consciousness, but rather that our attention is not applied to them. This is supported by the fact that in group conditions of cooperation, change blindness, and inattentional blindness are greatly diminished at a time scale that suggests items were already accessible to our consciousness, just not attended to.

Here we propose that in conditions of crisis, (1) frequency of change blindness events and related failures of micro-perceptions may be higher; (2) micro-perceptual failures impact longer term cognitive processes such as reality-based belief, reasoning, and judgment; and (3) same failures of micro-perceptions across individuals in a population may have impact on faulty consensus assessments of reality of situation and missed opportunities for solution building on group or collective level.

In relation to the current pandemic, climate, and other large-scale crisis conditions, one may only speculate the various levels of perception at which implications of heightened change and inattentional blindness may be relevant-not just visually but more broadly perhaps failing to perceive details that can indicate both threats as well as possible solutions to the current crises. As reality-based belief is built upon consensus, revealing the blind spots of our not only individual but also group perceptual capacity is imperative to understanding how reality-based belief is constructed.

Group cooperation and teamwork minimizes impact of change and inattentional blindness, and strategies for effective cooperation and intentional team work on tasks addressing assessment of a crisis situation and solution building may mitigate impact of change and inattentional blindness. Mole's writing on "collective agents performing attentively" (Mole, 2011, p.166 ) may be a possible avenue to consider solutions to this perceptual problem. Mole theorizes the possibility of single conscious agent not only as one human but conditions when the agent of an attentively performed task is not one human but rather a collective. In instances of collectives, an activity or task can be accomplished due to division of labor, and no individual member of the collective understands the collective activity in its entirety (sports teams are Mole's simple example of this). A collective, unlike its individual members, is not conscious, yet there are tasks accomplishable only through collective attention to aspects of task that will lead to its accomplishment. In Mole's theory, attention is not treated as an aspect of consciousness but rather a mode of taskperformance.

Can this be extended to large collectives trying to solve complex society-wide problems such as COVID-19 pandemic? Ultimately, both change blindness and inattentional blindness are deficits of attention, across majority of individuals within a group for similar items. The fact that these perceptual failures appear to be remedied by group cooperation and communication supports Mole's assertion that inattention leads to "somewhat depleted" consciousness, but leaves consciousness of gist intact (Christopher, 2011, p. 162) . The unattended to items may still persist in a gist form, such that they are not perceived as particulars and their features are not processed in a way to apply specific object categorization to them (Christopher, 2011, p.163) . As the unattended to objects are not classified specifically, it is thus impossible to know which one if any are changing.

Our access to environmental details depends on practical knowledge of the way in which our actions and sensory stimulation depends on each other (Noë, 2002) . In periods of crisis, when dramatic change may be shifting accustomed to relationships between what we do and sensory stimulation, we may be susceptible to failures of access to relevant details in our environments simply because we do not know yet the new relations being shaped by crisis and we are attending to environmental details which would be relevant under circumstances prior to the period of crisis change. It is precisely an awareness of this, and foregrounding this possibility, that this paper tries to do-we may be attuned to old things, and this makes it harder to perceive details in the environment relevant to the new conditions, details which may be important for helping us solve the problems presented by the new crises and evolving new relations between things.

Solution building in conditions of global crises is carried out in a context of a tremendous amount of unpredictability in our surrounding environments. Belief about our conditions and possible futures is built upon consensus, and revealing the blind spots of our not only individual but also group perceptual capacity is imperative to understanding how we see what is happening around us now and are able to in relation to that imagine what is possible. Data Availability Article uses published data/ review and is otherwise theoretical in nature.

Now you see me, now you don't: change blindness in pilot perception of weather symbology

Change blindness blindness: beliefs about the roles of intention and scene complexity in change detection

A model of crisis perception: a theoretical and empirical analysis

Whatever next? Predictive brains, situated agents, and the future of cognitive science

Some experiments on the recognition of speech, with one and with two ears

Attention is cognitive unison: an essay in philosophical psychology

Change blindness and eyewitness testimony

Verification of the change blindness phenomenon while managing critical events on a combat information display: human factors

Consciousness explained

The brain learns in unexpected ways

Predictive coding under the free-energy principle

Perceptual Error in Medical Practice

Inattentional blindness in anesthesiology: a simulation study

Crime blindness: how selective attention and inattentional blindness can disrupt eyewitness awareness and memory: policy insights from the behavioral and brain sciences

Computational modelling of visual attention

Inattentional blindness and failures to rescue the deteriorating patient in critical care, emergency and perioperative settings: Four case scenarios

The principles of psychology

Predictive coding: An account of the mirror neuron system

Phenomenology of perception

Eastern Air Lines, Incorporated, L-1011

Visual prediction: psychophysics and neurophysiology of compensation for time delays

Is the visual world a grand illusion

What does change blindness teach us about consciousness?

Cyto-and receptor architectonic mapping of the human brain

Mapping brain and behavior across spatiotemporal scales

To see or not to see: the need for attention to perceive changes in scenes: psychological science

Bayesian sensorimotor psychology: Bayesian sensorimotor psychology. Mind & Language

Situational awareness and communication experiment for military operations in urban terrain: Experiment 1: Defense Technical Information Center

Change blindness

Change blindness

How a crisis mindset activates intuitive decision process: Role of inattentional blindness

But did you see the gorilla? The Problem With Inattentional Blindness. Smithsonian Magazine

Failure to detect changes to people during a real-world interaction

Enhancing change detection of the unexpected in monitoring tasks-guiding visual attention in command and control assessment. 21st International Command and Control Reserach and Technology Symposium (ICCRTS)

Individual and team susceptibility to change blindness

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Conflict of Interest The authors declare that they have no conflict of interest.