I have made the primary claim that human language is understood through a simulation facility whose fundamental structure is shared across humanity. In the previous post, I expressed a view that subject-verb-object triples are a fundamental cognitive construct arising out of the structure of the human perceptual and behavioral systems combined with a model of agents that abstracts others as replicates of the self. These triples are one part of the simulation facility, but there is far more to a simulation of experience than just the abstract representation of who did what to whom, including the ability to answer questions of where, when, and how.
Over the course of the next few posts, I will discuss the representation of where, that is, of location and place. It will require some effort to present, not only because much is known about the underlying neuroscience, but also because the representation and manipulation of places and their content are a likely locus of recursion in the brain. The subject of recursion is a central theme of Chomskyan linguistics, and contrary to Chomsky, I will argue that recursive processing may be primarily a feature of long-distance, multi-scale navigation and only secondarily a feature of syntax. But that line of thought will require first that I lay the groundwork by describing what we know about how humans process space in the brain, and that subject will be the central topic of this post.
When scientists, engineers and other technologists design systems to answer the question of where, they typically prefer numeric systems of various kinds. For example, the Global Positioning System (GPS) provides a pair of coordinates describing any position on earth up to a certain accuracy. GPS can be used to navigate drones and self-driving cars towards a fixed location. To comprehend locations in terms of GPS, one must have a some understanding, however indirect, of what a sphere is and how a point on a sphere may be represented by two angles expressed as decimal numbers.
More generally, common mathematical abstractions tend to represent space as a single connected fabric with a uniform system of coordinates. Restated, we imagine space stretching outward from the origin in all directions, with each patch of space having a consistency and measurement just like any other. In mathematical terms, such a structure is called a normed vector space, characterized by the property called translation invariance, and the key example is the Euclidean space that most of us learned to use in high school. As a side note, this conception of space, which is perfectly appropriate for Newtonian physics, is insufficient to describe Einstein's theory of general relativity, where patches of space may stretch or sheer locally. It is also insufficient to describe how the lowest levels of the human mind represent space, for some of the same reasons.
The human mind cannot model an infinite space, nor does it need to. We are typically interested in a limited range of environments. The hunter is primarily interested in the places where prey may be found, not the spaces in between. Modern humans mainly satisfy our modern drives in environments such as the home, the office, the store, the beach, etc. These places that we frequent require detailed modeling, whereas the space between them can be modeled coarsely. Rather than modeling all space within the same frame, it is more memory-efficient to delimit space and conceive of places with boundaries.
The mind, then, abstracts an object-like representation of place based on the appearance and contents of a place, as we can observe from introspection and from the structure of location phrases in speech. You know that you are at home because you are surrounded by walls and your possessions are all around. You know that you are at the beach because there is sand under your feet, an ocean out to the horizon, and hopefully a bright sun in the sky. You are in the car because there is a steering wheel, instruments, a windshield, and usually a road. In each case, the place is an object describable in terms of the subordinate objects that allow it to be recognized.
Within each place, the mind builds a map for navigation that serves as a local coordinate system. This map is not necessarily fixed, and in fact, it may be reconstructed dynamically as needed. Importantly, though, the map coordinates are not centered on the self and do not change as a person moves around. In technical terms, the map is allocentric (from Greek allo- meaning other).
For the mathematically minded, this two-fold conception of space based on places and maps is more like a manifold than a vector space. A manifold is a mathematical space that is not necessarily “flat” in the sense that the space may stretch or fold like a piece of fabric but may not tear. Yet for any point on a manifold, there is a neighborhood of that point that can be described by local coordinate system called a chart. The manifold itself can be thought of as a patchwork garment made of overlapping charts. The coordinates used on each chart may be different in that they may have to bend or stretch the overlapping patches in different ways in order to lay them flat, but there is always a way to move smoothly from any patch to its neighbors. Similarly, as people move from place to place, the corresponding mental maps are swapped out one for the other.
The map-making mechanisms of the brain were discovered experimentally starting in the 1970's, and in 2014 the Nobel Prize for Physiology or Medicine was awarded to John O'Keefe, May-Britt Moser, and Edvard I. Moser for the discovery. At least four significant neuron types involved in spatial processing have been identified in rats, located primarily in the hippocampus and the surrounding entorhinal cortex, both at the bottom center of the brain. These are:
place cells that fire when the rat is present at a particular location;
grid cells that also fire when the rat is present at a particular location, but in a hexagonal grid pattern;
head-direction cells that fire selectively when the rat is facing a certain direction; and
boundary cells or border cells that fire when the rat is at a particular distance from a specific boundary.
Extensive experiments in rodents show that when a rat is placed in a new environment, the place and grid cells self-organize within as little as 10 minutes to map the environment (Weber & Sprekeler, 2018). The exact workings of this system are not yet known with certainty, but its rough outlines as a navigation system seem clear enough. The place cells provide the allocentric coordinate system for the maps mentioned above, permitting the rodent to self-locate. The boundary cells map the internal shape of the place. The grid cells (shown below) reveal the possible paths emanating from the current location in a hexadirectional pattern, with paths spaces at 60 degree intervals (Staudigl et al., 2018). If the boundary cells were to inhibit blocked paths, then only available paths would remain active to be selected by higher systems based on the current objective. The head-direction cells orient the rat within the space to provide integration between the navigation system and the rat's visual, auditory, and olfactory senses.
Studies have shown that humans share this navigational system with rodents. But these studies have shown even more. In humans, place and grid cells modulate their activity not only in response to navigation, but also in response to photographs (Staudigl et al., 2018). That is, exposure to an image alone is sufficient to for humans to build these navigational maps. And whereas in rats, these four cell types are limited to portions of the hippocampus and surrounding cortices, in humans, grid cells have been observed elsewhere, significantly including the medial prefrontal cortex (Ekstrom et al., 2003). Recall my claim that language understanding is in fact the operation of a simulation facility centered primarily in the prefrontal cortex. Here we have (limited) evidence for the representation of place within the prefrontal cortex using cell types evolved for allocentric navigation.
The allocentric system of places and maps provides a basis for navigation and path planning that is stable as body moves through the world, but it does not provide enough information to coordinate muscle movements when grasping, reaching, stepping, kicking, or climbing. For that purpose, the mind uses an egocentric representation of space, that is, one relative to the body itself and ultimately relative to each body part.
The egocentric interpretation of space is processed in the parietal lobe of the brain (shown above), which is sandwiched between the visual cortex (occipital lobe), the auditory cortex (temporal lobe), and the motor cortex. Sensory input enters the brain through the olfactory, auditory, and visual cortices and is then integrated in the parietal lobe to form a sense of space that is oriented with respect to the various body parts.
Regarding vision specifically, the visual cortex separates out object location within the visual field from the object's identity. Object identity flows downward to the dorsal side of the parietal lobe, while object location is streamed upward to the ventral side of the parietal lobe. The neighboring sensorimotor regions are topologically mapped to different regions (as shown in the image below for the motor cortex), so it is likely that the adjacent regions of the parietal lobe provide a representation of how a particular body part needs to move in order to get closer to the detected object.
Neurons in the posterior parietal region, adjacent to the visual cortex, respond to the location of stimuli in the retina, that is, they are retinotopic. Yet they also vary with the head and eye position and hence are believed to translate the visual reference frame from eye position into a body-centered frame (Cohen & Anderson, 2002). Experiments with monkeys reaching for objects demonstrates a transition from representing object location in terms of the eyes to a representation in terms of the hand movements required to grasp the object (Piserchia et al., 2017). This conversion of sight to hand motion is more popularly known as hand-eye coordination. Similar reference frame conversions transform auditory signals from a head-centered representation of location into a representation of how each body part would have to move to touch the source of the signal. Each stimulus or object may ultimately be encoded based on how each body part would have to move to interact with it by touching or grasping (Cohen & Anderson, 2002).
Evidence suggests that brain has two modes of space representation in the egocentric system, one categorical and the other based on continuous distance (Amorapanth et al., 2010). The categorical mode classifies objects based on whether the object is near or far, up or down, above or below, in front or behind, and so on. That is, it has categories for general direction and distinguishes distance in the terms we are familiar through language. In fact, it seems that language primarily represents egocentric space in categorical terms and uses these terms to organize knowledge, whereas the coordinate system represents continuous change in distance and is used to drive motor tasks. There is also a hemispheric bias: although both halves of the brain are involved in detecting both categories and coordinates, the left brain appears to have stronger contributions towards categories while the right brain more heavily favors coordinates (Amorapanth et al., 2010). Interestingly, this egocentric notion of space is also used to represent temporal and even emotional relationships (Peer et al., 2015), a topic to be covered further in later posts.
The egocentric spatial system thus interprets object location and shape into a form that directly enables behavioral interaction with the object, in contrast to the allocentric system which enables planning and navigation. As we have seen, the egocentric system is in direct contact with the senses. The allocentric system, however, is further away from most sensory regions and in any case must be derived from sensory input, which is necessarily egocentric. So the question arises of how the allocentric system acquires places and maps, and the answer must be that it relies on the output of the egocentric system at some stage of processing.
The exact nature of this processing remains unknown, but it seems that regions in the medial frontal cortex receive egocentric output of the parietal lobe and may interpret this information to construct the allocentric maps found the hippocampal formation. One may then conjecture that the medial frontal cortex or regions nearby maintain a higher order map of place and space that locates each observed object, including the self, with its position in allocentric space, with relative positions encoded both categorically (near/far) and in body-centered coordinates.
One can conjecture more, naturally with higher risk of error as the conjectures move away from the experimental grounding. I will of course proceed to take that risk, now that the reader has been sufficiently warned.
In the previous posts, I discussed how objects may be typecast as agents. I now conjecture that objects may also be typecast as places, so that each object is a place and each place is an object. To each place is associated a navigational map, which means that each object likewise receives a navigational map that represents its layout and parts. But places arise as containers for objects, and hence if an object is a place with a map, then we obtain a recursive hierarchy of objects, where each node in the hierarchy represents and object equated with a place, and where the children of that node are recorded in the associated place map.
This claim would also hold for abstract objects, which might arise through combinatorial operations on concrete places that we would recognize as analogies; these operations are the mechanisms of conceptual blending as developed by Fauconnier and Turner (2002).
The speculation that concept maps in the brain rely on brain regions evolved for spatial navigation is not new. Consider the following quote from Bottini & Doeller (2020):
Spatial navigation is made possible by the combination of two different frames of reference that model both the stable location of objects in the environment, and the self-centered agent that navigates this environment interacting with the various objects. Likewise, conceptual navigation may rely not only on the formation of stable multidimensional maps of knowledge across integrated selected dimensions, but also on the self-centered flexible interaction with this knowledge modeled after sensorimotor experience and orchestrated by the attentional system.
I will cover Doeller's work on cognitive maps with more detail in the next post, but I would add to this picture a claim that all recursion in thought or language may in fact originate in the equation of places as objects and objects as places. An evolutionary motive for this development could lie in the need to enable long-distance navigation. Whereas non-human primates travel within fairly restricted ranges, bipedal hominids migrated across longer distances within a single lifetime. The development of migratory behavior may have forced early humans to develop mental strategies for rapidly adapting to new places, and storing known places as though they were objects could be one such strategy. As a side effect, these same object-centered mapping capabilities could have helped early humans to mentally represent tool shapes and functions, leading to advances in tool use as well. The recursive syntax of most modern languages might likewise have a similar origin.
I want to close out this post with some comments about how places might fit into the human simulation facility. If places are represented by objects, then there should exist neural machinery that transforms any object into a place map, whose contents are a range of objects in a particular spatial configuration. Due to the fact that everyone is always located somewhere, every simulation would have to either specify or assume a place. When not stated, I presume that the relevant place is looked up based on the subject, verb, and object from some sort of content-associative map. Thus if I mention the principal, the mind imagines the place to be a prototypical school, whereas for a tree, the place is an outdoor scene. Events such as kicking would invoke places such as a soccer field or a taekwondo dojo. Obviously the combination of agent, event, and object would narrow the possible place considerably.
Yet the fact remains that every simulation requires a place, and without a place a simulation does not make sense. Consider what happens when you walk up to a group of friends in the middle of a discussion. You may feel quite confused for the first several seconds as you try to understand the topic of a conversation, but once you perceive the topic, the meaning of the words comes together. In this instance, the topic itself is a kind of place and provides a framework around which meaning may be assembled. As the prior examples show, places are not concrete and may lack substantial detail, as in the case of the putative school containing the principal. Places may also be abstract, as in the case of a conversation topic. But place or its analogues are never absent, even if unstated.
The choice of place provides a context that strongly influences what concepts and ideas are comprehensible in the course of speech. In a city, a bank is where you deposit money. Beside a river, a bank is a bit of ground. Location is one of many factors that disambiguates the interpretation of words.
But place does more than just disambiguate. Place fills a simulated scene with unstated details. If I make a statement such as George just bought a new couch for his living room, then we might imagine a scene that includes not only a couch, but also a TV, perhaps a fireplace and a lamp, and other details arising from our own personal experience with living rooms. In this way, the linguistic utterance triggers an experience much broader than just the words used by leveraging the human sense of place as a map full of objects, spatially arranged.
Concluding, the human sense of space breaks the world up into a wide range of overlapping places, each with an associated navigational map that is more or less detailed that incorporates the objects contained in a place and their relative placement. These maps are not centered on the self but provide absolute positioning in the environment, though they are derived from self-oriented representations in the parietal lobe that connect the sensory systems to motor control. The neural mechanisms enabling these maps have been identified in and around the hippocampus, but they are also distributed in the frontal cortex as well. Conceptually, places are both objects and containers for other objects, which creates a recursive tree structure that might be implemented in a content-associative memory. This recursion may have developed evolutionarily to support long-range navigation and may have served as a precondition enabling the subsequent development of cognitive abstractions and language.
In the next post, I will explicitly address the idea of place-based recursion through a primary typecast of place-as-object built on a content-associative memory. This point of view directly contradicts the idea that recursion is primarily a linguistic development, which was popular in the later in the latter half of the 20th century. I hope you are enjoying these posts, and that they are comprehensible, even though the subject matter is complex. As always, I look forward to hearing your comments and questions below.
References
S. N. Weber and H. Sprekeler Learning place cells, grid cells and invariances with excitatory and inhibitory plasticity eLife Sciences, 2018 https://elifesciences.org/articles/34560 T. Staudigl, M. Leszczynski, J. Jacobs, S. A. Sheth, C. E. Schroeder, O. Jensen, and C. F. Doeller Hexadirectional modulation of high-frequency electrophysiological activity in the human anterior medial temporal lobe maps visual space Current Biology 28(20), pp 3325--3329, 2018 https://www.cell.com/current-biology/fulltext/S0960-9822(18)31260-0 R. Bottini and C. F. Doeller Knowledge across reference frames: Cognitive maps and image spaces Trends in Cognitive Sciences, 2020 https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(20)30132-7 A. D. Ekstrom, M. J. Kahana, J. B. Caplan, T. A. Fields, E. A. Isham, E. L. Newman and I. Fried Cellular networks underlying human spatial navigation. Nature 425, 2003, pp 184-188 https://www.nature.com/articles/nature01964 Y. E. Cohen and R. A. Anderson A common reference frame for movement plans in the posterior parietal cortex Nat. Rev. Neurosci. 3(7), 2002, pp. 553-562. https://www.nature.com/articles/nrn873 V. Piserchia, R. Breveglieri, K. Hadjidimitrakis, F. Bertozzi, C. Galletti, and P. Fattori Mixed Body/Hand Reference Frame for Reaching in 3D Space in Macaque Parietal Area PEc Cerebral Cortex 27(3), 2017, pp 1976–1990 https://academic.oup.com/cercor/article/27/3/1976/3056371 A. Schindler and A. Bartels Parietal cortex codes for egocentric space beyond the field of view. Curr. Biol. 23, 2013, pp. 177-182 https://www.cell.com/current-biology/fulltext/S0960-9822(12)01440-6 P. Amorapanth, P. Widick, and A. Chatterjee The neural basis for spatial relations J Cogn Neurosci 22(8), 2010, pp 1739-1753 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2933471/ M. Peer, R. Salomon, I. Goldberg, Olaf Blanke, and Shahar Arzy Brain system for mental orientation in space, time, and person. Proc. Natl. Acad. Sci. 112, 2015, pp 11072-11077 https://www.pnas.org/content/112/35/11072 G. Fauconnier and M. Turner The Way We Think: Conceptual Blending and Mind's Hidden Complexities Basic Books, 2002. Figures from Wikimedia Commons Grid cells: https://commons.wikimedia.org/wiki/File:Grid_cells_properties.png Cerebral lobes: https://commons.wikimedia.org/wiki/File:Cerebrum_lobes.svg Motor cortex: https://commons.wikimedia.org/wiki/File:Figure_35_03_04.jpg
Interestingly, the hippocampus is one of the few regions of the brain where neurogenesis occurs into adulthood (at least this has been shown in lab animals, the last time I looked at the literature we had yet to come up with a definitive experiment to *ethically* show this in humans).
Its role in egocentric/allocentric navigation is well understood, and perhaps gets even more interesting when considering how it changes in clinical cases of defect such as depression, anxiety, or PTSD in soldiers. In these conditions, more neurons die than are generated, which literally shrinks the hippocampus - therapeutics such as SSRI's and exercise trigger neurogenesis, causing it to grow.
Considering the cognitive aspects of these defect conditions, one could argue the capacity for sensing "now", that is, instantaneous navigation and situational awareness - both literally and figuratively one's place in the world - are impaired by reduced processing hardware in this region. Its neighboring amygdala region of the brain and its limbic functions may also play an important role in ontology of the self.
Perhaps this is could be a nice thread to pull on for further evidence or impact of cognitive maps.