Rick Grush

The emulation theory of representation: Motor control, imagery, and perception

The emulation theory of representation is developed and explored as a framework that can revealingly synthesize a wide variety of representational functions of the brain. The framework is based on constructs from control theory (forward models) and signal processing (Kalman filters). The idea is that in addition to simply engaging with the body and environment, the brain constructs neural circuits that act as models of the body and environment. During overt sensorimotor engagement, these models are driven by efference copies in parallel with the body and environment, in order to provide expectations of the sensory feedback, and to enhance and process sensory information. These models can also be run off-line in order to produce imagery, estimate outcomes of different actions, and evaluate and develop motor plans. The framework is initially developed within the context of motor control, where it has been shown that inner models running in parallel with the body can reduce the effects of feedback delay problems. The same mechanisms can account for motor imagery as the off-line driving of the emulator via efference copies. The framework is extended to account for visual imagery as the off-line driving of an emulator of the motor-visual loop. I also show how such systems can provide for amodal spatial imagery. Perception, including visual perception, results from such models being used to form expectations of, and to interpret, sensory input. I close by briefly outlining other cognitive functions that might also be synthesized within this framework, including reasoning, theory of mind phenomena, and language.

Towards a cognitive robotics

There is a definite challenge in the air regarding the pivotal notion of internal representation. This challenge is explicit in, e.g., van Gelder, 1995; Beer, 1995; Thelen & Smith, 1994; Wheeler, 1994; and elsewhere. We think it is a challenge that can be met and that (importantly) can be met by arguing from within a general framework that accepts many of the basic premises of the work (in new robotics and in dynamical systems theory) that motivates such scepticism in the first place. Our strategy will be as follows. We begin (Section 1) by offering an account (an example and something close to a definition) of what we shall term Minimal Robust Representationalism (MRR). Sections 2 & 3 address some likely worries and questions about this notion. We end (Section 4) by making explicit the conditions under which, on our account, a science (e.g., robot ics) may claim to be addressing cognitive phenomena.

The architecture of representation

Abstract In this article I outline, apply, and defend a theory of natural representation. The main consequences of this theory are: (i) representational status is a matter of how physical entities are used, and specifically is not a matter of causation, nomic relations with the intentional object, or information; (ii) there are genuine (brain‐) internal representations; (iii) such representations are really representations, and not just farcical pseudo‐representations, such as attractors, principal components, state‐space partitions, or what‐have‐you; and (iv) the theory allows us to sharply distinguish those complex behaviors which are genuinely cognitive from those which are merely complex and adaptive.

In Defense of Some `Cartesian' Assumptions Concerning the Brain and Its Operation

I argue against a growing radical trend in current theoretical cognitive science that moves from the premises of embedded cognition, embodied cognition, dynamical systems theory and/or situated robotics to conclusions either to the effect that the mind is not in the brain or that cognition does not require representation, or both. I unearth the considerations at the foundation of this view: Haugelands bandwidth-component argument to the effect that the brain is not a component in cognitive activity, and arguments inspired by dynamical systems theory and situated robotics to the effect that cognitive activity does not involve representations. Both of these strands depend not only on a shift of emphasis from higher cognitive functions to things like sensorimotor processes, but also depend on a certain understanding of how sensorimotor processes are implemented - as closed-loop control systems. I describe a much more sophisticated model of sensorimotor processing that is not only more powerful and robust than simple closed-loop control, but for which there is great evidence that it is implemented in the nervous system. The is the emulation theory of representation, according to which the brain constructs inner dynamical models, or emulators, of the body and environment which are used in parallel with the body and environment to enhance motor control and perception and to provide faster feedback during motor processes, and can be run off-line to produce imagery and evaluate sensorimotor counterfactuals. I then show that the emulation framework is immune to the radical arguments, and makes apparent why the brain is a component in the cognitive activity, and exactly what the representations are in sensorimotor control.

Internal models and the construction of time: generalizing from state estimation to trajectory estimation to address temporal features of perception, including temporal illusions

The question of whether time is its own best representation is explored. Though there is theoretical debate between proponents of internal models and embedded cognition proponents concerning whether the world is its own best model, proponents of internal models are often content to let time be its own best representation. This happens via the time update of the model that simply allows the models state to evolve along with the state of the modeled domain. I argue that this is neither necessary nor advisable. I show that this is not necessary by describing how internal modeling approaches can be generalized to schemes that explicitly represent time by maintaining trajectory estimates rather than state estimates. Though there are a variety of ways this could be done, I illustrate the proposal with a scheme that combines filtering, smoothing and prediction to maintain an estimate of the modeled domains trajectory over time. I show that letting time be its own representation is not advisable by showing how trajectory estimation schemes can provide accounts of temporal illusions, such as apparent motion, that pose serious difficulties for any scheme that lets time be its own representation.

How to, and how not to, bridge computational cognitive neuroscience and Husserlian phenomenology of time consciousness

A number of recent attempts to bridge Husserlian phenomenology of time consciousness and contemporary tools and results from cognitive science or computational neuroscience are described and critiqued. An alternate proposal is outlined that lacks the weaknesses of existing accounts.

Skill theory v2.0 : dispositions, emulation, and spatial perception

An attempt is made to defend a general approach to the spatial content of perception, an approach according to which perception is imbued with spatial content in virtue of certain kinds of connections between perceiving organism’s sensory input and its behavioral output. The most important aspect of the defense involves clearly distinguishing two kinds of perceptuo-behavioral skills—the formation of dispositions, and a capacity for emulation. The former, the formation of dispositions, is argued to by the central pivot of spatial content. I provide a neural information processing interpretation of what these dispositions amount to, and describe how dispositions, so understood, are an obvious implementation of Gareth Evans’ proposal on the topic. Furthermore, I describe what sorts of contribution are made by emulation mechanisms, and I also describe exactly how the emulation framework differs from similar but distinct notions with which it is often unhelpfully confused, such as sensorimotor contingencies and forward models.

Berkeley and the Spatiality of Vision

Berkeleys Essay Towards a New Theory of Vision presents a theory of various aspects of the spatial content of visual experience that attempts to undercut not only the optico-geometric accounts of e.g., Descartes and Malebranche, but also elements of the empiricist account of Locke. My task in this paper is to shed light on some features of Berkeleys account that have not been adequately appreciated. After rehearsing a more detailed Lockean critique of the notion that depth is a proper object of vision, Berkeley directs arguments he takes to be entirely parallel against the notion that vision has two-dimensional planar contents as proper objects. I show that this argument fails due to an illicit slide unnoticed by both Berkeley and his commentators—a slide present but innocuously so in the case of depth. Berkeleys positive account, according to which the apparent spatial content of vision is a matter of associations between, on the one hand, tactile and motor contents, and on the other hand non-spatial visual contents, also fails because of an illicit slide—again, unnoticed by Berkeley and his commentators. I close by discerning the salvageable and correct core of Berkeleys theory of the spatiality of vision.

Blending in language, conceptual structure, and the cerebral cortex

Abstract The past decade has seen Cognitive Linguistics (CL) emerge as an important, exciting and promising theoretical alternative to Chomskyan approaches to the study of language. Even so, sheer numbers and institutional inertia make it the case that most current neurolinguistic research either assumes that the Chomskyan formalist story is more or less correct (and thus that the task of neurolinguistics is to determine how the brain implements GB, for instance), or that there are two possibilities, Chomskyanism or associationism/connectionism, and that the task of neurolinguistics is to discover which is really the way the brain does it. In either case, the theoretical apparatus of CL is not being explored by neurolinguistics, and hence the promise CL holds for making genuine fruitful contact with theoretical neurobiology is not materializing as quickly as one might hope. This paper is an attempt to make some initial steps at fulfilling this promise.

Visual Adaptation to a Remapped Spectrum:Lessons for Enactive Theories of Color Perception and Constancy, the Effect of Color on Aesthetic Judgments, and the Memory Color Effect

Many forms of visual adaptation have been studied, including spatial displacements ( Heuer & Hegele 2008 ), spatial inversions and rotations ( Heuer & Rapp 2011 ), removing or enhancing various colors in the visual spectrum ( Belmore & Shevell 2011 ; Kohler 1963 ), and even luminance inversion ( Anstis 1992 ). But there have been no studies that have assessed adaptation to an inverted spectrum , or more generally color rotation . We present the results of an adaptation protocol on two subjects who wore LCD goggles that were driven by a video camera, but such that the visual scene presented to subjects was color-rotated by 120°, so that blue objects appeared green, green objects appeared red, and red objects appeared blue (with non-primary colors being analogously remapped). One subject wore the apparatus intermittently for several hours per day for a week. The second subject wore the apparatus continually for six days, meaning that all his visual input for those six days was color rotated. Several experiments were run to assess the kinds and degrees of adaptation, including Stroop ( 1935 ), the memory color effect ( Hansen et al. 2006 ), and aesthetic judgments of food and people. Several additional phenomena were assessed and noticed, especially with respect to color constancy and phenomenal adaptation . The results were that color constancy initially was not present when colors were rotated, but both subjects adapted so that color constancy returned. However, there was no evidence of phenomenal color adaptation. Tomatoes continued to look blue, subjects did not adapt so that they started to look red again. We found no reliable Stroop result. But there was an adaptation to the memory color effect. Also, interesting differences were revealed in the way color affects aesthetic judgments of food versus people, and differences in adaptation to those effects.