A Philosophical Understanding of Representation for Neuroscience
AA Philosophical Understanding of Representation for Neuroscience
Ben Baker , Benjamin Lansdell , Konrad Kording Abstract
Typical systems neuroscience experiments correlate the activity of recorded neurons with controlled variations in stimuli or behavior. Such experiments are commonly taken to reveal that the neural activity is a representation of the stimulus or behavior. But what does this statement mean? Representation is a philosophical concept with a rich history going back at least to Aristotle. A careful reading of contemporary philosophy suggests four criteria of representation as they relate to neuroscience. First, a representation needs to correlate with the object in reality that it represents. Second, a representation needs to have some causal impact on behavior. Third, a representation should have its causal role exclusively -- the absence of the representation should block the behavior it enables. Fourth, a representation should have teleological justification that specifies the cases in which it is a misrepresentation. We argue that not taking all four criteria seriously leads to statements that are neither neuroscientifically nor philosophically satisfying. We discuss how neuroscience can move towards a deeper notion of representation. Introduction
Experimental data in neuroscience are interpreted in terms of certain key concepts, such as those of coding, representation, and computation. The problem of working out precisely how these concepts apply and relate to one another in different cases is essentially one that requires taking a philosophical perspective on the practice of neuroscience. This paper works in that vein, and is focused on the notion of representations in brains. The concept of representation is used ubiquitously in neuroscience. This concept has also received considerable scrutiny from philosophers. The idea of representations as essential to how the mind works is one over which much ink has been spilled in contemporary philosophy, and also, in some form, going back to the ancient period. By looking at a range of philosophical perspectives on the topic, alongside an overview of relevant methods and tools in neuroscience, we aim to disambiguate the notion of representation and clarify what neuroscience experiments can and cannot reveal about the representations at play in the brain.
The use and misuse of the concept of representation in Neuroscience
Neuroscience uses a varied and somewhat informal notion of representation. Roughly, representations in neuroscience can be characterized in the following way: a representation can be taken to be a state or set of states within the brain that is used by an animal as a stand in for some thing itself, acting as a kind of description or image of entities outside the brain. This intuitive definition breaks down into two components: that the representing states within the brain in some way relate to the things being represented, and that they are used by the animal -- they in some way determine behavior (deCharms and Department of Neuroscience, University of Pennsylvania Department of Bioengineering, University of Pennsylvania Department of Neuroscience, Department of Bioengineering, University of Pennsylvania, CIFAR ador 2000) . A neural representation of some object F is supposed to help us understand how the animal whose brain it is does interesting, F -related things. Motivated by this definition, in this section we draw out the different uses of the term representation, so that we can more precisely delineate the important features of representations, and how they are established within neuroscience. This provides context for the later sections in which, drawing from the rich philosophical history of the topic, we carefully spell out what a more satisfactory approach to neural representation entails. Representation from behavior
There are three ways in which neuroscientists think about and use the concept of representation. The first way comes purely from studying animal behavior. The informal definition above tells us that a representation must be used by the animal to drive adaptive behavior. Thus an animal’s behavior can say something about things the animal has a representation of. This is because representations allow for behaviors that are not just responses to immediate stimulation. If an animal possesses a representation of something, say a predator behind it, it can use this representation to make predictions about its future location, and to act accordingly, even without it currently being visible. Behaviors that have an anticipatory component in this sense thus naturally lead to hypotheses about representations. Indeed, this has been proposed as a basis for defining representations (Brette 2018; Mirski and Bickhard 2019; Bickhard 2009) . Not all animals possess this sort of anticipatory behavior: animals whose behavior is largely reactionary may not have rich representations of the world in this sense. The Hydra, for example, appears to have a relatively reactionary behavioral repertoire (Han et al. 2018) . Behavior is thus a key component driving hypotheses about what things animals represent (Krakauer et al. 2017) , yet by itself it cannot answer how an animal implements these representations. Representation from correspondence (and neural coding)
Second, answering this “how” question comes from additionally examining another aspect of how representations are studied in neuroscience: that the representation must in some way relate to the thing being represented. That is, neuroscience also studies representation by investigating how neural activity relates to features of the world. Thus typical neuroscience experiments involve recording the activity of neurons and how that activity correlates with changes in stimuli or behavior that experimenters can observe or control. They identify patterns of covariation between neural responses and whatever experimental variable is being set, either stimulus parameters or quantified behavior. The analysis of this usually involves tuning curves (deCharms and Zador 2000) . In contrast to the above discussion on behavior, it is easiest to quantify correlations to things in the world that an animal is currently experiencing. Thus such questions of neural representation are most well-studied in early sensory areas, where activity by and large only depends on incoming stimuli, thereby providing the cleanest connection between neural activity and features in the world (Purves et al. 2018) . Neuroscientists studying vision, for instance, will record from neurons in V1 and observe how their activity is correlated with visual stimuli it was concurrently presented with. It is common to infer content based on the strength of these correlations, although the soundness of such inferences has been challenged within neuroscience (Brette 2018) . That is, the neural activity may be taken as a representation of the thing in the outside world that the activity is correlated with. In practice, this is a largely separate approach to hypothesizing about representations to that suggested by studying behavior. Many neuroscientists also go further and interpret such correlational experiments in terms of a communication metaphor, which can be quantified with Shannon’s information theory (Simoncelli and Olshausen 2001) . This approach interprets neural activity as transmitting one among multiple possible signals that are sent through presynaptic connections and received by postsynaptic neural structures. This nterpretation allows for asking how much information about a given external variable is contained in the activity. Often neurally transmitted information is further interpreted to be an ‘encoded’ version of features of the outside world. This notion of a ‘neural code’ assumes that the relevant information from the environment is transduced by peripheral sensory mechanisms and encoded into the format in which the neurons communicate, and further that this neural activity is subsequently decoded by downstream processes in the brain. The notion of a neural code is ubiquitous in neuroscience, and may even be used interchangeably with a neural representation (e.g. (Hirokawa et al. 2019; Soh et al. 2018; Ruff and Cohen 2019) (Allen et al. 2017) ). However, while the neural code concept allows for a lot of beautiful theory to do with channel capacity and efficiency that can drive hypotheses about how neurons send information (Rieke 1997) , it generally does so by focusing almost exclusively on how neural activity correlates with external stimulus features, and not on a neural population’s role in driving adaptive behavior. Thus, even according to the rough definition we have outlined above, as a basis for studying representation focusing on neural codes and correlates alone is insufficient. Representation and computation
Third, which representation is studied comes from not just thinking about representations of things in the outside world, but also of internal variables. Many systems neuroscientists think about neural computation -- that is, about steps of information processing that supports decision making, memory, reasoning, etc. Following Marr (1982), this leads to thinking about what algorithms the brain may be using to achieve some computation, and how such algorithms may be implemented. Though not all philosophers may agree (Chalmers 1994; Piccinini 2008) , computation is generally taken to be representational in nature; that is, the algorithm operates over variables which have some semantic meaning, and there exists some mapping between the variables in the algorithm and neural states, thus accounting for how the algorithm is implemented. As an example, consider the ramping of spiking activity in area LIP when primates are presented with ambiguously moving stimuli. According to one theory, the spiking rate of neurons in LIP are proposed to represent the integrated evidence for the stimuli moving in one direction or the other (Shadlen and Newsome 2001; Pillow et al. 2008; Xuelong Zhao and Kording 2018; Latimer et al. 2015) ). This is a different form of representation to that discussed above, in that it is a representation of an internal computational state, not of some external feature of the world. Studying such representations thus requires the additional theoretical baggage of a computational framework and algorithm. Summary
We have outlined three different popular senses in which neuroscience uses and studies the concept of representation -- one based on behavior, one based on relations between neural activity and the world, and one used in the context of computational theories. Each approach most naturally focuses on different types of representations -- by studying representation from behavior one focuses on anticipatory action and capacities needed to store information about things no longer being presented to an animal; by studying representation from correlations with neural activity it is easiest, particularly experimentally, to focus on immediate stimuli or motor behaviors and early sensory and primary motor areas; and a focus on computation means an additional focus on algorithms and representation of internal variables. Each approach relies on different forms of evidence and tends to make different commitments and assumptions -- a focus on behavior may make no commitments about how a given representation is implemented, and focus on neural correlates may often, even if implicitly, assume such correlates play a role in implementing a representation, and a focus on computation proposes that particular algorithms are being implemented by the brain. Here we are not concerned with questions of neural computation, but instead focused on neural representations of things in the outside world. Even when focused on this form of epresentation, the term is used across many domains of neuroscience to mean quite different things and with different degrees of formality. The result of this heterogeneity is an imprecise and vague notion of representation in neuroscience. Notably, representation is foundational to much contemporary work in philosophy of mind , and neuroscience seems not well-informed by this related discipline. The goal of this paper is to improve on this state of affairs. Philosophically Grounded Criteria for Representation in Neuroscience
In broad terms, philosophers define representations as entities that have semantic content and acknowledge that they can come in various types or modes. In other words, a representation is about something, and it might represent what it is about in a number of ways. For example, a map, a painting, and an utterance could all represent the same place in different ways. Representationalism is the idea that mental processes essentially involve some form of representation, and this idea has been prevalent in philosophy for a long time. In the era of brain research there have been various philosophical accounts of what representation involves, which generate different assessments of the way neuroscience experiments try to reveal representation (Field 1978; Fodor and Fodor 1987; Dretske 1988; Hatfield 1991; Churchland and Sejnowski 1990; Markman and Dietrich 2000; Grush 2004; Eliasmith 2005; Ramsey 2007; Egan 2014; Shea 2018) . There have also been recent critiques from within neuroscience of the theoretical gap between what typical experiments reveal about neural activity and the explanatory role neural activity is interpreted to play (Jones and Kording 2019; Brette 2018) . We build on these critiques and draw on the range of philosophical views of representation to distinguish four criteria neuroscientists should use to formulate and test hypotheses about representation. We show the import of each of these four and what neuroscience does or could do to establish each, and we spell out why a satisfying notion of representation depends on all four. Criterion 1: Correspondence
An intuitive first step to defining inner representations would be to describe them as matching or corresponding to the things they represent. So one says, for instance, it is partly by having inner states that match or correspond to red fruit in its environment that an animal can recognize that fruit. More specifically, one might say there must be an isomorphism between internal representations and the external states they represent, which would result in an observable correlation in the way internal and external states vary. In general, this criterion says that if some neural activity N represents some event or feature of the world F , then one should be able to find evidence of a correspondence between N and F in the form of correlational data. This kind of correspondence is central to representations as they figure in typical neuroscientific research and is deeply rooted in the history of philosophical thinking about representations. A prevailing view among early modern philosophers was a resemblance-based view of representation. For example, Locke (1690) and Hume (1739) conceived of internal “impressions” or “ideas” of external objects as representing those objects in virtue of resembling them (Locke 1948; Hume 2019) . Isomorphism is important to philosophers of mind at least as far back as Aristotle’s De Anima , which conceives of registration by perceptual organs on analogy to the symbol that remains in the shape of some wax after a ignet ring has been pressed into it (Book II, Chapter 12). Philosophers found compelling the notion that clever behavior depended on internal states whose “shapes” and interactions mirrored things in the outside world long before any were aware of the brain’s capacities to implement complex formal relationships. There have been some that try to define representation just in terms of a kind of correspondence, such as a measure of mutual information (Usher 2001) or higher-order structural similarities (O’Brien and Opie 2004) . However the bulk of the philosophical literature on the topic suggests that correspondence alone does not establish the sort of relevance to behavior that representations are supposed to have. Most accounts appeal to theoretical principles other than and often in addition to correspondence to determine the content of representations, as described in our discussion of the later criteria. The first criterion is insufficient for representation because, in short, any statistical or morphological form of correspondence is sure to exist between neural processes and all sorts of things they are not representations of. This is brought out most clearly by cases where the behavioral stakes are high and the cost of a false negative is significantly different from that of a false positive. Consider, for example, that many animals respond with avoidance behaviors to relatively simple features of visual stimuli, such as a looming dark object (Schiff, Caviness, and Gibson 1962; Xinyu Zhao, Liu, and Cang 2014) . Observable neural structures over short time-spans may reliably lead to avoidance behavior, as in the case where these neural activities correspond with “looming” described in strictly visual terms. However it is widely assumed in such cases that there is represented content that merits the avoidance behavior - something dangerous, which looming patterns of light are not. On many prominent philosophical views, correspondence (often in information-theoretic terms) is a crucial piece of the puzzle of representation, but some further theoretical resources are needed to determine which correspondences are relevant (Dretske 1981; Fodor and Fodor 1987; Bickhard 2000; Millikan 2001; Shea 2018) . The notion that brain activity must in some way reflect the things being represented is well explored in neuroscience and might be called the dominant paradigm. Most systems neuroscience experiments correlate neural activity with features of an animal’s environment, e.g. stimuli or movement, and theorize on the basis of such models. At least since the work of Edgar Adrian, the observed correspondence between neural activity and stimulus features was recognized as important, and as potentially related to representation or coding. In fact, the most recent nobel prize on the finding of correlations of neural activity with external variables just happened in 2014 (Burgess 2014) while the first one was awarded in 1963 (Huxley 1963) . Experimentally, such observations are possible even with single electrode recordings in conjunction with presented stimuli or observed behavior. Thus, they have formed the basis for many theories and conceptions about how the brain represents features of the world and how the brain generates adaptive behavior. Criterion 2: Causing Behavior
Our second criterion follows directly from the way representations are supposed to figure in explanations of behavior. Representations help us to make sense of how an animal acts on the basis of information it has access to that is relevant to its needs and wants. Representations whose content is about external states and the opportunities and threats they pose are part of what drive creatures to act in regular ways - toward what they need, away from what is dangerous, etc. Thus, representations typically figure in causal explanations. For example, a representation of a predator might be offered as part of a causal explanation of an animal’s flight. Generally, this criterion says that to support a claim that neural activity is a representation of F , one should find evidence that N ’s correspondence with F is causally involved in one or more F -related behaviors. Indeed, most philosophical accounts of representation at least implicitly assume a causal role for the representation. Prominent philosophers in the late 20th century were concerned to capture this aspect of representations, and tried to account for the sort of causal relations that must exist between a representation, its contents, and other parts of a representational system. Fodor’s articulation of this puzzle was influential, as was his answer in terms of “asymmetric causal dependency” between representations and their contents (Fodor 1990) . Other notable philosophers in discourse with Fodor also clearly state and wrestle with the problem of accounting for inner states that are taken to cause behavior (Dretske 1988; Horgan 1989; Millikan 1984) . A lively discourse emerged around the issue, and philosophers today are still actively debating the central ideas from these views of representation. We point to these classical philosophical formulations of the problem simply to flesh out the idea that correspondence does not suffice to classify the kind of internal states of interest. Representations figure in causal interactions that produce behavior, so without any evidence of this causal role one does not have evidence that neural activity is a representation. Causality is often recognized as important to studying representations in neuroscience, but in many settings it is not sufficiently established (Marinescu, Lawlor, and Kording 2018) . Compared to estimating correlations, establishing a causal effect on behavior for some neural circuits is more difficult. There are many ways neuroscientists can probe causal relations -- including lesion and knockout experiments, electrophysiological stimulation (Isitan et al. 2020; Penfield and Rasmussen 1950) , and optogenetics (Deisseroth et al. 2006) . Nonetheless these experiments tend to be hard; performing targeted interventions on the right neurons, and only those neurons, in the right way, still is a challenge in many organisms. Even when perturbations are possible, they are generally low-dimensional, making it hard to say much about the causal effects of components of a representation as opposed to the entire representation. We argue that this criterion of causality is usually not established by systems neuroscience experiments but emphasize that new tools make establishing causal relations much more feasible. Criterion 3: Exclusivity / Specialized Functional Role
Further reflection on the explanatory role representations are meant to play supports our third criterion, which says the causal contribution that a representation makes to behavior should be one that it makes exclusively. For example, we may have two neural populations that are anatomically parallel in such a way, that either is sufficient to drive behavior. In that case, it can not be argued well that each one of them is the representation. To clarify this thought, suppose a pattern of neural activity N is found to be tuned to some environmental feature F , and that intervening to block that pattern precludes the animal’s adaptive behavior with respect to that environmental feature. So, our first two criteria are met with respect to N as a representation of F . However, N might represent something more general or specific than F , or N could be just a part of a larger representation of F , or N could be a causal relay involved in F -related behaviors without representing anything in particular. As such, many patterns of neural activity involved in F -related behaviors are likely to meet the first two criteria, in which case the claim that N has representational content of F does no explanatory work. To justify calling N a representation of F , one must be in a position to rule out other neural processes as representations of F , meaning one must specify how N ’s correspondence with F distinctively supports the behavior(s) in question. This means that representational claims should be accompanied by a hypothesis of the behavior-generating mechanism(s) n which the representation plays a part. Placing N in the context of a specific mechanism is a way of describing an exclusive causal role for N , meeting our third criterion. Philosophers from wide ranging intellectual traditions have championed the idea that even highly complex behaviors can be understood in terms of coordinated, internal mechanisms. Descartes famously endeavored to rigorously specify how behavior is subserved by correspondences between the world and nervous system (Descartes 1633) . Although many of Descartes’ specific hypotheses have been disconfirmed, his seminal work exemplifies the aim of mechanistically describing the internal processes that generate behavior. Further back, a mechanistic approach to understanding behavior is also expressed in ancient India, in the Samkhya-Yoga school of thought (Radhakrishnan and Moore 1957) See Patanjali’s Yoga-Sutras, and especially verses 23 and 27 of the Samkhya-Karika text and associated commentaries, available in (Radhakrishnan and Moore 1957) . Notwithstanding major advancements in our observational and theoretical techniques since the time of these theories, they exemplify the project of understanding perceptually and cognitively guided behavior in terms of physically inner processes that correspond in specific ways to external processes. Thus, we stand on well-trod ground in asserting that a basic class of internal processes in terms of which we explain behavior must do more than satisfy criteria 1 and 2. In contemporary philosophical work, support for this third criterion is expressed in terms of the idea that representations are states with information content that plays a specific functional role (Markman and Dietrich 2000; Piccinini and Scarantino 2011; Egan 2014) . In other words, an internal state is said to earn its explanatory keep as a representation only insofar as some behavior-supporting process makes use of the information content in that state. This idea is often further cashed out in terms of a causal role in a hypothesized mechanism, similarly to our brief description above. How this mechanistic approach is to be precisely spelled out is a major topic of recent philosophical literature (Cummins 1975; Cartwright and Cartwright 1999; Machamer, Darden, and Craver 2000; Glennan 2002; Hardcastle 2002; Bechtel and Abrahamsen 2005; Craver 2007; Baumgartner and Gebharter 2015) . Some notable dissenters have argued that the challenge of finding distinctive causal roles that internal states play in virtue of the information they carry cannot plausibly be met, and so they deny the explanatory value of representation altogether (Ramsey 2007; Egan 2020) . Our point here is just that these strains of philosophy all suggest that representations are partly defined by their contribution in a specific mechanism underlying behavior. This supports our contention that neuroscientists should claim that some pattern of neural activity N represents F only when they can describe the distinctive role that N ’s correspondence with F plays in generating behavior, thereby ruling out other F -corresponding states as potential representations. In practice, the criterion means that neural activity with overlapping information and causal roles must not be seen in parallel streams: if other brain regions have similar activity and a similar effect on behavior, this activity is either also implicated in the representation, or such activity is not distinctive enough to have a specific representational role. To properly establish the causal role a particular neural circuit plays in mediating behavior, and hence its capacity for representation, a comprehensive view of the nervous system’s activity or anatomy must therefore be obtained. Whole-brain imaging studies that promise to probe this level of detail can now be performed in some smaller organisms (e.g. Hydra, zebrafish larvae, and C. elegans). Exclusivity is difficult to establish in neuroscience settings but modern developments make it more feasible. riterion 4: Teleology
The fourth criterion says that a representation must serve a specific aim, goal or purpose (“telos” is ancient Greek for purpose). This criterion is perhaps most clearly motivated by a consideration of mistakes, illusions, and other cases that deviate from ordinary instances of successful perception and movement. Such cases highlight the fallibility of representations, revealing that representational claims typically entail a standard that actual representations can fall short of. To illustrate by way of non-neural examples, a map or an assertion can be accurate or inaccurate, true or false, or otherwise meet or fail to meet a standard that is integral to the kind of thing the map or assertion is - a representation of some place or fact. The claim that a map is a map of a certain place implies some accuracy conditions according to which the map is more or less able to serve the aims of the map-user. Similarly, perceptual states are sometimes inaccurate, beliefs are sometimes false, and inferences are sometimes unwarranted by one's own evidence. Neuroscience aims to explain patterns of misperception and misdirected behavior as well as the more mundane “good cases,” so fallibility is essential to representations that neuroscientists try to identify. Given a pattern of neural activity N whose correspondence with F plays a role in a hypothesized mechanism - that is, given a process that meets our first three criteria - the question is left open as to what justifies a teleological interpretation of that correspondence. Some basis is needed for saying that N is supposed to correspond with F even if, say, corresponding with G would usually fulfill the same mechanistic role. Therefore the claim that N is a representation ultimately requires a commitment to a teleological framework in which to interpret N ’s role in a behavior-generating mechanism. Philosophers of mind going back to antiquity have been focused on analyzing the problem of erroneous mental states (Caston 2019; Taieb 2018) . Owing to the famous use of the term by Brentano in 1874, this is also commonly understood in philosophy as the problem of intentionality (Brentano 2012) . Mental representations are supposed to help us understand how mistakes and failures of reference happen, which is largely why they are said to exhibit intentionality; in short, with no intention there can be no error. Thus, one is in no position to say when a physical system is representing something unless one also is in position to say when it is misrepresenting (Bogdan 1986) . By relying on a teleological view of why some neural activity and the mechanism it is part of is aimed at achieving a particular result, one can determine the difference between purpose-achieving representations and defective ones (misrepresentations). Contemporary philosophical views of what determines the purposive or “proper'' functional role of some component in a mechanistic process can be distinguished according to whether they are backward-looking or forward-looking. Backwards-looking, also called “etiological” approaches essentially propose that a component’s function is what it did in the past to contribute to its current presence in the larger system. On this approach, a component’s function explains “why it is there,” in a causal, evolutionary sense, or in other words the function is what the component was *selected for* (Wright 1973; Millikan 1989; Papineau 1987; Dretske 1988; Godfrey-Smith 1994; Neander 1995; Garson 2017; Shea 2018) In the (organismal) systems of interest, this idea of selection often involves that of reproductively established families. For example, given a chain of self-reproducing systems that all have a heart as a component, and assuming that it was past hearts’ contribution to the circulation of blood that mainly explains the presence of similar hearts today, one should infer, on this view, that blood-circulating is the proper function of the heart. While these hearts have other interesting characteristic properties - for instance, they make a rhythmic sound and reflect certain wavelengths of light more than others - these do not qualify as the heart’s function because these are not what it was selected for. In analogous fashion, a backward-looking Teleological view of neural representation would derive the content of such epresentations from the selection history of the neural activity. While the neural activity of interest might correspond with many different things, only a subset of those things might be relevant in the selection history of that activity, so only that subset would count for what the neural activity represents, on this approach. To confidently say what the neural activity represents then would require careful considerations of the evolutionary or developmental trajectory of that kind of activity. Forward-looking Teleological views, sometimes called Goal-Contribution views, are based on a characterization of systems that exhibit goal-directed behavior, and these views understand functional components in terms of their contributions to specific goals in such systems. On this approach, one must start with some way of discerning which systems have goals, at least in general terms and perhaps also in formal terms. Typically some measures of mutability and robustness are taken to be evidence for the goal-directedness of some system’s behavior. That is, one asks how various are the circumstances from which the system will proceed to the outcome (goal), and how various are the perturbations or obstacles that the processes to that outcome can accommodate. Different philosophers have articulated these notions somewhat differently, but they similarly try to account for goals as outcomes that the system’s behavior would flexibly adjust to ensure across a wide range of counterfactual scenarios (Nagel 1977; Bigelow and Pargetter 1987; Boorse 2002; Mossio, Saborido, and Moreno 2009; Cao 2012) . On this kind of view the systems of interest are *constitutively teleological,* which means that a distinction between successes and failures is entailed in the basic description of the behavior of the system of interest. This teleology at the system-level is what determines the proper roles of functional components. This provides for a (non-etiological) notion of “why the component is there,” which justifies treating other features of the component as functionally irrelevant and justifies treating some activities as malfunctional. For instance, on this view one can claim that “the heart is there to circulate blood” insofar as blood-circulation is involved in some goal-directed activity of the bodily system in which the heart functions. Accordingly, whether some neural activity N has the function of corresponding with some F would depend on the ensemble of functional components that together generate a specific, goal-directed activity. Backward- and forward- looking teleological views each have their virtues and each face certain theoretical challenges. A key issue for backwards-looking views is to define the relevant causal (non-intentional) notion of “selection.” It is worth noting that there are intuitive cases of proper functions involving objects too novel to have influenced reproductive selection. For instance, human neural activities might represent technologies and social structures that did not exist one or a few generations ago. Relatedly, sometimes components seem to suddenly acquire new functions, as might have happened the first time wings were used for digging or hands used for signing. Proponents of a backward-looking approach have offered ways of addressing novel functions, but examining their arguments in detail is beyond our scope here. Forward-looking views can straightforwardly make sense of novel functions, but face the basic hurdle of characterizing goal-directedness in non-trivial, general terms. One can reasonably ask this camp to answer how a naive investigator should distinguish goal-directed systems from non goal-directed ones. This suggests the need for a way to quantify the kind of flexibility and robustness that is to count as evidence of goal-directedness. Whatever formal approach one takes, one must also deal with the complicating possibility of one system having conflicting goals. Again, we leave more thorough Briefly, some have appealed to the idea of a component having a function to acquire a function, such that acquired functions can be shaped according to novel circumstances (Millikan 1984; Garson and Papineau 2019) . Garson (Garson 2017) offers an account of selection in terms of differential retention in a population, which can occur without reproductive lineages and which, he argues, applies to the selection of synapses. nalyses of these issues for a different discussion. Whether just one of these two approaches to teleology can ultimately resolve all the cases of interest does not matter here; perhaps a pluralistic view will prove best (Preston 1998) . Both views provide a frame for analyzing the components of living things that affirms what is distinctive about them (Mayr, Alexander Agassiz Professor of Zoology (Emeritus) Museum of Comparative Zoology Ernst Mayr, and Ernst (Harvard University Mayr, Massachusetts) 2004) . We have tried to clarify what is involved in a teleological view of neural components of behavior, because a satisfactory notion of representation in neuroscience must rely on such a view. Arguably, a teleological view operates in the background of most neuroscientific research, implicit in the researchers’ choices of which external variables to investigate and sometimes made explicit. For instance, in the study of chemotaxis in the worm C elegans , researchers set out to find neural representations of chemical gradients that are important to the survival of the creature. In this case a teleological interpretation of the information-carrying function of the neural activity in question is well-supported by widely shared, general hypotheses about selection history (backward looking), or by ascribing the goal of survival to the creature (forward looking). For organisms like this, whose behavioral (and thus representational) repertoire is, arguably, simple enough, one can be relatively safe from doubt about the purpose some process serves, and backward- and forward-looking views will tend to agree. When there are only a few plausible purposes to choose from, a careful consideration of one’s teleological framework is not essential and, at least in some domains, these teleological arguments are relatively straightforward and widely shared among neuroscientists. Teleological views are also prevalent in the study of more complex behaviors with high contextual variability. However, in such cases, it becomes highly non-obvious which goals an organism pursues and which mechanisms it relies on to do so, and also non-obvious how such mechanisms served the reproductive successes of its ancestors. For example, when it comes to the function of primary visual cortex, one line of research views it in terms of sparsifying neural codes in response to natural scenes (Olshausen and Field 1996) , potentially to minimize the energy consumption of the system (Attwell and Laughlin 2001) . Others suggest that the early parts of the visual system have been optimized to support object recognition, comparing tuning in brains with those of neural networks (Kriegeskorte 2015; Yamins and DiCarlo 2016) . Yet further authors suggest a role of the representation in establishing certainty (Knill and Pouget 2004) . In such complex cases, leaving teleological assumptions unquestioned obfuscates crucial aspects of what one is claiming to have identified as a representation. While the sub-discipline of normative models in neuroscience aims to clarify our teleological assumptions, they are usually only explicitly discussed in the computational literature. This removes a significant basis for potential disagreement from the discussion, setting the stage for scientists to talk past one another rather than communicate meaningful hypotheses about how neurons help generate behavior. Work aiming to uncover neural representations in moderately complex cases should clearly explain one’s teleological interpretation, either in backward- or forward-looking terms or both. To bring teleology more into the foreground in neuroscience, one approach would be to rely on data on the evolutionary development of the neural process under investigation. Some recent work in this There are forward-looking views with minimal requirements on the complexity of goal-directedness, for which systems like a prokaryotic cell, a Watt Governor, and Ashby’s homeostat (1960), would all count as goal-directed (Nagel 1977; Boorse 2002) . Others defend more substantive requirements, typically including mechanisms to maintain a constantly-decaying body that contains those mechanisms (Mossio, Saborido, and Moreno 2009; Bigelow and Pargetter 1987) pirit relies on knowledge about the phylogenetic tree to argue for constraints on the kinds of behaviors that neuroscience should study (Cisek 2019) . Although this work does not speak directly to identifying neural representations, it aligns with the general principle of looking to teleology derived from reproductive success to refine neuroscientific research. An approach more aligned with forward-looking teleology is to start by inferring organismal goals from macroscopic behavior (Ng et al. 2000; Körding 2007) , carefully observing the kinds of errors the organism is prone to make, and look for representations that seem essential to explain the observed patterns of successful and errant behavior (Krakauer et al. 2017) . In section 2 we noted that a common approach to representation starts by trying to find what some neural activity corresponds with, leaving aside questions to do with the purposive behavior of the whole organism. We have tried to show how inadequate such an approach is. To properly investigate representations, observations of what neural activity corresponds with must be integrated into a well-described causal and teleological framework. Conclusion
Our analysis here was aimed at clarifying the role that representations are supposed to play in neuroscience, and illuminating what is needed for them to fulfill that role. It is often nominally acknowledged that representations are meant to explain behavior and that they figure in complex, functional processes, but it is not so often that the theoretical implications of representations having these features are seriously examined. Understanding the function of a given brain region, neural circuit or neuron is a common aim of neuroscientific studies - insofar as talk of “functions” entails some notion of what fulfills the goals of a species or individual, this suggests a strong commitment to some form of teleology. However, teleological ideas are often hidden or ignored, as experimental data itself can usually only speak to the question of encoding or even only to correlation. This casual approach to thinking about the purposive structure of representations offers no principled basis on which to distinguish the cases of misrepresentation, wherein the observed encoding relationship breaks down but the neurons are used as they normally are, leading to inappropriate behavior. We have tried to show that neuroscientists who want to establish neural representations should be committed to all four of our criteria, which means many current efforts to identify representations lack significant theoretical support. Our point is not that all neuroscientific work should try to discover representations using the four criteria that we have outlined here. Clearer communication in the field will be facilitated by avoiding overstating hypotheses in terms that have a deeper meaning than what a particular experiment can establish. Partly this will mean being careful to describe results in merely correlational, or causal-mechanistic terms where it is not possible to establish all four criteria. Research that does not establish all four criteria can still be quite useful - there are things it would be good to know about neural systems besides what content is represented in which processes. For neuroscientists who do want to look into representation, there is no way to do so that avoids the issue of accounting for possible misrepresentation, so these neuroscientists should openly embrace the teleology their claims depend on. For many interesting neural processes, relevant evolutionary data may not be available and the right purposive description of the process may be unclear, making it extremely difficult or impossible to establish our third or fourth criteria. These difficulties may act in combination with a disregard of philosophy within parts of neuroscience (Laplane et al. 2019) . Neuroscientists sometimes seem to be at pains to elude the abstract conceptual relations involved in our criteria, even when trying to delve into the irreducibly philosophical notion of representation. This can result in a concerning disconnect between the statements that neuroscientists want to make and those that they are epistemologically entitled to make. e propose that, by restricting claims about representation according to whether the four criteria above are clearly substantiated, neuroscientists will be able to do more significant and illuminating research. More broadly, we have tried to show how neuroscientific research should take seriously the philosophical assumptions that underlie it, especially when it comes to identifying representations. Philosophers are not well-positioned to make specific hypotheses about how neural systems represent things, but they have thought long and hard about the intricate conceptual interplay among notions of correspondence, causation, teleology, explanation, and related ideas. To improve our understanding of how complex behavior depends on interactions among neurons, we should not ignore ideas from a millenia-spanning sub-discipline full of competing analyses about the source of such behavior. This paper means to exemplify the sorely needed collaboration between neuroscience and philosophy. We hope to have shown how the investigation of neural systems should be grounded in philosophical reflection on what representation involves. Acknowledgements: This work was supported by NIH (1-R01-EB028162-01, R01EY021579) and by the
University of Pennsylvania Office of the Vice Provost for Research. _______________________________ References
Allen, William E., Isaac V. Kauvar, Michael Z. Chen, Ethan B. Richman, Samuel J. Yang, Ken Chan, Viviana Gradinaru, Benjamin E. Deverman, Liqun Luo, and Karl Deisseroth. 2017. “Global Representations of Goal-Directed Behavior in Distinct Cell Types of Mouse Neocortex.” Neuron
94 (4): 891–907.e6. Attwell, D., and S. B. Laughlin. 2001. “An Energy Budget for Signaling in the Grey Matter of the Brain.”
Journal of Cerebral Blood Flow and Metabolism: Official Journal of the International Society of Cerebral Blood Flow and Metabolism
21 (10): 1133–45. Baumgartner, Michael, and Alexander Gebharter. 2015. “Constitutive Relevance, Mutual Manipulability, and Fat-Handedness.” The British Journal for the Philosophy of Science
67 (3): 731–56. Bechtel, William, and Adele Abrahamsen. 2005. “Explanation: A Mechanist Alternative.” Studies in History and Philosophy of Biological and Biomedical Sciences
36 (2): 421–41. Bickhard, Mark H. 2000. “Information and Representation in Autonomous Agents.” Cognitive Systems Research Synthese
166 (3): 547–91. Bigelow, John, and Robert Pargetter. 1987. “Function.” The Journal of Philosophy
84 (4): 181–96. Bogdan, R. 1986. Belief: Form, Content, and Function . New York: Oxford University Press. Boorse, Christopher. 2002. “A Rebuttal on Functions.” Functions: New Essays in the Philosophy of Psychology and Biology , 63–112. Brentano, Franz. 2012. Psychology from an Empirical Standpoint . Routledge. Brette, Romain. 2018. “Is Coding a Relevant Metaphor for the Brain?” The Behavioral and Brain Sciences
42 (July): e215. Burgess, Neil. 2014. “The 2014 Nobel Prize in Physiology or Medicine: A Spatial Model for Cognitive Neuroscience.” Neuron
84 (6): 1120–25. Cao, Rosa. 2012. “A Teleosemantic Approach to Information in the Brain.” Biology & Philosophy
27 (1): 49–71. Cartwright, Nancy, and of Philosophy Nancy Cartwright. 1999. The Dappled World: A Study of the Boundaries of Science . Cambridge University Press. Caston, Victor. 2019. “Intentionality in Ancient Philosophy.” In The Stanford Encyclopedia of Philosophy , edited by Edward N. Zalta, Winter 2019. Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/win2019/entries/intentionality-ancient/ . halmers, David J. 1994. “On Implementing a Computation.” Minds and Machines Philosophical Perspectives. A Supplement to Nous
4: 343–82. Cisek, Paul. 2019. “Resynthesizing Behavior through Phylogenetic Refinement.” Attention, Perception & Psychophysics
81 (7): 2265–87. Craver, Carl F. 2007. Explaining the Brain: Mechanisms and the Mosaic Unity of Neuroscience . Clarendon Press. Cummins, Robert. 1975. “Functional Explanation.” The Journal of Philosophy
72: 741–64. deCharms, R. C., and A. Zador. 2000. “Neural Representation and the Cortical Code.” Annual Review of Neuroscience
23: 613–47. Deisseroth, Karl, Guoping Feng, Ania K. Majewska, Gero Miesenböck, Alice Ting, and Mark J. Schnitzer. 2006. “Next-Generation Optical Technologies for Illuminating Genetically Targeted Brain Circuits.” The Journal of Neuroscience: The Official Journal of the Society for Neuroscience
26 (41): 10380–86. Descartes, René. 1633/1972. Treatise of Man: French Text with Translation and Commentary, Trans. Thomas Steele Hall . Cambridge, Mass.: Harvard University Press. Dretske, Fred. 1981. “Knowledge and the Flow of Information.” https://philarchive.org/rec/DREKAT?fId=&eId=DREKAT&gId=&cId=&tSort=ct+desc . ———. 1988. “The Explanatory Role of Content.” In Contents of Thought , edited by Robert H. Grimm and D. D. Merrill. University of Arizona Press. Egan, Frances. 2014. “How to Think about Mental Content.” Philosophical Studies
170 (1): 115–35. ———. 2020. “A Deflationary Account of Mental Representation.” What Are Mental Representations . https://books.google.com/books?hl=en&lr=&id=Af7-DwAAQBAJ&oi=fnd&pg=PA26&dq=egan+deflationary&ots=do8SceQMa6&sig=eiRGrXhv_6aWHT7GbGM__Z2-QLc . Eliasmith, Chris. 2005. “A New Perspective on Representational Problems.” Journal of Cognitive Science
6: 97–123. Field, Hartry H. 1978. “Mental Representation.” Erkenntnis. An International Journal of Analytic Philosophy
13 (1): 9–61. Fodor, Jerry A. 1990. “A Theory of Content and Other Essays” 270. https://psycnet.apa.org/fulltext/1990-98119-000.pdf . Fodor, Jerry A., and Jerry A. Fodor. 1987. Psychosemantics: The Problem of Meaning in the Philosophy of Mind . MIT Press. Garson, Justin. 2017. “A Generalized Selected Effects Theory of Function.” Philosophy of Science
84 (3): 523–43. Garson, Justin, and David Papineau. 2019. “Teleosemantics, Selection and Novel Contents.” Biology & Philosophy
34 (3). https://doi.org/ . Glennan, Stuart. 2002. “Rethinking Mechanistic Explanation.” Philosophy of Science
69 (S3): S342–53. Godfrey-Smith, Peter. 1994. “A Modern History Theory of Functions.” Noûs
28 (3): 344–62. Grush, Rick. 2004. “The Emulation Theory of Representation: Motor Control, Imagery, and Perception.”
The Behavioral and Brain Sciences
27 (3): 377–96; discussion 396–442. Han, Shuting, Ekaterina Taralova, Christophe Dupre, and Rafael Yuste. 2018. “Comprehensive Machine Learning Analysis of Hydra Behavior Reveals a Stable Basal Behavioral Repertoire.” eLife . Hardcastle, Valerie Gray. 2002. “On the Normativity of Functions.” Functions: New Essays in the Philosophy of Psychology and Biology , 144–56. Hatfield, Gary. 1991. “Representation in Perception and Cognition: Connectionist Affordances.” Philosophy and Connectionist Theory , 163–95. Hirokawa, Junya, Alexander Vaughan, Paul Masset, Torben Ott, and Adam Kepecs. 2019. “Frontal Cortex Neuron Types Categorically Encode Single Decision Variables.” Nature
576 (7787): 446–51. Horgan, Terence. 1989. “Mental Quausation.” Philosophical Perspectives. A Supplement to Nous
3: 47–76. ume, David. 2019. A Treatise of Human Nature . e-artnow. Huxley, A. F. 1963. “The Quantitative Analysis of Excitation and Conduction in Nerve.” Les Prix Nobel Epilepsy Research
166 (October): 106363. Jones, Ilenna Simone, and Konrad Paul Kording. 2019. “Quantifying the Role of Neurons for Behavior Is a Mediation Question.” The Behavioral and Brain Sciences . Knill, David C., and Alexandre Pouget. 2004. “The Bayesian Brain: The Role of Uncertainty in Neural Coding and Computation.” Trends in Neurosciences
27 (12): 712–19. Körding, K. 2007. “Decision Theory: What‘ Should’ the Nervous System Do?” Science . https://science.sciencemag.org/content/318/5850/606.abstract?casa_token=u0pAIbETdKEAAAAA:_id8EUEZyNDlXkFJuL5ld-yNkZaNpBK6f3tnC5gZOhMhcCODXj-Y2gvtamTP5_aGh35tVz7IgcgCMw . Krakauer, John W., Asif A. Ghazanfar, Alex Gomez-Marin, Malcolm A. MacIver, and David Poeppel. 2017. “Neuroscience Needs Behavior: Correcting a Reductionist Bias.” Neuron
93 (3): 480–90. Kriegeskorte, Nikolaus. 2015. “Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing.” Annual Review of Vision Science Proceedings of the National Academy of Sciences of the United States of America
116 (10): 3948–52. Latimer, Kenneth W., Jacob L. Yates, Miriam L. R. Meister, Alexander C. Huk, and Jonathan W. Pillow. 2015. “NEURONAL MODELING. Single-Trial Spike Trains in Parietal Cortex Reveal Discrete Steps during Decision-Making.” Science
349 (6244): 184–87. Locke, John. 1948. “An Essay Concerning Human Understanding, 1690.” https://psycnet.apa.org/record/2006-10213-008 . Machamer, Peter, Lindley Darden, and Carl F. Craver. 2000. “Thinking about Mechanisms.” Philosophy of Science
67 (1): 1–25. Marinescu, Ioana E., Patrick N. Lawlor, and Konrad P. Kording. 2018. “Quasi-Experimental Causality in Neuroscience and Behavioural Research.” Nature Human Behaviour Cognitive Psychology
40 (2): 138–71. Mayr, Ernst, Alexander Agassiz Professor of Zoology (Emeritus) Museum of Comparative Zoology Ernst Mayr, and Ernst (Harvard University Mayr, Massachusetts). 2004. What Makes Biology Unique?: Considerations on the Autonomy of a Scientific Discipline . Cambridge University Press. Millikan, Ruth Garrett. 1984. Language, Thought, and Other Biological Categories: New Foundations for Realism . MIT Press. ———. 1989. “In Defense of Proper Functions.” Philosophy of Science
56 (2): 288–302. ———. 2001. “What Has Natural Information to Do with Intentional Representation?” Royal Institute Of Philosophy Supplement , 105–26. Mirski, Robert, and Mark H. Bickhard. 2019. “Encodingism Is Not Just a Bad Metaphor.” The Behavioral and Brain Sciences . Mossio, Matteo, Cristian Saborido, and Alvaro Moreno. 2009. “An Organizational Account of Biological Functions.” The British Journal for the Philosophy of Science
60 (4): 813–41. Nagel, Ernest. 1977. “Teleology Revisited.” https://philarchive.org/rec/NAGTR . Neander, Karen. 1995. “Misrepresenting & Malfunctioning.” Philosophical Studies
79 (2): 109–41. Ng, Andrew Y., Stuart J. Russell, and Others. 2000. “Algorithms for Inverse Reinforcement Learning.” In
Icml , 1:2. eecs.harvard.edu. O’Brien, Gerard, and Jon Opie. 2004. “Chapter 1 - Notes Toward a Structuralist Theory of Mental Representation.” In Representation in Mind , edited by Hugh Clapin, Phillip Staines, and Peter lezak, 1:1–20. Oxford: Elsevier. Olshausen, B. A., and D. J. Field. 1996. “Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images.” Nature
381 (6583): 607–9. Papineau, D. 1987. “Representation and Reality.” Penfield, Wilder, and Theodore Rasmussen. 1950. The Cerebral Cortex of Man; a Clinical Study of Localization of Function, by Wilder Penfield and Theodore Rasmussen . Piccinini, Gualtiero. 2008. “Computation without Representation.” Philosophical Studies
137 (2): 205–41. Piccinini, Gualtiero, and Andrea Scarantino. 2011. “Information Processing, Computation, and Cognition.” Journal of Biological Physics
37 (1): 1–38. Pillow, Jonathan W., Jonathon Shlens, Liam Paninski, Alexander Sher, Alan M. Litke, E. J. Chichilnisky, and Eero P. Simoncelli. 2008. “Spatio-Temporal Correlations and Visual Signalling in a Complete Neuronal Population.” Nature
454 (7207): 995–99. Preston, Beth. 1998. “Why Is a Wing Like a Spoon? A Pluralist Theory of Function.” The Journal of Philosophy
95 (5): 215–54. Purves, Dale, George Augustine, David Fitzpatrick, William Hall, Anthony LaMantia, Leonard White, Richard Mooney, and Michael Platt. 2018. Neuroscience . Radhakrishnan, Sarvepalli, and Charles A. Moore. 1957. “A Sourcebook in Indian.” Philosophy . Ramsey, William M. 2007. Representation Reconsidered . Cambridge University Press. Rieke, Fred. 1997. Spikes: Exploring the Neural Code . Bradford Books. Ruff, Douglas A., and Marlene R. Cohen. 2019. “Simultaneous Multi-Area Recordings Suggest That Attention Improves Performance by Reshaping Stimulus Representations.” Nature Neuroscience
22 (10): 1669–76. Schiff, W., J. A. Caviness, and J. J. Gibson. 1962. “Persistent Fear Responses in Rhesus Monkeys to the Optical Stimulus of ‘Looming.’” Science
136 (3520): 982–83. Shadlen, M. N., and W. T. Newsome. 2001. “Neural Basis of a Perceptual Decision in the Parietal Cortex (area LIP) of the Rhesus Monkey.” Journal of Neurophysiology
86 (4): 1916–36. Shea, Nicholas. 2018. Representation in Cognitive Science . Oxford University Press. Simoncelli, E. P., and B. A. Olshausen. 2001. “Natural Image Statistics and Neural Representation.” Annual Review of Neuroscience
24: 1193–1216. Soh, Zu, Kazuma Sakamoto, Michiyo Suzuki, Yuichi Iino, and Toshio Tsuji. 2018. “A Computational Model of Internal Representations of Chemical Gradients in Environments for Chemotaxis of Caenorhabditis Elegans.” Scientific Reports Relational Intentionality: Brentano and the Aristotelian Tradition . Springer, Cham. Usher, Marius. 2001. “A Statistical Referential Theory of Content: Using Information Theory to Account for Misrepresentation.” Mind & Language
16 (3): 311–34. Wright, Larry. 1973. “Function.” The Philosophical Review
82: 139–68. Yamins, Daniel L. K., and James J. DiCarlo. 2016. “Using Goal-Driven Deep Learning Models to Understand Sensory Cortex.” Nature Neuroscience
19 (3): 356–65. Zhao, Xinyu, Mingna Liu, and Jianhua Cang. 2014. “Visual Cortex Modulates the Magnitude but Not the Selectivity of Looming-Evoked Responses in the Superior Colliculus of Awake Mice.” Neuron
84 (1): 202–13. Zhao, Xuelong, and Konrad P. Kording. 2018. “Rate Fluctuations Not Steps Dominate LIP Activity during Decision-Making.” Cold Spring Harbor Laboratory . https://doi.org/ ..