Cogn Process (2007) 8:115–131 DOI 10.1007/s10339-007-0164-0
RESEARCH REPORT
The symbol detachment problem Giovanni Pezzulo Æ Cristiano Castelfranchi
Received: 9 January 2007 / Revised: 28 February 2007 / Accepted: 6 March 2007 / Published online: 4 April 2007 Marta Olivetti Belardinelli and Springer-Verlag 2007
Abstract In situated and embodied approaches it is commonly assumed that the dynamics of sensorimotor engagement between an adaptive agent and its environment are crucial in understanding natural cognition. This perspective permits to address the symbol grounding problem, since the aboutness of any mental state arising during agent-environment engagement is guaranteed by their continuous coupling. However, cognitive agents are also able to formulate representations that are detached from the current state of affairs, such as expectations and goals. Moreover, they can act on their representations before—or instead of—acting directly on the environment, for example building the plan of a bridge and not directly the bridge. On the basis of representations, actions such as planning, remembering or imagining are possible that are disengaged from the current sensorimotor cycle, and often functional to future-oriented conducts. A new problem thus has to be acknowledged, the symbol detachment problem: how and why do situated agents develop representations that are detached from their current sensorimotor interaction, but nevertheless preserve grounding and aboutness? How do cognitive agents progressively acquire a range of capabilities permitting them to deal not only with the current situation but also with alternative, in particular future states of affairs? How do they develop the capability of acting on their representations instead of acting directly on the world? In a theoretical and developmental perspective, we propose that anticipation plays a crucial role in the G. Pezzulo (&) C. Castelfranchi Istituto di Scienze e Tecnologie della Cognizione, CNR, Via S. Martino della Battaglia, 44-00185 Rome, Italy e-mail:
[email protected] C. Castelfranchi e-mail:
[email protected]
detachment process: anticipatory representations, originally detached from the sensorimotor cycle for the sake of action control, are successively exapted for bootstrapping increasingly complex cognitive capabilities. Keywords Anticipation Detachment Disengagement Symbol Representation Autonomy
Introduction A great emphasis has been put in the last decade in the symbol grounding problem (Harnad 1990), especially in situated and embodied approaches to cognition (Brooks 1991; Glenberg 1997). However, another basilar problem has been given much less attention, the symbol detachment problem—that is, why and how agents evolve representations which are detached from their sensorimotor cycle, and disengaged from immediate action? Cognitive agents are able not only to adapt to the present circumstances by means of sensorimotor interaction, but also to conceive the past and the future; to pursue goals which are out of their immediate range of action; to formulate, compare and select potential future courses of action instead of or before acting in the environment. A problem arises in understanding those capabilities. It is now well accepted that representations have to be (at least ontogenetically or phylogenetically speaking) grounded in sensorimotor interaction. At the same time, they are however, detached from the current sensorimotor interaction, in the sense that they are not necessarily about the current state of affairs but, in the case of expectations and goals, about states of affairs which are still to be realized and possibly will never be realized. Moreover, it has to be explained how representations can acquire the capability to
123
116
substitute their referent in the piagetian sense (Piaget 1954): it is possible to use representations instead of the object they represent. Another formulation of the problem is that cognitive agents are engaged in sensorimotor interaction with their environment. Embodied cognition stresses the tight coupling between brain, body and environment. At the same time, however, cognitive agents are also able to disengage from the current state of affair, and to represent and act in different states of affairs, for example planning by imagining and comparing different courses of actions. Detachment and disengagement are two sides of the same coin.1 The investigation of the developmental pathway leading from sensorimotor engagement to symbolic manipulation is not novel in cognitive science. The most comprehensive account is provided by Piaget (1954) who distinguishes between sensorimotor schemas, which are goal-directed structures of practical activity emerging from the child’s physical interaction with the environment, and symbols, which have representational content, and proposes a number of stages between the level of perception and the intellectual level. Conceptual knowledge of a child begins after the sensorimotor period, when his representations acquire a new function: ‘‘what happens to these corresponding or isomorphic events? Subsequently they should detach themselves from the simple correspondence function, they must stop working as simple mirrors and start to substitute’’. Differently from Piaget (1954), however, we will argue that manipulation of representations, and substitution, can be executed even before the symbolic phase. As an example, consider that a mechanic can assemble and dismantle a motor in his mind before doing it in practice: this activity could imply re-enactment of the same structures used for sensorimotor interaction and not full-fledged symbols. Overview of the paper How and why representations detach from current sensorimotor engagement? This problem has many facets. It is 1
A caveat: with the term ‘‘detachment’’ we are not embracing here the idea that cognition is detached from the nature and functioning of organisms or their environments. We claim instead that some organisms have internal models of their environments serving the function of representing it, although not being causally determined by current stimuli (here comes the term ‘‘detachment’’); that the organisms can act on their representations before or instead of acting directly on their environments (here comes the term ‘‘disengagement’’); and that nevertheless the final goal of this form of cognition is acting in the environment. Our point here thus in not to defend the idea that acting on representations is the unique, or a privileged form of cognition, but to investigate how and why it is possible in natural and artificial organisms.
123
Cogn Process (2007) 8:115–131
for example questionable which is the level of detachment that representations achieve, and recent theories of embodied cognition have challenged traditional cognitive science and suggested that sensorimotor interaction is involved also in high level cognition. Another relevant problem is how representations can detach, although preserving grounding and aboutness, which is the mechanism permitting such development, and in which phases. Lastly, it has to be addressed the problem of why all that happens, which are the evolutionary pressures for this process, and which are the functional roles of representations at different states of detachment. In Sect. 2 two features of representations, detachment and intentionality, are introduced for distinguishing them from mere information states. In Sect. 3 the symbol detachment problem is formulated, and related difficulties, such as how representations can preserve aboutness, relevance and normativity, are introduced. In Sect. 4 it is proposed that detachment depends mainly on the development of anticipatory capabilities that originally served for action control, and were successively exapted. On the basis of the previous analysis of representations, it is also discussed why this mechanism permits to preserve their grounding while detaching. In Sect. 5 an evolutionary pathway for the detachment process, involving several stages, is proposed. With respect to the existing literature, this part is the main contribution of the paper, since we suggest several related steps, also illustrating a parallel detachment process for representations and goals. In the conclusions the implications of this work for natural and artificial cognition are discussed.
Features and uses of representations Representations are often conceived as ‘‘vicarious’’ stimuli, i.e. internally stored states which serve to activate and regulate behavior. However, this simple definition fails to discriminate between information states (or presentations) and true representations. According to Clark and Grush (1999): ‘‘The difference is thus between inner states that continuously link the processing to the on-going evolution of extra-neural reality and inner states that recapitulate the dynamics of extra-neural reality without depending on a constant physical linkage between the inner states and what they are about. Only the latter constitute cases of what we are calling ‘strong internal representation’.’’ Information states guide our activities by creating closed loop interactions with the environment; as an example, consider the thermostat that measures temperature and acts for modifying it. In order to maintain aboutness of information states it is however necessary to have a continuous coupling (provided by sensors and
Cogn Process (2007) 8:115–131
117
actuators) with the environment, otherwise the system stops working properly. Representations have instead both intentionality and detachment. A definition of intentionality is provided by Brentano (1985): ‘‘As every mental phenomenon is characterized by what the scholastics of the middle ages called the intentional (or mental) inexistence of an object, and what we might call, though not wholly unambiguously, reference to a content, direction toward an object (which is not to be understood here as meaning a thing), or immanent objectivity. Every mental phenomenon includes something as object within itself [...]’’. Thus representations stand-in for, or are about something else. It is not required that representations are related to the content of the represented entity (their object): a representation of ice is not cold. However, they must have a specific functional relationship with their object (e.g. to co-vary in a certain way). If it is broken, the representation loses aboutness and grounding. The other important point is that the dynamics of representations depend at least partially on processes which are internal to the cognitive system, while the dynamics regulating the information states depend only on the environment. Thus, differently from inner states that require a continuous coupling with the environment, representations can be detached. According to Taylor (1971): ‘‘To be able to talk about things is to be potentially aware of them outside of any particular transaction with them; it is to be potentially aware of them not just in their behavioral relevance to some activity we are now engaged in, but also in a disengaged way’’. Gardenfors (2004) uses a similar notion of detachment for distinguishing agents which are able to refer to situations which are not motivated by actual or recent stimuli. Detachment permits: (1) to break the sensorimotor coupling to an extent which is impossible for information states: representations are conceived for dealing with the absence of their referent, thus mechanisms based on them are robust with respect to lacking stimuli and (2) to act directly on representations instead of acting on their object. On the contrary, information states serve only to act on the environment (in the case of a thermostat, raising the temperature). For exampple, by using representations an agent can build up, evaluate, select or discard one of his plans before acting in the world, and let its hypotheses die in its stead (Popper 1996).
human cognition and we call its medium a representation. We argue, largely in accordance with recent situated and embodied approaches (Damasio 1989; Barsalou 1999; Glenberg 1997) that representations originate from the sensorimotor apparatus and remain related to it; for this reason, conceptual knowledge of objects and actions retain a sensorimotor content. In the developmental perspective we propose, we also argue that representations originate from internal, predictive models of agent–environment interactions (Clark and Grush 1999; Gallese and Metzinger 2003; Grush 2004; Smith 1996) and successively undergo a progressive detachment process that permit them to gain an increasingly high degree of autonomy both from current stimuli and from immediate action, as described in Sect. 5. A representation provides information, but not every entity able to provide information to an epistemically active subject is a representation. Representation is a relational notion: representation of something else, informing about something else. It is in fact often pointless to act on representations for the sake of acting on them, while typically agents act on representations for the sake of acting on their object. A representation is an information structure R used for having information about another information structure O (object), with whom it is in a special systematic relationship such that it preserve some of the information actually or potentially contained in O. For example, the environment, through the interaction with it, is the main and more important source of information about the environment itself.2 As pointed out in the situated cognition literature and by externalists in philosophy (Clark and Chalmers 1998), many states or events in the environment, in a certain interaction with the brain and body of an agent, can serve as a representation. Without denying all that, we will however, argue that typically appropriate brain structures evolved for the sake of situated action serve later on as the neural substratum for representing, as we will discuss in Sect. 5. Generating and manipulating representations can be considered a demarcation criterion for the true mental life. A cognitive agent can in fact endogenously generate representations which are not totally determined by actual sensed stimuli but derived from internal models, and to use them in order to regulate its present conducts (and in some species even future ones). Many researchers such as Clark and Grush (1999) point out that even if representations
Cognitive agents have representations
2
We argue here that some organisms have brain structures whose function is to serve as representations of other entities, to be about them. Using those brain structures as representations permits many cognitive capabilities such as formulating and pursuing distal goals, which are impossible without them. Thus, representing is a function of
However, it is not—just for this—a ‘‘representation’’ of itself, unless it satisfies the usual criteria for representation. That is, the environment can be used as a representation of itself only under specific circumstances. Not only it is not ‘‘the best representation of itself’’, as claimed by Brooks (1991), since it is noisy, complex, and full of information hard to be accessed; but in many cases it is not a representation at all. Never we should mix up ‘id quod intelligitur’ (that which we know, the object the representation is about) with ‘id quo intelligitur’ (that by which we know, the vehicle) (Aquinas 1947).
123
118
have to be connected with a specific relationship with the environment in order to preserve their aboutness, the internal processes for generating and manipulating representations do not share the environmental dynamics. This feature is crucial for permitting representations to (progressively) detach from the current sensorimotor cycle and to be used instead of the environment itself. This approach do not imply that there is an ‘‘homunculus’’, a ‘‘spectator’’ of representations; on the contrary, they belong to the machinery that cognitive agents use for setting the stage for acting, as many other functional systems in the brain do. What is special about representations is that their functional role is to permit disengagement from the current sensorimotor cycle, and thus the mechanisms permitting to manipulate them follow different dynamics, whereas probably many other functional systems of the brain (in particular in the peripheries) remain more closely coupled with it. Similar action-oriented and non-encodingist approaches to representation exist. Elaborating the notion of schema in constructivism (Piaget 1954), interactivists such as Bickhard (2001) propose that representations are ways for setting up indications of further interactive potentialities; and those indications constitute representational content. The clear focus on action, and in particular on anticipation of (potential) action, will be crucial also in our discussion of how representations emerge for the need of prediction, as already suggested by many researchers (Clark 1997; Clark and Grush 1999; Hurley 2005; Smith 1996). On the advantages of having and manipulating representations The literature on situated cognition stresses the fact that many simple cognitive tasks can be performed thanks to a continuous agent-environment coupling, without any internal representational structure. However, detachment and disengagement, provided that they do not lose grounding, offer many additional advantages to cognitive systems with respect to merely adapted ones. The main reason is that representations permit substitution: at least some of the operations that the agent was performing in and on an object O are instead performed in/on its representation R, even if functionally acting on R is instrumental to ends that the agent has about O and in O, and eventually should generate acts in O. Which are the advantages of building and operating on detached representations of something instead of directly interact with it? The most immediate gain is becoming able to perform operations that are either impossible, or much more expensive, or irreversible, or risky in reality. An agent endowed with representations can explore, can make experiments, can solve problems about O, just working on R,
123
Cogn Process (2007) 8:115–131
before or without acting out on O. Working on the map or on the model is much safer and cheaper: just imagine to build bridges by trials and errors instead than on pictures and ‘small scale models’. Intelligence in strict sense (not in a trivially broad sense where just it means efficiency, adaptiveness of the behavior, like in insects) is precisely this; the capacity to build a mental representation of the problem, and to work on it (e.g. reasoning), solving the problem ‘mentally’, that is working on the internal representation, which is necessarily at least in part detached since the agent has to modify something, to simulate, to imagine something which is not already there. Perhaps, on the mental ‘map’ the agent will act just by trials and errors, but it will not do so in its external behavior. Tolman (1932) for example discusses how mental activity permits vicarious trial and error, i.e. learning as if experience had really happened. Representations are not only effective when high level cognitive abilities are required, but also in simpler activities such as navigating in noisy environments, selecting among multiple possibilities for action or affordances, coordinating actions (e.g. eye-hand), etc. Many empirical and simulative studies (Desmurget and Grafton 2000; Miall and Wolpert 1996; Pezzulo and Calvi 2006a) report the adaptive advantage of exploiting anticipatory representations instead of just relying on the sensorimotor flow. The main reason is that they permit to operate on/with O even ‘in absentia’ of it, when O is not accessible, not present as stimulus, or temporarily disappears. This permits a range of capabilities such as object permanence (Piaget 1954) or dealing with situations in which something is perceptually absent but conceptually present (e.g. ‘‘the cat who misses the tail’’). Not only, in fact, representations can serve to maintain reference to the present situation, but also to other situations, such as future ones: this entails the crucial advantage of anticipating. In this case O is not accessible because it is not yet there, it is the future; in order to have a future-oriented conduct (such as planning for future goals) the future state has to be maintained, and this can be done only with an explicit representation, since there are no available cues in the current perceptual state. Acting on representations is parsimonious, too, and permits to select and refine only the relevant information for a given use or goal instead of using all the available information. Consider for example that we can find many and quite different ‘maps’ of the same territory: an archaeological map with the archaeological sites; a botanic map, with the various plants and their areas; a geologic and mineralogy map, with the various kinds of rocks and minerals; etc. We use the map containing the relevant information about that territory, depending of our interests and goals about it (e.g. visiting monuments or finding water); evidences in literature report that we are able to shape task-specific concepts on the fly and retrieve
Cogn Process (2007) 8:115–131
accordingly the most appropriate information (Barsalou 1983). Representations permit thus to simplify the complexity of the environment: selectively store only relevant information, erase something which cannot be eliminated from O, adding something which is not there. There is a consequence of this operation of simplification that is of paramount importance for human cognition: we learn to manipulate representations in an increasingly arbitrary way, for example by combining different kinds of representations and of symbols that can not be combined in reality, or building up concepts for the non existent. Detachment has thus also lead to the development of a semiotic sphere that is remarkably autonomous with respect to the environment and its affordances. The arbitrariness of the semiotic code in fields such as arts is so high, and the capability of human cultures to create meaning and desires is so powerful that they are as important as situated activity for explaining human behavior. Of course, this is not intended to deny the situated and embodied roots of these practices, but the point is that part of this competence is unique of human cognition and comes from culture and in particular language (although it is questionable whether language use is responsible for, or a byproduct of our capabilities to deal with abstract symbols). Social life consists in a continuous process of internalization of external structures such as language and institutions, and externalization of internal, mental structures, as required for creating new artifacts (Vygotsky 1978). Another possibility offered by representation is to maintain multiple models (of the same or of different phenomena) at once. This capability, that is impossible by only being engaged with the environment, is widespread in individual and social cognition. For example, cognitive agents can produce and evaluate different alternative courses of events in planning, or maintain multiple concurrent interpretations of the same situation (see Dennett (1991)’s multiple drafts model); or take and compare one’s own and other’s perspectives on the same situation. Selecting the best perspective when resolving problems is crucial in simplifying them: for example, allocentric perspectives are better than egocentric ones in resolving labyrinths. Autonomy versus adaptivity Taken together, all these capabilities based on representations give to cognitive agents a remarkable autonomy from their environment (Castelfranchi 1995). Merely reactive, stimulus-driven agents are not autonomous at all from the environment. Cognition precisely provides agents with autonomy from stimuli, since they do not respond directly to them, but to the internal representations of them, to their interpretation and meaning, and also to their anticipation and simulation of external objects and events.
119
Usually representations come from (the interaction with) the environment and should be adapted to the environment for effectively acting in and upon it. For this reason, epistemic activity tends to make/maintain its representations close and faithful to the environmental-interactive results. But this is not the whole story, since autonomous agents can also conceive the non-existent. Representational life can become fully autonomous and one can—in some domains—live just out of representations, dream, and hallucinate. But conceiving the non-existent provides also the ability to conceive hypotheses (the future is just one of them) and possibilities, and to work internally on them, comparing and selecting the best actions, or discovering future opportunities, as in simulate planning. The most remarkable effect of this extreme form of autonomy is, however, to invert the direction of adaptivity. Building representations of how the world is not, gives the agent the astonishing possibility not to make the representation equal to the world, but vice versa to try to make the world conform to a creative, arbitrary, endogenous idea. This actually makes the agent adapt the world to itself and not just viceversa, materially changing the world through actions in order to make it matching with a non-currently-true R. The mind gets not only autonomy but priority over the world: the world has to change to match our mind, not the other way around. This is precisely the nature and the function of Goals, of our explicit motivational representations.
Symbol detachment and related problems One may wonder if those detached representations really exist in natural cognition; the traditional concept of representation has been in fact widely attacked from many perspectives. We refer to (Pezzulo G, submitted) for a critical discussion about these issues; here we instead assume that detached representations exist and analyze how and why they emerge. In fact, they are not ‘born detached’, but undergo a process in which they progressively detach from the current sensorimotor cycle. This hypothesis faces a crucial challenge, the symbol detachment problem: how and why situated agent develop representations which are detached but nevertheless remain grounded and preserve their aboutness? What we have at the beginning are information states used in the sensorimotor cycle, and what we have at the end are detached representations which can for example be used for planning or imagining distal goals. But, how can we be guaranteed that during the detachment process representations do not lose their aboutness and intentionality? In a sense, detachment and intentionality seem to be contradictory requirements, since the former indicates a degree of abstraction and arbitrariness, while the second indicates
123
120
aboutness and situatedness. There is thus the risk that, when detaching, representations lose intentionality. This problem has been recognized by many researchers. For example, Barsalou (1999) discusses how amodal symbol systems, which have lost all the characteristics of the represented entity, offer an unsatisfactory account of situated action. Harnad (1990) discusses how arbitrary symbols as traditionally used by many AI researchers, and directly encoded into computer programs, have the requisite of detachment, but suffer of the symbol grounding problem since they are not constrained by physical interaction with the environment. The proposed solution is a purely bottom-up sensory grounding that has several limitations and does not clarify how and why all kinds of representations, including conceptual ones, can have such a grounding; see Sloman and Chappell (2005). We have argued in Sect. 2 that representing is a functional role that some information states can have, assuming that they have the right requisites. How can an agent be guaranteed in exploiting information states (in the brain or in the environment) as representations? The problem is even more complex, since substitution has to be safe: which are the guarantees for using a representation for its object, a sign for its meaning? There are three main difficulties. Maintaining aboutness The first and foremost problem is that of intentionality and aboutness. While sensorimotor coupling implicitly maintains direct reference to external reality, what guarantees that a certain relationship exist between a representation and an object that is not indeed present as stimulus, such as a memory or an expectation? What guarantees that the manipulations of such a representation make sense in reality, too? Under which conditions is it possible to use representations for substituting? For example, what guarantees that if my model of a bridge is solid, the real bridge will be, too? Or, what guarantees that if I expect a prey to appear in front of me if I turn left, than a prey really appears when I turn left? How can we be sure that in those cases R is representing the right features and states of O? Maintaining relevance Not only the aboutness but also the relevance is guarantied by coupling and engagement, since in that cases the information available to an agent during its sensorimotor cycle is exactly what is relevant for its pragmatic purposes, its actions. On the contrary, building detached representations must preserve relevance of the coded information about O in relation to possible uses. Which information should be preserved in R, which aspects of O must be selected by R and coded in R? R for sure will not preserve all the information of O; this is precisely one of the main advantages of building and using
123
Cogn Process (2007) 8:115–131
representations of O instead of accessing directly the object of knowledge. Maintaining normativity Not only the intentionality of R has to bee guaranteed when detaching, but also its normativity, the possibility to match it with the represented state of O. Since representations can be true or false, it is not their actual truthfulness that has to be guaranteed, but in any case there must be a way to verify, match or compare R and O. Different criteria could exist, depending on the kind and format of representation. For example, perceptual representations (such as the belief ‘this apple is red’) could be directly matched on the basis of a common sensorimotor code, while in the case of more abstract ones (such as the expectation ‘I will get lost’ or the goal ‘I want to become famous’) this could be impossible.
Detachment originates from anticipation We will now discuss how detached representations can emerge thanks to anticipatory capabilities firstly developed for the sake of action control; and how this account resolves the symbol detachment problem and related ones. Firstly, we offer our argument at a glance. In little organisms such as insects the distance between sensors and actuators is negligible, too; they can exploit the sensorimotor flow as such in order to adapt to their environments. Maintaining adaptivity while growing, however, is very difficult for an organism that only has sensors and actuators, because the sensorimotor flux has delays. Evolution thus discovered that a good solution is creating internal, emulative loops. Representations, and in particular anticipatory representations, thus emerged in many organisms for two related reasons: correction (with respect to errors and delays) and selection (filtering only relevant info and parsimony). Both of them were initially relate to the here-and-now, to the control of action. Once in place, anticipatory representations were however exapted for increasingly complex cognitive functions, which emerged at different stages. Parallel to detachment of representation (the possibility to represent what is not here-and-now) there is disengagement of action (the possibility to act not for the here-and-now), that is mediated by normative anticipatory representations, goals, that progressively detach from drives. We will thus argue that many human conducts depend on anticipatory mechanisms; that representations and goals emerge and detach thanks to those mechanisms, and for this reason they retain an anticipatory nature. These claims have recently received a good support in empirical and theoretical studies, indicating that an evolutionary process, based on anticipation, has bootstrapped many facets of high level cognition; in a slogan, thinking is an exaptation of anticipating.
Cogn Process (2007) 8:115–131
The anticipatory perspective on cognition and representation Recently a rather coherent action-based understanding of the nature of cognition (and representation) has emerged both in the empirical and theoretical literature. Many converging evidences indicate that neural substrates involved in performing, observing, simulating and imitating actions in fact largely overlaps, and representations are intimately related to the sensorimotor apparatus and have a crucial anticipatory nature. The most striking evidence is the discovery of mirror neurons in the ventral premotor cortex of monkeys, and successively in other brain areas of monkeys and even in humans; see (Rizzolatti et al. 2001) for a review. These neurons fire both when the monkey grasps or manipulates an object, and when the monkey observes another individual performing a similar goal-directed action. Recent findings (Fogassi et al. 2005) also indicate that some mirror neurons do not only code the observed motor act but also indicate an anticipation of the whole distal goal, i.e. understanding of the agent’s intentions. These findings indicate a crucial role for anticipatory, simulative mechanisms, and an action-oriented nature of representations, as indicated by the simulative view of Gallese (2000): ‘‘Looking at objects means to unconsciously ‘simulate’ a potential action. In other words, the object-representation is transiently integrated with the action-simulation (the ongoing simulation of the potential action)’’. In a similar vein, Damasio (1994) provides evidences for dispositional representations that are schemas of potential neuronal activity endogenously eliciting/reconstructing images of appropriate content and emotional states such as somatic markers. At the same time, Kawato (1999) proposes that the brain uses internal models, which mimic the behavior of external processes, for all motor control of action. In particular, forward models permit to generate expectations about the next sensed stimuli, given the actual state and motor command. Inverse models instead take as input actual stimuli and the goal state and provide as output the motor commands necessary to reach the desired state. Taken together, inverse and forward models permit not only to perform motor plans but also to regulate behavior despite noise and dynamicity in the environment. In neurosciences forward models have been claimed to be involved in compensating for delays in sensori feedback, cancel the self-produced part of the input from sensori stimuli, etc. (Wolpert et al. 1995; Wolpert and Kawato 1998) and empirical evidences exist for their involvement in visuomotor tasks (Mehta and Schaal 2002). Jeannerod (2001) proposed a motor-based understanding of cognition based on simulation, and anticipatory structures have also been claimed to be involved in visual attention (Balkenius and Hulth 1999) and imagery (Jeannerod 1994, 2001; Kosslyn
121
and Sussman 1994). Psychologists such as Hoffmann (2003); Kunde et al. (2004) show that many human conducts which seem to be automatic involve instead anticipatory, ideomotor representations, and reward prediction has been indicated as a key mechanism for learning and metalearning (Doya 2002). Internal models for simulating actual sensorimotor engagement can also be exploited for increasingly complex future-oriented activities, in the individual and social spheres, that are widespread in natural cognition, such as action control, attention, and planning; see (Pezzulo G, submitted) for a review. For example, one of the distinctive features of high level cognition is the possibility to test potential actions ‘by simulation’ and thus avoid dangers (Damasio 1994). Many evidences suggest in fact that imagined and performed actions share a common timing and neural substratum (Decety 1996). Another related capability fitting this framework is formulating and comparing in simulation multiple alternative courses of actions. A possible neural substratum is a ‘loop’ (Middleton and Strick 2000) between the cerebellum, which produces sensory predictions, and the basal ganglia, which are involved in action selection and movement initiation. In the social sphere, a role of mirror neurons and simulative processes has been claimed for imitation (Iacoboni 2003; Rizzolatti et al. 2001), distinguishing self from others (Decety and Chaminade 2003), mind reading (Gallese and Goldman 1998), communication (Gardenfors and Orvath 2005), language production and understanding (Rizzolatti and Arbib 1998), and understanding other’s actions on the basis of their movements (Blakemore and Decety 2001). The role of the mirror system as a common substrate for the individual and social spaces is also suggested by Gallese (2001), who proposed the shared manifold hypothesis for explaining empathy; and by Hurley (2005) who developed the shared circuits hypothesis as a unified framework relating control, imitation, and simulation. Anticipation and representation Recent motor-based or simulative theories of cognition, historically originating from the productive perspective of Kant (1998), claim that cognition does not consist in passively receipting the structure of the world; on the contrary, many cognitive capabilities are based on productive mechanisms that allow to generate anticipations of the effects of actual and potential conducts, thus selecting and regulating behavior. Even the capability to represent depends on such anticipatory mechanisms. Bickhard (2001), Clark and Grush (1999), Smith (1996) propose that representations arise from the need for prediction, and stay intimately coupled with it: they consist exactly in the anticipation and understanding of potential interactions with the environment. The fact that it is possible to operate directly on
123
122
representations (e.g. simulate and select among multiple possible potential actions) derives from the fact that they originally serve for situated action control and maintain their action-oriented nature and coding. Anticipation is thus at the core of cognition: concepts are essentially predictive models of the phenomena, and this permits to ground representations of non-yet-experienced phenomena. According to Grush (2004)’s emulation theory, representation is the ability to emulate internally part of external reality by means of internal models such as Kalman filters, which can also ‘nest’ and realize abstraction. Similarly, Hesslow (2002)’s simulation hypothesis maintains that representing is engaging in simulated interaction with the environment by means of internal predictive models which can be chained and form ‘loops’. Barsalou (1999) proposes the perceptual symbol system theory; arguing against amodal and disembodied notions of representations, he champions a situated view in which they retain part of their original sensorimotor structure. Building on perceptual systems, Barsalou proposes that concepts emerge as productive, simulative structures that can be used by the agent in order to simulate actual or expected sensorimotor engagements on the basis of past situated action, producing understanding of perceptual and abstract concepts. Smith (1996) argues that thanks to prediction an intentional dance between subject and object arises. The ‘subject’ makes predictions on the basis of a partial internalization of external reality. But since the world has to be predictable to permit that, the ‘object’ can only be conceived as structured in a way that permits predictability; thus the ‘dance’ produces the ‘phenomenon’ in kantian terms.3 Two kinds of anticipatory representations Many human conducts are regulated by anticipatory mechanisms, and this explains why representations arise, why they have an eminently anticipatory nature, and why they remain related to action control. Here we define two main kinds of anticipatory representations, expectations and goals, that have different roles in regulating conduct. Expectation An expectation is a representation of a possible future (in the brain or in an artificial system), for example in the form of an expected sensori stimulus. It is not simply a prediction, since it is about something the agent is currently concerned with. An expectation is produced endogenously by predictive mechanisms such as internal 3
Our emphasis on simulative theories is not intended to deny the role of other dynamics such as social ones: we learn many concepts and practices via social interaction and coordinated actions over a shared physical world (Vygotsky 1978). However, simulation can be crucially involved in those dynamics, too, as indicated by the empirical literature already reviewed.
123
Cogn Process (2007) 8:115–131
forward models, or even by more sophisticated cognitive processes such as inferences. There can be expectations at different level of abstraction, such as the next perceptual stimulus, a feeling, but even a belief about the future of the world; they can also be about the past, for example in counterfactuals. Expectations have mainly a regulatory role in several cognitive functions, ranging from action control, to decision making and planning; and can trigger emotions such as fear, and be part of complex emotions such as regret. Goal A goal is an anticipatory representation that has also motivational and prescriptive roles: to constrain, monitor and regulate behavior toward the realization of some specific state of affairs. In a cognitive agent, an endogenously generated goal, and not only a stimulus (as in stimulus-response systems), can be the trigger of action. A clear prototype of really goal-directed, purposive or intentional action is the TOTE model of Rosenblueth et al. (1943). Here the system starts from the representation of the goal (how the world should be) and compares it against the representation of the current state of the world (perception). If the goal matches with no need for action, this serves to stop behavioral activation and satisfy the agent; in case of mismatch there is instead activation and search for the appropriate action whose memorized/expected result overlaps with the goal. However, a goal is a goal not only when we are pursuing it, but it has important roles even after having been realized, or when it can not be realized, or when it is not (yet) selected. In fact, independently on any actual action, a goal serves to evaluate states of the world with respect to the compliance with the goal content (Castelfranchi 2005; Castelfranchi and Paglieri 2007) (is the world as I want it to be? is the goal realized?). Goals also afford on-line and off-line action selection: they have, in fact, a ‘‘value’’, an ‘‘utility’’ that can be evaluated before any action starts, and thus can serve for selecting among alternative future-oriented conducts.4 4 Due to its multiple functions, the term goal has been ambiguosly used for explaining different phenomena, that can also imply different coding: (1) guiding and regulating actual action, and (2) conceiving possible outcomes outside any actual immediate action. This point is clearly illustrated by Gallese and Metzinger (2003), who firstly discuss about the role of goals in the action: ‘‘In the monkey brain microcosm so far explored, the goal of grasping an object is still almost completely overlapping with the action-control strategies. Action control actually equates to the definition of the action goal: the goal is represented as a goal-state, namely, as a successfully terminated action pattern’’ and thereafter introduce a notion of goals which is not engaged in here-and-now action: ‘‘However, the presence in the monkey brain of neurons coding the goal of grasping, regardless of which effector (hand or mouth) will eventually achieve it, in our opinion provides evolutionary evidence of the dawning of a more complex—and abstract— coding of intention and volition.’’ We will discuss in Sect. 5 that these phenomena occur at different stages of the detachment process.
Cogn Process (2007) 8:115–131
Anticipation and the symbol detachment problem The anticipatory view on representation suggests a suitable way to understand the process of detachment: representations detach from current sensorimotor interaction in order to anticipate it, and successively gains more and more autonomy from it, also providing many additional advantages to cognitive agents. When we interact with an object, in fact, we also learn to anticipate the results of our actions. This is obtained by splitting the sensorimotor cycle in two parts: the former is coupled (the control loop), the latter is decoupled and consists in producing expectations by means, for example, of internal forward models (Kawato 1999). In origin, the decoupled path has only the function to replace or integrate missing or unreliable stimuli. However, it already serves a representing function in the inner loop, since it stands-in for something other and can deal with the temporary ‘‘absentia’’ of its reference; and its groundedness is guaranteed by sensorimotor engagement. Later on the decoupled part is exapted for dealing with objects when their absentia is prolonged, for example by repeatedly replacing actual stimuli with expected ones; and for dealing with class of objects that could be predicted and simulated with the same machinery, giving rise to categories; see (Barsalou 1999). However, the most important innovation is that cognitive agents can now substitute, and use detached representations for acting on and about reality. In fact, these representations preserve a functional relation with their objects that permits to address the main difficulties of symbol detachment. Maintaining aboutness Anticipation is a suitable solution, for two reasons. Firstly, representations originate from the decoupled part of anticipatory mechanisms, and retain their reference to the object they permit to interact with, being perceptual or more conceptual. Secondly, simulative and anticipatory mechanisms permit to extend the range of grounded representations from actual interaction to possible (simulate) interaction and even counterfactuals. In this spirit, Roy (2005) develops a concept of grounding which depends on two mechanisms relating agent and environment: causation (from environment to agent) and anticipation (from agent to environment). According to this idea, concepts for objects which are e.g. reachable or graspable are grounded by schemas which regulate actual behavior and at the same time encode predictions on the consequences of an expected interaction. It is important to remark how what is required is not actual engagement, but possible engagement, much in the spirit of the interactivist (Bickhard 2001) idea that representation is about future potentialities of interaction, and emerges in the presuppositions of anticipatory interactive
123
processes. Computational systems have been developed which illustrate these ideas, too. On the basis of the theory of perceptual symbol systems (Barsalou 1999), in Pezzulo and Calvi (2006b) interaction schemas are evolved that ground the concepts of predator and prey in actual and possible interaction with those entities. Hoffmann (2007) uses internal simulation of possible trajectories for grounding concepts related to navigation; for example, distance from obstacles is grounded and estimated by running simulations until they encounter the obstacle. Dead-ends are recognized through simulated obstacle avoidance, while passages are grounded in successfully terminated simulations of navigation. The robot Ripley (Roy 2005) can build up representations of objects based on their sensorimotor structures and maintain an internal, simulative model of its environment that permits to work internally on its representations before acting on the objects (e.g. picking up them). It can also operate on the basis of verbal instructions which are understood in sensorimotor terms. Maintaining relevance During sensorimotor engagement the agent learns to extract from the environment only meaningful information. Since representing means engaging in simulated interaction with its environment, the representing agent uses the same mechanisms it uses for predicting and controlling actual action, and can only operate if those mechanisms preserve the relevant part of the data they need (such as those used for predicting); anticipation thus provides a criterion for selecting only relevant information. For a representing agent attention is selection for action (Balkenius and Hulth 1999); it attends only information that is expected to be relevant with respect to its actual and potential goals. While the agent can have multiple mechanisms for predicting (such as schemas with forward models (Wolpert and Kawato 1998)), the most active and successful ones, operating in reality or in simulation, are those that are actually serving its goals. Since they are also responsible for selecting stimuli, the agent’s epistemic activity is canalized toward relevant information, either in the environment or in memory. For example, in the schema-based computational system described in (Pezzulo and Calvi 2006a) multiple competing schemas exist for realizing several goals, but only information relevant to the currently active ones is gathered: the schemas not only guide pragmatic action, but also orient epistemic activity, guiding the sensors toward stimuli that are expected to be useful for the active schemas. Maintaining normativity If representations are anticipatory and serve to predict the effects of actual and potential interaction, normativity can be guaranteed with a pragmatic criterion, since truthfulness of the representations is
123
124
implied by success in actions. In fact, when an agent acts according to its expectations, it implicitly assumes a lot of things about the world. If a pattern of actions which is activated for achieving an expected result actually achieves it, this is the guarantee that the expectations and assumptions were likely to be true. If I take my cup, without looking at it, and I succeed in drinking, this confirms the implicit idea that it was not empty presupposed by my act. On the contrary, failure in expectation-driven actions means also a failure in the criteria for truthfulness of the involved representations. Of course, the pragmatic criterion may fail for several reasons; however, it is wort noting that since we have not a direct ‘epistemic access’ to the world, the only way to establish normativity is via pragmatic actions that also have epistemic implications. Interactivism (Bickhard 2001) suggests a similar perspective: if an active interaction fails, than the ‘‘indication for action’’ (and the content of the representation) is false. This pragmatic criterion has the important property that the system can learn to exploit these failures. For example, it has been exploited in schema-based robotic systems for assigning different responsibilities to active schemas (Pezzulo and Calvi 2006a; Wolpert et al. 1995; Wolpert and Kawato 1998). By introducing several competing combined forward-inverse models, competing motor plans can be generated and maintained for the same or for different targets, and choice depends on predictive accuracy. Models predicting better are selected; the rationale is that they are well attuned with the current context, and their representational content is true. As an example, Wolpert et al. (1995); Wolpert and Kawato (1998) use competing models as alternative motor plans entailing alternative hypotheses about the context (full glass and empty glass); prediction of the right sensorimotor flow is thus used for an evaluation of the hypotheses and the selection of the best modules. A similar schema-based system is exploited by Pezzulo and Calvi (2006a) for realizing agents having multiple goals; in this case success in prediction is not only used for selecting among possible actions for realizing a goal, but also as one of the criteria for selecting among goals (the other one being its value). If an appropriate perceptual schema for tracking preys succeeds in its prediction, this is assumed by the system as an information that the prey is actually there (or is expected to be there). The active schema can thus be exploited as a representation of the prey, for example for activating and guiding motor schemas for following or catching it. The problem of maintaining normativity, however, becomes more complex, when a direct perceptual matching is not possible. For example, the goal ‘eating a pizza’ could be satisfied in multiple ways, including feeling the specific taste of pizza, or by the fact that ‘pizza’ is written on the
123
Cogn Process (2007) 8:115–131
menu we choose (if we do not know the taste of pizza). However, even in the latter case the pragmatic criterion can be in play, with more complex schemas. In an hypothetical eat-a-pizza schema, there could be no match with a perceptual representation of the pizza to be eat, but the criterion could be the compliance of a set of operations which involve a chain of expectations, finally leading to the success of the whole schema. Some expectations can be formulated at a quite abstract level, and their representational content can be another running process, i.e. another simulation. This view is also proposed by Grush (2004): simulations can nest to produce increasingly abstract levels of description, in which the criteria for ‘matching’ are increasingly distant from perceptual matching, although they remain grounded on sensorimotor interaction, or emulation of interaction. In a schema-based perspective, the process of assigning success of a goal, or matching to an expectation, can be described as schemas that learn to formulate expectations about the compliance of other active schemas. Also Barsalou (1999) discusses the understanding of abstract concepts such as ‘‘true’’ in the internal, simulative dynamics and in the capability to have processes that observe and evaluate other processes. Expectations can also have the form of abstract and even conventional codes and symbols, that nevertheless must have specific ways and rules to be matched with the sensory information coming from the world; see (Castelfranchi and Paglieri 2007). We can thus conclude that it is possible to perform pragmatic actions with an additional epistemic result, that is acting in the world and, as an additional result, have also information about the world. Matching representations and reality is so crucial that cognitive agents have also developed a set of pragmatic actions (such as controlling, looking for, checking whether) that are operated not for themselves, but in order to achieve an epistemic result—in that case the pragmatic act is only the vehicle of the epistemic one.
An evolutionary pathway toward cognitive systems A hypothesis on the evolution of cognition is presented here, involving successive stages: from exploiting sensorimotor contingencies, to forming action-expectation relations and successively ‘reverse’ them, activating actions when their expected consequences are desired, to the acquisition of the capability to autonomously generate goals. We are however not suggesting that cognitive capabilities at a later stage replace those at the earlier ones. Although there are transitions from one form of behaviororganization to the other all of them can co-exist in natural cognitive systems, and complex behavioral organizations
Cogn Process (2007) 8:115–131
can arise combining them in just one and the same structure.5 Stage 1: Information states Adapted system are engaged in a closed-loop with the environment. They do not exploit representations but only use information from their sensors (although in some cases such information can be re-produced internally by a reverberant neural architecture which permits to compensate for time delays due to the fact that the distance between sensors and actuators increases during growth). In this case only reactive,stimulus fi action (S--A) behavior is possible, although it can be considered implicitly anticipatory in the sense that A is selected for responding not to S itself, but to a situation which is successive to S. We call this form of anticipation implicit or behavioral: an anticipatory conduct without an anticipatory representation. The point is that the origin of those mechanisms is functionally anticipatory: we can define it an implicitly preparatory behavior since the agent learns to use S to prepare to a successive event and responds accordingly. A plausible example is the apparently goal-oriented behavior of an animal reacting to a noise by escaping. This behavior has not been selected by evolution for reacting to the noise, but for reacting to a future danger, whose sign is in this case a noise. It is not necessary that the escaping animal has an anticipatory representation of the danger, such as a predator, but only that it has evolved the capability to coordinate directly with present stimuli which are signs of future states of affairs. According to the theory of active vision (Gibson 1966), sensing implies an implicitly anticipatory mechanism based on the exploitation of learned patterns of sensorimotor transformations, and the environment is used as the best representation of itself (Brooks 1991). A similar account of 5
This fact makes harder empirical investigations of these phenomena. Another source of difficulty is the fact that some cognitive functionalities do not simply emerge at a certain stage, but change over time, and this somewhat erroneously suggest that the same mechanism could be in play. This is for example the case of the development of objects permanence in children, that becomes increasingly complex at different stages of detachment and is performed using different forms of anticipation, first implicit and later on explicit. As reported by Piaget (1954), initially children have no object permanence. Successively, when they develop implicit anticipatory capabilities, object permanence is realized procedurally, by a continuous coupling between the mechanism and the object to track. At a later stage, only after having acquired a detached representation of the object to track, it is possible acquire episodic and not only categorical knowledge, i.e. to express: ‘take that ball’ and not only ‘take a ball’. The object is now affordable for abstract thinking: it is possible to refer to a never perceived or to a non existent ball, to formulate beliefs about the ball, and to take other’s perspective about the ball.
125
how to deal with the future without any anticipatory representation was provided by O’Regan and Noe (2001). According to their sensorimotor view, perception is coordinating with the dynamical structure of sensori stimuli, which is acquired by learning structure in stimulation from the environment. This activity is clearly anticipatory in the sense that the sensory apparatus learns to attend to relevant stimuli by anticipating e.g. the movement of an object or the patterns of transformation of sensory stimuli under a certain motor operation. However, there is no explicit representation of stimuli. For example, in tracking a target the agent learns to foveate beyond the target in anticipation of its movement, but it has no anticipatory representation of the next visual stimulus. Implicit forms of anticipation are thus very powerful mechanisms for engaging a successful dynamical interaction with the environment. However, we argue that cognitive agents have also developed more explicit, representations-based forms of anticipation, which permit to disengage from the current sensorimotor cycle. Stage 2: Anticipation for action control The evolutionary and theoretical ‘bridge’ between adapted and cognitive agents is represented by the evolution of anticipatory mechanisms from implicit to explicit forms. That is, agent pass from the rehearsal of incoming stimuli via a reverberant neural architecture, and from mechanisms of sensorimotor coordination to the internalization of part of the environment with the generation of internal models permitting to anticipate the effects of its actions. Anticipatory mechanisms, such as forward models producing perceptual expectations, permit thus to an adapted agent to run an ‘inner information processing loop’ which is disengaged from the sensorimotor loop. In this initial stage there is no real detachment, since the ‘inner information processing loop’ proceeds in synchrony with the sensorimotor loop, and this guarantees that it can be used on-line for many purposes. Figure 1 shows the structure of a reactive agent (left) and of an agent having also an inner loop (right). The evolution of anticipatory structures such as an inner loop is due to the evolutive advantage of some functionalities which are not prima facie related to representation, but only to the control of action and learning. Two converging evolutive pressures are filtering and replacement of stimuli. In fact, expectations can be used both for filtering noise and for replacing missing or unreliable stimuli (Kawato 1999; Wolpert et al. 1995; Wolpert and Kawato 1998). Some other important functionalities, such as compensation of time delays, could be a consequence of both filtering and replacement. According to Hoffmann (2003), action-induced anticipations are needed for stabilizing perception;
123
126
Fig. 1 Left an agent engaged in a sensorimotor loop with the environment. Right an agent having also an inner loop which parallels the sensorimotor loop
this is the reafference principle of Holst and Mittelstaedt (1950) which is recently proposed by Grush (2004). Moreover, if a stimulus is missing or unreliable, under certain conditions it can be replaced or integrated by the expected stimulus, creating internal simulative loops, as happens in Kalman Filters. Anticipatory mechanisms can also be used for stabilizing feedback. For example in Smith predictors a forward model is used both for mimicking the target plant and for canceling the predictable part of the feedback; in this way only the unpredictable part of the feedback is used for correcting errors within the feedback loop (Miall and Wolpert 1996). Cancellation of self-generated stimuli is another relevant advantage of anticipation (Desmurget and Grafton 2000). For example, an agent which is able to track other agents can avoid tracking its own arms moving in its visual field by erasing the part of the incoming stimuli which is predictable. All these evolutionary pressures could thus have lead to the emergence of mechanisms, such as forward models, producingaction fi expectation (A--E) pairs (expectations about the consequences of own actions). The same, or other mechanisms could be in play for learningstimulus fi expectation (S--E) pairs; in this case stimuli can not only depend on self produced actions, but can be regularities in the environment, and for this reason are probably more complex to learn. Once the cognitive system is able to endogenously generate expectations, several other functions can emerge. The most important one is monitoring of action. In fact, by comparing actual with anticipated stimuli, an agent can regulate its behavior ‘from the future’. According to Adams (1971), anticipations can be used as a reference signal for the control of voluntary acts: ‘‘Beginning the movement brings an anticipatory arousal of the (perceptual) trace, and the feedback from the ongoing movement is compared with it’’. Another relevant function can be called explicit preparatory behavior. Expectations, in fact, permit to react to present signs of an anticipated future, or ‘reacting to the future’: a stimulus is interpreted as a sign of a successive (distal) event and the agent reacts to that event. Explicit preparatory behavior is slightly more complex than implicit preparatory behavior. For example, it permits to react to a
123
Cogn Process (2007) 8:115–131
future danger in a more selective way, depending on the nature of the perceptual anticipation. Obviously, the more long-termed and accurate the prediction, the more is possible for an agent to react far from the event. The influence of expected stimuli for orienting attention has been reported in humans (Balkenius and Hulth 1999), monkeys (Colombo and Graziano 1994) and pigeons (Roitblat 1980). Another related functionality, selection of context, has been demonstrated in motor control (Wolpert et al. 1995; Wolpert and Kawato 1998): matching of expectations produced by concurrent schemas provide an indication of their appropriateness. Lastly, anticipating is crucial for the development of a body schema: understanding the boundaries of own prediction and control permit to discriminate self from others and from the environment (Piaget 1954). In this phase, even if expectations are used for action control, actions are still selected in a reactive way, since the trigger is the stimulus. This functional mechanism,stimulus fi action fi expectation (S--A--E) (also called anticipatory classifier (Butz and Hoffmann 2002)), in which the stimulus plays the role of trigger and the expectation is used for action monitoring and control, is a crucial evolutionary step and could account for several evidences in reinforcement learning. It is goal-oriented in the teleological but not the intentional sense. Since it includes an explicitly formulated expectation, it could be mistakenly assumed to be intentional, but we only consider it a forerunner of true goal-directed, intentional action: a proto-intention. In fact, its expectation can not by yet considered the true ‘goal’ of the action, since does not activate, select, drive and control the action. In order to do so, the mechanism has (1) to undergo a functional inversion permitting to activate from right to left, from the expected result to the selected action; (2) to fire the action not directly from the activated ‘goal’ but on the basis of the mismatch between the goal and the current state of the world. Stage 3: Goal directed behavior Learned action-expectation pairs are a bridge from merely anticipatory behavior (in the previous sense) and goal directed behavior, in which expectations are used for the selection of action. Hommel et al. (2001) describes the emergence of action control as a two-stage process, presupposing the formation of bidirectional action-effect relations: ‘‘At Stage 1, the motor pattern producing a particular effect is automatically integrated with the cognitive codes representing this effect. At Stage 2, the motor pattern is intentionally executed by activating the cognitive codes that represent its expected effect’’. According to Hoffmann (2003), anticipations precede voluntary acts for
Cogn Process (2007) 8:115–131
their selection: this is the ideomotor principle (James 1890). By inverting the action-effect relation, expectations can be used as goals states, and the goal (not the stimulus) is the trigger of action: goals differ from expectations for their motivational side. This mechanisms in brief consists in re-enacting the same mnestic trace which constitutes the representation of the goal state, that is integrated with the motor program leading to it: a representation of the objective precedes the planning and the motor program for realizing it. This mechanism also explains the common coding between goals and other representations (Hommel et al. 2001), and why action patterns are stored with the final goal state. After the inversion, the functional mechanism can thus have the form:expectation fi action. The Formation of goals It remains to be explained how and why the inversion occurs; not all expectations leads in fact to such an inversion. Arguably, inversion is due to an evaluation process that assigns a value to the expected state, that can be for example rewarding for the system; there are in fact evidences that reward prediction modulates learning (Doya 2002). This also explains why goals retain a value and can serve for evaluating the world, even when no action is triggered. Evaluation of an expectation can depend for example on drives, or on anticipated drives. In the somatic marker hypothesis (Damasio 1994) expected results that are marked negatively stop actions. We are suggesting also that if expectations are produced that are evaluated (or expected to be evaluated) positively, they can serve as goal states and trigger behavior. Moreover, (long term) expectations which are evaluated by expected drives can be stored in memory, to be successively retrieved and serve as goal states. These states in fact have the main features of goals: (1) under certain conditions they can be expected, and this means that they are achievable by the system and can thus be used for selecting and initiating an action; (2) under certain conditions they can be desired, because they are already positively evaluated. In this case we could say that the system has new terminal goals (that do no depend on other goal or on drives). It is also possible that states which were never sensed but only expected can be ‘vicariously’ evaluated by a similar mechanism and stored as goal states, permitting to generate truly novel goals and to free the system from homeostasis.6
6
We are not supposing here, however, that all goals have an hedonistic nature, since the valence they carry on can acquire a sufficient degree of autonomy from immediate drives (e.g. the value of money), and they can have different origins. Many goals and goal-oriented conducts have a social origin and do not depend immediately from drives; suitable mechanisms can be for example imitation and internalization of cultural practices (Vygotsky 1978).
127
Stage 4: Anticipatory off-line processing The next stage occurs when representations provided by forward models begin to detach from the pseudo-closed loop, and they do not parallel any more the sensorimotor loop. Detached anticipatory representations, exapted from the decoupled part of the inner loop, can be used in increasingly sophisticated ways, as already reviewed in Sect. 2. An example is using the forward model just to simulate possible engagement and without producing motor outputs; the aim of this operation is for instance off-line planning. It is worth noticing that in this case the temporal coupling with the phenomenon is broken; again, this could be due to the evolutive pressure of anticipation, since typically off-line processing occurs before encountering a real phenomenon to deal with. At this state representations begin to be manipulated by dynamics that are different from those of the environment. The temporal relationships in R and O, in fact, become different; for example, simulations can run quicker than the simulated interaction. What is equally important is that representations that detach from current sensorimotor engagement have the potential to become more and more abstract. In the forward models, in fact, the description level is not all phenomenic: unperceived constraints or regularities can be in fact learned, for example in order to simplify the data. Abstraction can arise from a technical reason: since long term predictions become less and less accurate, it is possible that coarse-grained, schematic predictions are instead generated to be evaluated, leading to the construction of types. If we were perfect predictors, probably we would not need to generalize and to build up types such as corridor or door, but we could directly operate on objects and tokens. However, anticipatory representations having whichever grain, even of detached by the sensorimotor loop, are still grounded, since they are produced by a mechanism whose origin is paralleling the sensorimotor loop. Those novel, conceptual entities can be used by the agent as representations, actually reasoning at a different level of abstraction. A computational architecture inspired by Piaget (1954) is for example implemented by Drescher (1991) that interactively enlarges its ontology by learning new synthetic items which are conceived as the common cause of a set of related interactions. In a related way, Grush (2004) discusses how simulations can be ‘nested’ and produce increasingly abstract representations. In this case, not only temporal relationships, but even other forms of coupling are broken, since those representations are typically more schematic than real phenomena and only include relevant features; see (Barsalou 2003) for a discussion. Thanks to anticipatory mechanisms functioning offline novel cognitive functions are possible, such as simulative
123
128
planning. According to the simulation hypothesis, (Hesslow 2002), long term simulations are executed and used for action selection. Multiple candidate scenarios can be produced, compared and evaluated and a goal is selected. Planning can thus be realized just by running the forward models (by inhibiting the coupled part, thus without sending any motor command to the actuators) and producing simulated scenarios to choose among: e.g. different ways to grasp an object, or different paths to a goal state. This planning mechanism uses the same sensorimotor schemas to be used while interacting with the objects, just in absence of the objects rather than internal copies of the objects, as topographical maps are. The conceptual difference is in the fact that the (missing) sensor stimuli are not retrieved in maps but produced on-line by the forward models, i.e. the mechanism is inherently anticipatory. However, the most distinctive feature of detached representation is that an agent can manipulate them instead of manipulating the environment: they can substitute. This process could depend on the storing of ‘chunks’ of representations, e.g. previously used for planning or reasoning, that becomes available as declarative knowledge to reason with. The detachment process, at least in human cognition, undergoes successive steps, leading to increasingly complex structure of beliefs, evidences and supports, out of simple mnestic traces activated by association. By off-line reasoning on detached representations the cognitive agent can thus formulate more abstract plans, referring to possible futures, and also postpone some decisions when future opportunities are met. It remains to be understood how much of the sensorimotor nature is preserved in detached representations. Barsalou (1999) reports evidences that even in declarative knowledge some sensorimotor aspects are retained, suggesting a partial re-enactment of the mechanisms related to situated action; but it is still unclear to which extent representations can detach and how much of their originary relation to motor action they can lose. An intriguing hypothesis is suggested by the fact that in order to generate representations to be used off-line the motor commands have to be inhibited. Representations which are used systematically off-line, such as abstract ones, could be encoded in dedicated brain circuits which have lost their connectivity with motor ones thanks to the systematical inhibition.7 Aboutness and normativity of such detached representations, however, is guaranteed by their evolu7
Several theories have been recently proposed that situate representations and conceptual knowledge in different brain areas. The most extreme ‘embodied cognition’ position is taken by Gallese and Lakoff (2005), who argue that integrative processes take place into the sensorimotor system, thus denying any role for higher association areas. This position is disputed by Mahon and Caramazza (2005) on the basis of neurobiological findings that are difficult to explain on the basis of a purely motor theory of action.
123
Cogn Process (2007) 8:115–131
tionary pathway, even when they acquire a very high degree of autonomy from the current sensorimotor engagement. The detachment of goals Goals representations, as any other representation, undergo a process of detachment from sensorimotor interaction; at the same time, goals undergo a parallel process of detachment from drives in which they partially preserve their characteristics related to values and motivations. As already discussed, the ideomotor control structureexpectation fi action permits to initiate actions from their expected consequences, and to monitor behavior. But progressively goals assume other roles, as exemplified by the TOTE model (Rosenblueth et al. 1943) of goal-directed, intentional action, that has the form:goal (+ mismatch) fi action + expectation; see also (Pezzulo et al. 2006) for a discussion of TOTE and the ideomotor principle. Autonomy of goals offers in fact two main functions: disengaged, in particular future-oriented action, and evaluation that is independent of any actual action. Not only, in fact, they serve as reference signals for immediate action, but also can prescribe and evaluate future actions, serving as reference signals for opportunities to occur in the future. Hoffmann (2003) reports evidences for start anticipations: anticipation of appropriate preconditions for execution, that trigger actions when the right opportunities in the environment exist. A progressive functional transformation is therefore necessary to fully autonomize the goal representation and to use it for evaluating states of the world independently of any action. In the more complex cases, detached goals do not fire actions (immediate or future), but serve only to be compared with the world or with other ones: this is the case of deliberating by reasoning on their values, that thus have to be accessed somewhat autonomously from the behavioral control system. Before acting, thus, a cognitive agent evaluates the world, in order to see if it fits its desires. Moreover, after acting it evaluates the success of its action and can also attribute the success to itself. In both cases, these processes might, however, imply a partial re-enactment of sensorimotor structures. It has been argued, in fact, that anticipation of the consequences of one’s own actions is deeply involved in the sense of agency, and that deficits in the brain circuits permitting to predict lead to failure in selfattribution of actions (Frith et al. 2000). Arbitrary symbols: another stage? There could be another path for the emergence of detached representations, perhaps only available to humans. Some representations, instead of becoming detached, could be ‘born detached’ for an autonomous use; this could be the case of arbitrary (or semi-arbitrary) symbols such as those used in verbal communication or other advanced forms of semiosis, that
Cogn Process (2007) 8:115–131
have a remarkable autonomy and for which it is difficult to trace an origin back to situated cognition. What is remarkable in human cognition is that it is possible to reason both by using simulated interaction (as we have argued) and by exploiting directly a symbolic code, for example ‘thinking with words’. It is possible both to compose music by hearing (or rehearsing) auditory information, and to compose music by only using the symbolic music notation, which is a new code having its own rules (we could call the first way ‘Mozartian’ and the latter ‘Bachian’). Although the latter ability is clearly successive, two questions remain open: whether or not it is a new ability, and in the case of a positive answer how it can be produced. Assuming now that it is a new ability, in order to maintain the same naturalistic framework, we propose two (not mutually exclusive) directions of research. The former follows the above formulated hypothesis that representations can be encoded in dedicated brain circuits which have lost connectivity with motor ones. Using those representations could not imply (or imply only in minimal part) the re-enactment of sensorimotor circuits, and this also explains the fact that the declarative code is not ‘directly executable’, and that deliberated actions are not immediately put in play. The latter focuses on the similarities between situated activity and social practices such as cultural learning. Evidences for a common neural substrate for understanding the self and the others are reported in literature (Iacoboni 2003; Rizzolatti and Arbib 1998; Rizzolatti et al. 2001), and it has also been argued that a ‘we’ space could be precedent to the self (Gallese 2001; Hurley 2005). Perhaps even most importantly, several authors such as Vygotsky (1978) stress the crucial, unique role of language and linguistic practices for the formation of arbitrary symbols, observing that communication, arts and other semiotic activities are themselves forms of life, and humans can learn to make sense of them by learning, individually or culturally, to form predictive schemas of these practices, just as it happens in more standard situated activity. Exploiting such culturally and linguistically mediated schemas could be thus very similar to exploiting schemas learned by engaging in interaction with the environment, and still provide much more sophisticated cognitive abilities. As an example of a naturalistic attempt to understand language and other practices in the conceptual framework of situated action, Barsalou (1999) investigates how arbitrary linguistic symbols become attached to already grounded concepts. Another direction of research is indicated by cognitive linguists (Lakoff 1987), that propose that image schemas and metaphorical processes mediate the understanding of abstract situations in terms of concrete ones.
129
Conclusions The crucial difference between an adapted and a cognitive system is the presence and use of representations. The former engages in closed-loop with the environment and is therefore only capable of on-line processing, while the latter develops decoupled, stand-in representations that can be manipulated even ‘in absentia’ of stimuli, therefore remaining adapted but gaining many additional capabilities. We have illustrated here the symbol detachment problem, or the understanding of how and why situated agents develop representations which are detached from their current sensorimotor interaction, but nevertheless preserve grounding and aboutness. We argue that this process depends crucially on anticipatory capabilities. An inner, simulative loop is firstly evolved and exploited for tuning the control of action; there are in fact many advantages in having a pseudo-closed loop of control, such as the possibility to replace or integrate missing or unreliable information. Successively anticipations are exapted for increasingly complex uses, leading to detachment in representations and disengagement in action. Since representations are developed for engaging in actual and simulated interaction, they remain functionally related to the entity they represent. This is why cognitive systems, which evolve from adapted systems, even having decoupled representations remain attuned to their environment; and this makes them able to survive. We have discussed how this approach deals with the problems of detachment, such as maintaining aboutness, relevance and normativity. It also offers several additional advantages, such as defining a clear evolutionary history and an unitary perspective for the detachment of representations and goals. It also motivates the fact that representations retain an action-oriented nature and some of their sensorimotor aspects (Barsalou 1999), and gives an explanation of why and how representations can be used to substitute, that is their most innovative role. With respect to the existing literature, the main contribution of this paper consists in describing the successive phases of detachment in which several cognitive capabilities arise and change. We have also suggested that representations and goals undergo a parallel detachment process from stimuli and drives. Understanding the evolutionary pathway we have indicated, is also a suitable way to investigate how cognition, and in particular high level cognition, might arise in artificial systems. Organisms pass from stimuli to representations, and from drives to goals thanks to anticipation. Understanding the whole range of anticipatory phenomena is thus the key issue for endowing artificial systems not only with present-directed, but even with future-directed conducts, and to bootstrap increasingly complex cognitive functionalities.
123
130 Acknowledgments This work is supported by the EU-funded project MindRACES: from Reactive to Anticipatory Cognitive Embodied Systems (FP6-511931). Thanks to Joachim Hoffmann for insightful discussions.
References Adams JA (1971) A closed-loop theory of motor learning. J Mot Behav 3:111–149 Aquinas ST (1947) The Summa Theologica. Benziger Bros Balkenius C, Hulth N (1999) Attention as selection-for-action: a scheme for active perception. In: Proceedings of EuRobot-1999, Zurich Barsalou LW (1983) Ad hoc categories. Mem Cognit 11(3):211–227 Barsalou LW (1999) Perceptual symbol systems. Behav Brain Sci 22:577–600 Barsalou LW (2003) Abstraction in perceptual symbol systems. Philos Trans R Soc Lond B Biol Sci 358(1435):1177–1187 Bickhard MH (2001) Function, anticipation and representation. In: Dubois DM (ed) Computing anticipatory systems. CASYS 2000Fourth international conference. American Institute of Physics, Melville, pp 459–469 Blakemore S-J, Decety J (2001) From the perception of action to the understanding of intention. Nat Rev Neurosci 2:561–7 Brentano F (1985) Psychology from an empirical standpoint. Translated by Rancurello AC, Terrell DB, McAlister LL Routledge, London Brooks RA (1991) Intelligence without representation. Artif Intell 47(47):139–159 Butz MV, Hoffmann J (2002) Anticipations control behavior: animal behavior in an anticipatory learning classifier system. Adapt Behav 10(2):75–96 Castelfranchi C (1995) Guarantees for autonomy in cognitive agent architecture. In: Wooldridge M, Jennings NR (eds) Intelligent agents: theories, architectures, and languages. number 890 in LNAI, Springer, pp 56–70 Castelfranchi C (2005) Mind as an anticipatory device: for a theory of expectations. In: BVAI 2005, pp 258–276 Castelfranchi C, Paglieri F (2007) The role of beliefs in goal dynamics: prolegomena to a constructive theory of intentions. Synthese, to appear Clark A (1997) Being there. Putting brain, body, and world together. MIT Press, Cambridge, USA Clark A, Chalmers DJ (1998) The extended mind. Analysis 58:10–23 Clark A, Grush R (1999) Towards a cognitive robotics. Adapt Behav 7(1):5–16 Colombo M, Graziano M (1994) Effects of auditory and visual interference on auditory-visual delayed matching to sample in monkeys (maca fascicularis). Behav Neurosci 108:636–639 Damasio AR (1989) Time-locked multiregional retroactivation: a systems-level proposal for the neural substrates of recall and recognition. Cognition 33:25–62 Damasio AR (1994) Descartes’ error: emotion, reason and the human brain. Grosset/Putnam, New York Decety J (1996) Do imagined and executed actions share the same neural substrate? Brain Res Cogn 3:87–93 Decety J, Chaminade T (2003) When the self represents the other: a new cognitive neuroscience view on psychological identification. Conscious Cogn 12(20):577–596 Dennett DC (1991) Consciousness explained. Little Brown & Co. Boston, USA Desmurget M, Grafton S (2000) Forward modeling allows feedback control for fast reaching movements. Trends Cogn Sci 4:423– 431
123
Cogn Process (2007) 8:115–131 Doya K (2002) Metalearning and neuromodulation. Neural Netw 15(4–6):495–506 Drescher GL (1991) Made-up minds: a constructivist approach to artificial intelligence. MIT Press, Cambridge Fogassi L, Ferrari P, Chersi F, Gesierich B, Rozzi S, Rizzolatti G (2005) Parietal lobe: from action organization to intention understanding. Science 308:662–667 Frith CD, Blakemore S, Wolpert DM (2000) Explaining the symptoms of schizophrenia: abnormalities in the awareness of action. Brain Res Brain Res Rev 31(2–3):357–363 Gallese V (2000) The inner sense of action: agency and motor representations. J Conscious Stud 7:23–40 Gallese V (2001) The ’shared manifold’ hypothesis. from mirror neurons to empathy. J Conscious Stud 8:5–87 Gallese V, Goldman A (1998) Mirror neurons and the simulation theory of mind-reading. Trends Cogn Sci 2(12):493–501 Gallese V, Lakoff G (2005) The brain’s concepts: The role of the sensory-motor system in reason and language. Cogn Neuropsychol 22:455–479 Gallese V, Metzinger T (2003) Motor ontology: the representational reality of goals, actions, and selves. Philos Psychol 13(3):365– 388 Gardenfors P (2004) Cooperation and the evolution of symbolic communication. In: Oller K, Griebel U (eds) Evolution of communication systems: a comparative approach. MIT Press, Cambridge, pp 237–256 Gardenfors P, Orvath M (2005) The evolution of anticipatory cognition as a precursor to symbolic communication. In: Brook S (eds) In: Proceedings of the Morris symposium on the evolution of language. NY, USA Gibson J (1966) The senses considered as perceptual systems. Houghton Mifflin, Boston Glenberg A (1997) What memory is for. Behav Brain Sci 20:1–55 Grush R (2004) The emulation theory of representation: motor control, imagery, and perception. Behav Brain Sci 27(3):377– 396 Harnad S (1990) The symbol grounding problem. Physica D 42:335– 346 Hesslow G (2002) Conscious thought as simulation of behaviour and perception. Trends Cogn Sci 6:242–247 Hoffmann J (2003) Anticipatory behavioral control. In: Butz M, et al. (eds) Anticipatory behavior in adaptive learning systems: foundations, theories, and systems. Springer, Heidelberg, pp 44–65 Hoffmann H (2007) Perception through visuomotor anticipation in a mobile robot. Neural Netw 20:22–33 von Holst E, Mittelstaedt H (1950) Das reafferenzprinzip. Naturwissenschaften 37:464–476 Hommel B, Musseler J, Aschersleben G, Prinz W (2001) The theory of event coding (tec): a framework for perception and action planning. Behav Brain Sci 24(5):849–78 Hurley SL (2005) Active perception and perceiving action: the shared circuits hypothesis. In: Gendler T, Hawthorne J (eds) Perceptual experience. Oxford University Press, Oxford Iacoboni M (2003) Understanding others: Imitation, language, empathy. In: Hurley S, Chater N (eds) Perspectives on imitation: from cognitive neuroscience to social science. MIT Press, Cambridge James W (1890) The principles of psychology. Dover, New York Jeannerod M (1994) The representing brain: neural correlates of motor intention and imagery. Behav Brain Sci 17:187–245 Jeannerod M (2001) Neural simulation of action: a unifying mechanism for motor cognition. NeuroImage 14:S103–S109 Kant I (1781/1998) Critique of pure reason. Cambridge University Press, Cambridge
Cogn Process (2007) 8:115–131 Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9:718–27 Kosslyn SM, Sussman A (1994) Roles of imagery in perception: or, there is no such thing as immaculate perception. In: Gazzaniga M (ed) The cognitive neurosciences. MIT Press, Cambridge, pp 1035–1042 Kunde W, Koch I, Hoffmann J (2004) Anticipated action effects affect the selection, initiation and execution of actions. Q J Exp Psychol A 57(1):87–106 Lakoff G (1987) Women, fire, and dangerous things: what categories reveal about the mind. University of Chicago Press, Chicago Mahon B, Caramazza A (2005) The orchestration of the sensorymotor systems: clues from neuropsychology. Cogn Neuropsychol 22:480–494 Mehta B, Schaal S (2002) Forward models in visuomotor control. J Neurophysiol 88:942–953 Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9(8):1265–1279 Middleton FA, Strick PL (2000) Basal ganglia output and cognition: evidence from anatomical, behavioral, and clinical studies. Brain Cogn 42(2):183–200 O’Regan J, Noe A (2001) A sensorimotor account of vision and visual consciousness. Behav Brain Sci 24(5):883–917 Pezzulo G, Calvi G (2006a) A schema based model of the praying mantis. In: Nolfi S, et al (eds) From animals to animats 9, Proceedings of the ninth international conference on simulation of adaptive behaviour. vol LNAI 4095. Springer, Berlin, pp 211– 223 Pezzulo G, Calvi G (2006b) Toward a perceptual symbol system. In: Proceedings of the sixth international conference on epigenetic robotics: modeling cognitive development in robotic systems. Lund University Cognitive Science Studies 118
131 Pezzulo G, Baldassarre G, Butz MV, Castelfranchi C, Hoffmann J (2006) An analysis of the ideomotor principle and tote. In: Butz M, et al (eds) Proceedings of the third workshop on anticipatory behavior in adaptive learning systems (ABiALS 2006) Piaget J (1954) The construction of reality in the child. Ballentine ¨ ber Erkenntnis, Popper KR (1996) Alles Leben ist Problemlo¨sen. U Geschichte und Politik. R. Piper-Verlag, Mu¨nchen Rizzolatti G, Arbib MA (1998) Language within our grasp. Trends Neurosci 21(5):188–194 Rizzolatti G, Fogassi L, Gallese V (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci 2:661–670 Roitblat H (1980) Codes and coding processes in pigeon short-term memory. Anim Learn Behav 8:341–351 Rosenblueth A, Wiener N, Bigelow J (1943) Behavior, purpose and teleology. Philos Sci 10(1):18–24 Roy D (2005) Semiotic schemas: a framework for grounding language in action and perception. Artif Intell 167(1–2):170–205 Sloman A, Chappell J (2005) The altricial-precocial spectrum for robots. In: Proceedings of IJCAI Smith BC (1996) On the origin of objects. MIT Press, Cambridge Taylor C (1971) What is involved in a genetic psychology? In: Mischel T (ed) Cognitive development and epistemology. Academic, New York, pp 393–416 Tolman E (1932) Purposive behavior in animals and men. AppletonCentury-Crofts, New York Vygotsky LS (1978) Mind in society: The development of higher psychological processes. Harvard University Press, Cambridge Wolpert DM, Kawato M (1998) Multiple paired forward and inverse models for motor control. Neural Netw 11(7–8):1317–1329 Wolpert DM, Gharamani Z, Jordan M (1995) An internal model for sensorimotor integration. Science 269:1179–1182
123
