preprint of article in Phenomenology and the Cognitive Sciences, doi: 10.1007/s11097-014-9396-5 published online 3 October 2014 http://link.springer.com/article/10.1007/s11097-014-9396-5

The origin of agency, consciousness, and free will J. H. van Hateren Johann Bernoulli Institute for Mathematics and Computer Science, University of Groningen, Groningen, The Netherlands e-mail: [email protected] Abstract Living organisms appear to have agency, the ability to act freely, and humans appear to have free will, the ability to rationally decide what to do. However, it is not clear how such properties can be produced by naturalistic processes, and there are indeed neuroscientific measurements that cast doubt on the existence of free will. Here I present a naturalistic theory of agency, consciousness, and free will. Elementary forms of agency evolved very early in the evolution of life, utilizing an extension of the basic Darwinian scheme that combines and entangles deterministic and stochastic causation. The extension effectively produces an active form of causation, as well as meaning intrinsic to the organism. Consciousness arose when animals evolved advanced nervous systems and social lifestyles that enabled communication mutually affecting each animal's intrinsic meaning. I argue that various forms of agency that subsequently arose in evolution, preconscious, conscious, intelligent, symbolic, and rational agency, still coexist in humans. This coexistence, combined with the fact that agency is not instantaneous but takes time to build up, makes simple interpretations of neuroscientific results on free will (taken here as rational, symbolic agency) problematic. The conclusion of the study is that conscious free will, including a form of agent causation, is fully consistent with a naturalistic world-view and can be produced by the specific processes discussed. The result closely resembles existing approaches to life and mind that are inspired by phenomenology. Keywords Meaning ∙ Agency ∙ Consciousness ∙ Thought ∙ Free will ∙ Intentionality 1 Introduction Whether and to what extent humans have free will is a contentious topic. Although this question was originally primarily debated from the perspectives of philosophy and theology, it has increasingly been drawn into the realm of the natural sciences. This has happened as part of the general trend that the dominant world-view has become more and more mechanistic over the last centuries. At present, agency and free will are under intense investigation by the neurosciences, with several results casting doubt on whether free will exists at all. These doubts partly originate from and are enhanced by implicit assumptions on how the physical world works, namely as a fully predictable mechanistic and deterministic system. Darwin's theory of evolution from the mid-19th century, the revolution of molecular biology from the mid-20th century, and the new neuroscience techniques from the late 20th century have indeed shown that life and mind are amenable to mechanistic analysis, at least to some extent. However, from a more phenomenological perspective, both life and mind appear to be less mechanistic than suggested by such analyses. Phenomena like consciousness, values, and free will seem to be difficult to reconcile with mechanistic models. Thus there is, apparently, a severe conceptual conflict between naturalistic and phenomenological approaches to understanding the mind in general and free will in particular. Here I will argue that this conflict can be fully resolved, and I will present plausible schematic models showing how a naturalistic approach can explain genuine agency, goals, consciousness, and free will. The theory presented here is conjectural, and should be seen primarily as a draft proposal rather than a detailed model. Nevertheless, much of it depends on evolutionary and developmental mechanisms that are partly known and amenable to further experimental tests. The theory can also be used to make predictions that are testable at a phenomenological level. But the primary purpose of this article is to show that phenomena that are indeed best investigated at a high level of abstraction, such 1

as free will, are nevertheless consistent with the underlying physical world. Interestingly, although the case is built up from components traditionally affiliated with the natural sciences, the result bears many resemblances to existing approaches that are traditionally affiliated with the humanities, such as phenomenology and semiotics. The method I am using here is one of construction rather than reduction, in the spirit of Anderson (1972). In a constructionist approach, one starts with simple components and shows how combining them can lead to complex systems with new, emerging properties. In a reductionist approach, one starts with a complex system, and tries to identify simple components that can together explain the system. Reductionism is one of the standard theoretical and experimental approaches in the natural sciences, and often leads to powerful quantitative models. The disadvantage is, however, that it has a limited scope: models will quickly become intractable when too many levels of complexity are traversed, risking oversimplification when not properly acknowledged. How many levels are feasible depends on the field and the topic. Constructionism can traverse more levels of complexity, as hopefully shown in this article. More levels are possible, because the method only aims at an approximate, qualitative explanation of the more complex phenomena. Thus it will not produce comprehensive quantitative models covering its entire range, but nor can reductionist methods do this, as just discussed. Although testing predictions of the theory presented here may be possible in key areas, overall support can only come from accumulating circumstantial evidence. The insight the theory provides may therefore remain based on conjecture for some time to come. The article is organized as follows. In Section 2 and its subsections the theory is presented, ranging from the origin of agency to the emergence of consciousness and free will. In Section 3 the theory is first compared with several existing approaches to free will, and its consequences for neuroscientific experiments on free will are assessed. It is subsequently argued that the theory implies intentionality (Section 3.4) and that it is consistent with and related to approaches that are associated with phenomenology, in particular those that take life and mind to be enacted, embodied, embedded, and situated (Section 3.5). 2 Theory Free will is constructed here from components that originated over a span of billions of years. It can therefore best be explained and understood from an evolutionary perspective, including reference to present-day organisms and mechanisms that remained close to the original ones. Some of the components have been explained before in more detail elsewhere, but although I will refer to that where appropriate, the present article should be sufficiently self-contained to understand the rationale. Below I will subsequently propose evolutionary and developmental mechanisms for the origin of agency, goals and meaning, consciousness, language and thought, and free will. 2.1 Agency An organism has an elementary form of agency when it can freely steer its own behaviour, at least to some extent (for a similar concept of agency see Heisenberg 2009 and Brembs 2011). Such behavioural freedom is difficult to reconcile with deterministic forms of causation. Deterministic causation is illustrated in Fig. 1a, where the trace (called ‘variable’ below) symbolizes the time course of some aspect of a system, such as a system parameter or state, or a more abstract property of the system. The arrows symbolize a causal relationship with earlier and later variables, either of the same or of another system. The variable shown is in the middle of a chain of causation, with its time course completely determined by its causes, which may be many. It subsequently contributes to causing the time course of other variables. An organism that can be fully specified as a deterministic system may be able to steer itself, but certainly not freely in the sense as intended here – thus different from the restricted form of free as defined by e.g. Dennett 1984, see Section 3.1 below. During the end of the 19th and the beginning of the 20th century it became clear that deterministic causation is not the only form of causation present in nature. In addition, stochastic processes, such as thermal and quantum noise, can start new causal chains spontaneously, as illustrated in Fig. 1b. Such causes are therefore uncaused themselves (or at least not caused by identifiable causes), and are random for all practical purposes. Random does not mean completely unpredictable, 2

Fig. 1 Origin of agency and intrinsic meaning. (a) In deterministic causation, a time-varying variable (trace, representing a system parameter, state, or property) is caused by (left arrow) and causes (right arrow) other variables. (b) In stochastic causation, a random variable is the start of new chains of causation. (c) In modulated stochastic causation, a non-negative deterministically caused variable drives the variance of a stochastic process. (d) Behaviour of an organism ultimately depends on Darwinian fitness ftrue, which is assumed to be approximated by an internal estimate fest, driving the variable component of behaviour that cannot be chosen based on previous learning. The scheme is evolvable when low fest produces high variability and high fest low variability, symbolized by ~1/ fest. Arrows 1-3 correspond to those in (c). Continually utilizing modulated stochastic causation in a feedback loop A running through the organism produces an elementary form of agency, and intrinsic meaning as embodied in the form of fest.

because the statistical properties of the variable, such as mean and variance, may be known. However, such statistical properties are assumed to be constant in time and can therefore not act as a cause (acting as a cause is taken here as producing a change in time). The individual, random fluctuations of the variable are the source of causation. Although random fluctuations may start microscopically, nonlinear interactions between system components can easily amplify them to macroscopic scales, such as behaviour (reviewed in Faisal et al. 2008). But an organism that lets its behaviour steer by randomness could at best be called spontaneous (in the weak sense of stochastically caused), not free. Free requires spontaneity in combination with a certain deliberateness. A form of causation that combines deterministic and stochastic causation is illustrated in Fig. 1c. It is called modulated stochastic causation. A non-negative variable that is caused deterministically (left trace) drives the variance of a stochastic variable (right trace). The mean of the stochastic variable remains constant. The resulting causation (arrow 3) is almost completely stochastic, except for the deterministic variance modulation. It is clear that this type of causation is theoretically possible. That it also exists in nature became clear from results from molecular biology. When cells divide, their hereditary properties may change (mutate) through a range of mechanisms. In principle, mutation is a stochastic process, long thought to be characterized by a genetically controlled, but fixed mutation rate. However, in the last decade or two it has gradually become clear that cells can change their mutation rate also during their lifetime (reviewed in Galhardo et al. 2007). When a cell is under stress, for example because of adverse environmental conditions, it tends to increase its mutation rate. Although this leads to many detrimental mutations in its offspring, it also increases the chances that a mutation occurs that is so beneficial that it can save the cell's line of descent from extinction. Overall it is an evolvable strategy (Ram and Hadany 2012) because of the special characteristics of life: many cell lines become extinct, but the ones that are well adapted can grow exponentially in numbers. The mechanism of stress-modulated mutation rate conforms to the modulated stochastic causation of Fig. 1c, with mutation rate in the role of variance. The genetic mechanisms mentioned above only work at evolutionary timescales. However, an equivalent mechanism can work at the behavioural timescale that is relevant for the phenomena discussed in this article. Figure 1d shows the basic scheme. An organism is embedded in a timevarying environment (including other organisms), and its potential success is quantified by the Darwinian fitness ftrue. Fitness is defined here in its most basic form as the expected number of offspring of an organism (over its lifetime), but for social and cultural species it should include indirect components reflecting the success of related organisms or the success of groups as reflecting on the individual and its kin. It is a predictive, probabilistic variable, a function of time that continuously tracks the potential success. When circumstances deteriorate, such as during a famine, it declines, and it can rise again when circumstances improve. 3

When the environment changes during an organism's lifetime, the organism can respond to that through behavioural change. Such changes are particularly easy in organisms with a well developed nervous system, and I will restrict the further discussion in this article to such animals. When an environmental change has occurred before, an adequate response may be implicitly known to the animal, leading to what is called the ‘learned component’ in Fig. 1d. This may have a genetic component, such as in reflexes that have been established through previous natural selection, or it may have physiological or neural components as established through some form of learning during the animal's lifetime. Usually it will be a combination, with genetic mechanisms enabling, though not specifying, development and learning. When an environmental change has not occurred before, the animal has no proven course of action. However, when the change decreases the animal's fitness, not changing behaviour could eventually lead to death and is therefore a poor strategy. Without knowledge of the right course of action, the change of behaviour must be in a random direction. It is conjectured here that the average size of the behavioural change (its expected variance) should be a function of the fitness of the animal, by analogy with the mutation rate being a function of the cellular stress response. In the latter case, high stress implies low fitness and causes high genetic variance, and the genetic variance is thus inversely related to fitness. Similarly, the behavioural variance should be inversely related to fitness. A complication is that fitness itself, ftrue, is not directly available to the animal. It is not observable, and could only be computed, in principle, through an accurate simulation of the animal's dynamical structure and its interactions with the environment (including other organisms). That is clearly infeasible for the animal. However, an approximate, estimated version of ftrue, called selfestimated fitness or fest here, is feasible. The term ‘estimate’ is used in this article in the theoretical, technical sense (as in estimation theory), not necessarily perceived by the animal. An estimate fest could be based on a (large) range of indicators of fitness, such as the internal physiological state of the animal and external indicators obtained through its senses. The self-estimated fitness is expected to be present only implicitly in the animal's physiology and nervous system, in a distributed way. Similarly, its influence on behavioural variability is expected to be diffuse as well. In Fig. 1d, the inverse relationship between the variable component of behaviour and fest is symbolized by ~1/ fest, although the actual relationship may be more complicated. Both this relationship and the form of fest are important for the animal's chances of survival and reproduction, and are therefore under selection pressure, driving them to forms that promote fitness. The A loop in Fig. 1d is in fact a feedback loop1 running through the organism, with causation conforming to modulated stochastic causation (arrows 1-3 correspond to those in Fig. 1c). Each time the loop is traversed, behaviour is changed, a little when fest is high and more when fest is low. The resulting behaviour is then evaluated again, through fest, in the next looping (in a manner of speaking: in reality the loopings are not distinct because the feedback is acting continually). The result is that behaviour remains fairly constant when fest is high, changing just a little such that it does not prevent finding behaviour that is even better. However, when fest is low, large changes are made in each looping, often resulting in no improvement or even deterioration, but occasionally in an improved fest. When the latter occurs, subsequent behavioural variability is decreased, thus keeping behaviour close to the improved variant, only slowly diffusing away. Simulations show that this optimization through variance modulation is indeed an evolvable strategy (van Hateren 2014a). Suitable behaviour is found more quickly than without modulation, i.e., with a fixed behavioural variance. Thus merely using an estimate of fitness in a feedback loop as in Fig. 1d is already sufficient to improve finding suitable behaviour, even if the consequences of behavioural variations are completely unknown. The A loop of Fig. 1d produces an elementary form of agency. The reason is that it uses modulated stochastic causation continually, each time the loop is traversed. The resulting behavioural trajectory (the chain of subsequent behaviours) is spontaneous and deterministic at the same time. Each behavioural change is almost completely stochastic, with only its expected variance determined (by fest). So a single step in the behavioural trajectory could not be distinguished reliably from a random change. However, a tiny part of this step has a deterministic origin (the part determined by the variance as driven by fest), and this tiny part accumulates with each subsequent step in the trajectory. A The terms ‘feedback’ and ‘feedback loop’ are used in this article in the technical sense as any kind of cyclic, circular, or mutual causation, and not as denoting only stabilizing (negative) feedback. 1

4

trajectory consisting of a large number of such steps is strongly determined by fest. A behavioural trajectory, as a whole, is therefore neither completely stochastic (because it is partly determined by fest), nor completely deterministic (because each step is primarily stochastic). This form of causation, also called ‘active causation’, therefore bears the signature of agency: the behaviour is spontaneous and determined in a way that could not be disentangled. Each step of the feedback loop depends, through fest and the realized noise, on all previous steps – in addition to other factors, such as incorporated in the time-varying environment. As indicated in Fig. 1d, in addition to the variable component of behaviour, there is also a learned component. The double arrow symbolizes that it is continually used for actual behaviour, but also continually updated by the behaviour as it confronts the environment and as it is modified by the variable component of behaviour. Such interactions can be highly complex and detailed modelling of them is beyond the scope of this article. The main point is that the learned and variable components of behaviour do interact, with the variability produced on top of established behaviour that is expected to work well, and with established behaviour modified when new variations turn out to work well. The crucial point for the topic of this article is that purely executing a learned behavioural program involves no agency – it is automatic, even if it would be adaptive in a deterministic way. Only the variable, stochastic component of behaviour, controlled by fest, is the ultimate source of agency. Nevertheless, this variability will usually be executed within the fairly controlled envelope of established behavioural patterns of known utility, and this is indeed necessary for the more complex extensions of agency as discussed below. 2.2 Goal-directedness and intrinsic meaning The A loop of Fig. 1d produces not only agency, but also an intrinsic goal-directedness in the animal. The reason is that the loop tends to increase fest, usually after fest has declined first because of variations in the environment. Increasing (or at least maintaining) fitness is a tendency of all organismal lineages subjected to basic Darwinian evolution, through ftrue. But it should be realized that ftrue and fest are fundamentally different from the point of view of the organism. For the organism, ftrue is an external variable that is neither observable nor directly controllable. Through ftrue evolution runs its course like any other physicochemical system. There are no values or goals involved, nor agency. Survival and reproduction (high ftrue) are merely consequences of how the system is structured. The system is valuefree, and life is not valued higher than death (see Davies 2009, pp 86-87). However, this is dramatically different for an animal that has internalized ftrue as fest. Now fest is an intrinsic variable, produced and controlled by the organism. Moreover, the form of fest is not entirely fixed. It could incorporate varying mixtures of internal and external variables into its estimate of ftrue, as long as it is sufficiently close to ftrue to obtain a fitness advantage (as ultimately determined by natural selection). Because fest is intrinsic, modifiable, controllable, and part of a control loop that implements agency, high fest is a true goal of the organism. In contrast, ftrue is at most an ‘as if’ goal. In practice, the goal of high fest will consist of a bundle of sub-goals, such as obtaining enough food, finding mates, and preparing for the winter, all contributing to the overall goal of high fest. If an animal has genuine intrinsic goals and agency, it must mean that these goals have value to the animal. This internal value is called ‘intrinsic meaning’ here, where meaning is used in the broad sense of value, import, significance, and purpose (the sense as in the ‘meaning of an action’, not as in the more narrow ‘meaning of a word’). Intrinsic meaning is embodied in the form of fest. The form of fest determines which internal and external variables are incorporated in the animal's self-estimated fitness, and how strongly they are weighted. In other words, the form of fest signifies what the animal estimates to be important for its own survival and reproduction (and more general forms of survival and reproduction, such as in the case of social and cultural species). 2.3 Consciousness As argued above, the particular characteristics of the A loop of Fig. 1d produce intrinsic meaning and an elementary form of agency in the organism. However, these are presumably present in all current forms of life, including unicellular life and plants, and thus are still a long way from consciousness. It is conjectured here that an elementary form of consciousness first arose in animals capable of a form 5

of communication that modified their intrinsic meaning, and that more complex forms developed from there. Animals of a particular species may increase their fitness by coordinating their behaviour, and an important way to accomplish such coordination is through communication. Ultimately, at an evolutionary timescale, systems of communication can only remain stable if they promote high fitness. Communication can affect fitness by three basic pathways. The simplest is directly through ftrue, where communication is steered by genetically determined behavioural programs, for example as in mating rituals in insects. Behaviour is reflex-like and inflexible, and this form of communication is called ‘instrumental’ because it resembles communication within and between machines. A second pathway is indirectly, through fest. Because this involves the agency produced by the A loop of Fig. 1d, there is some behavioural freedom during communication. It also involves the intrinsic meaning embodied in fest, and it is therefore called ‘meaningful communication’ (though the meaning is only implicit, and is not explicitly experienced). A third pathway utilizes the fact that fest is under control of the animal, and therefore can be optimized to specific goals, as served by the communication, during the lifetime of the animal. Because this communication results in modifying the form of fest, it is called ‘formative communication’. Note that modifying the form of fest as intended here goes beyond regular learning. Regular learning only requires adjusting the parameters of a specific form of fest, restricting it to a fixed space of possibilities. In contrast, modifying the very form of fest – including making it depend on new parameters in unanticipated ways – produces a malleable, unpredictable and potentially expandable space of possibilities. An example of formative communication may be the formation, through communication, of the mother-infant bond in mammals (and especially in humans, Trevarthen and Aitken 2001), and more generally the formation of pair-bonds, such as in birds and mammals. It is conjectured here that formative communication, i.e., an act of communication that modifies an animal's intrinsic meaning, is accompanied by a subjective, conscious experience. The rationale for this conjecture is the following. Intrinsic meaning is considered to be a real physical phenomenon, not just a theoretical construct. ‘Physical’ is used here in a general sense, indicating a relatedness to the natural world, as opposed to the world of ideas as produced by the human mind. Notwithstanding its reality, in most organisms intrinsic meaning is only present implicitly, distributed throughout the organism. But once it becomes involved in acts of communication, aspects of it must be made sufficiently explicit such that they can be coded into regular physical behaviour. The recipient of the communicative act then has to perceive that behaviour and decode it such that it can be assimilated into its own intrinsic meaning. Usually it will then lead to reciprocated communication, as in a dialogue. The main conjecture is that modifying a real physical phenomenon (intrinsic meaning) through acts of formative communication, either on the sending or receiving end, leads to another real physical phenomenon, subjective experience. For a discussion of the plausibility of this conjecture see Section 3. Fig. 2a shows the basic act of formative communication in a symbolic form. Two individuals, S1 and S2, modify each other's intrinsic meaning (with the modifiability symbolized by the oblique arrows through fest) in a dialogue (the double-headed arrow). For mammals the initial, most elementary formative communication is presumably the establishment and maintenance of the mother-infant bond (as extensively studied in humans; Trevarthen and Aitken 2001). This elementary dialogue can subsequently be used as a stepping stone to build more complex forms of consciousness, through development and learning (Mead 1934; Reddy 2003). The precise way this works may be complex, and the simple steps presented here should again be taken as a draft proposal rather than a proposal for a detailed model. The explanation concentrates here on conscious perception of an object, but similar sequences can be constructed for the perception of other individuals, of groups of individuals, and of the self (van Hateren 2014b). In Fig. 2b S1 stands for the infant, and S2 for the adult. Initially, S1 has a (nonverbal) dialogue with S2, which is therefore subjectively experienced. S1 can also perceive an object O, but cannot yet experience that subjectively (single-headed arrow). At the same time, the adult S2 interacts consciously with O as if in a formative dialogue, aided by an internalized version of O (symbolized by Ô). Gradually, S1 internalizes S2 (Ŝ2 in the second diagram), including the interaction of S2 with O. Over time, the internalized version of S2 becomes sufficiently realistic to function even in the absence of S2 (third diagram). Finally, Ŝ2 as an intermediary for interacting with O becomes obsolete, and S1 can interact consciously with O in a direct way, aided by an internalized version of O, or can even interact with Ô without the presence of O (fourth diagram). 6

Fig. 2 Origin of consciousness, language, and thought. (a) In formative communication, as evolved in animals with advanced nervous systems and social lifestyles, dialogue (double-headed arrow) between two individuals S1 and S2 modifies the form of fest (symbolized by the oblique arrows). This implies modifying intrinsic meaning, assumed to lead to subjective experience. (b) Elementary consciousness of an infant S1 when communicating with an adult S2 can gradually develop into more complex forms of consciousness, illustrated here for object consciousness. First, the object O is perceived, but not consciously (single-headed arrow), along with the interaction of S2 with O, using implicitly an internalized version Ô. First S1 gradually internalizes S2, subsequently S1 can have a dialogue with Ŝ2 even when S2 is absent, and finally S1 does not need Ŝ2 for dialogic interaction with O or Ô. (c) As in (b), but now the dialogue with S2 is retained, where adults S1 and S2 communicate about a concrete (O) or abstract (X) item, utilizing internalized versions of each other. This results in symbolic dialogue (language) and its internalized variant (thought, final diagram).

The elementary and derived forms of consciousness explained above are presumably widespread in the animal kingdom, at least present in animals with strong bonding and social lifestyles, as common in mammals and birds. They lead to subjectively experienced forms of agency. However, such forms still lack the symbolic and rational aspects necessary for free will. How such aspects might develop is explained below, with the schemes inspired by experiments and theories on the development of mind as can be found in Trevarthen and Aitken (2001), Reddy (2003), Tomasello and Carpenter (2007), and earlier work by Mead (1934) and Vygotsky (1986). 2.4 Language and thought Figure 2c starts with the second diagram of Fig. 2b. In this state, S1 has a dialogue with both S2 and an internalized version of S2. In contrast to Fig. 2b, the dialogue with S2 is maintained in the next step, and the perception of O is transformed into a consciously experienced simulated dialogue. This is illustrated in the second diagram of Fig. 2c, where S2 has a similar dialogue with Ŝ1 and O. The result is an advanced form of dialogue, where S1 can communicate with S2 about O using the perspective of S2 (through Ŝ2) and S2 can communicate with S1 about O using the perspective of S1 (through Ŝ1). Here and below, S1 and S2 could stand for communicating adults. In its basic form, the behavioural act used for communication (the ‘sign’) could resemble the object it refers to, similar to an icon referring to a concrete object. But both icons and objects may gradually become more abstract (third diagram), leading to symbolic dialogue about abstract objects (X), such as categories of concrete objects, social situations, or events happening at another place or time. Human language is the prime, perhaps only, example of symbolic dialogue about (usually) abstract objects. Again the dialogue of S1 with S2 can be internalized, with S1 having a simulated dialogue with an internalized version of S2 or of itself (Ŝ1). In either case, there is a subjectively experienced dialogue about X (symbolic thought). The final diagram of Fig. 2c symbolizes this. This scheme and the diagrams before bear some resemblance to the Peircian variant of semiotics (e.g., Chandler 2007), with signs carrying the symbolic communication, and their relationship with X or O being mediated by an interpretative process performed by the S1- Ŝ1, S1- Ŝ2 or S2- Ŝ1 dialogues. Thus there is only an indirect connection between the sign and the object (consistent with figure 1.5 in Chandler 2007). The construction of symbolic language and thought opens the way to developing full-blown free will, as proposed in the next section.

7

Fig. 3 Various forms of agency. (a) No agency, behaviour controlled by ftrue, reflex-like interaction with perceived and handled object. (b) Preconscious agency, controlled by fest in the A loop of Fig. 1d; there is some agency, but no subjective experience as there is no formative communication modifying fest. (c) Conscious agency, subjectively experienced because there is a simulated dialogue with O and its internalized version Ô. (d) Intelligent agency, where elementary, internal operations on Ô enable targeted modification of fest, thus producing more complex interactions with O. (e) Symbolic agency, where the internal operations on X can become highly sophisticated by using symbolic, abstract reasoning. (f) Rational agency, similar to (e), with X now strongly dependent on cultural, intersubjective values and norms.

2.5 Agency and free will Agency and free will lie on a continuous scale, variants of which can be found throughout the kingdoms of life. To gain some perspective on this, Fig. 3 shows several of the possibilities. The scheme is not meant to be complete, because there are intermediate and more complex versions. However, it is clear that the simplest forms must be very ancient, and the most complex form (human free will) arose only fairly recently. Again, the explanation concentrates on interaction with and perception of objects, but can be readily extended to other forms of dialogue. Figure 3a shows a case where O is perceived (left straight arrow) and reacted to (right straight arrow) without agency. The interaction is evaluated directly through ftrue, and is automatic and reflexlike. An example is the reflexive withdrawal of a hand when touching a hot object. Similar interactions can be found in any life-form when vital functions are involved. In Fig. 3b, the interaction is evaluated through fest, providing some behavioural freedom through the A loop of Fig. 1d (symbolized in Fig. 3b-f by the arrows connecting fest and the individual). It could be called ‘preconscious agency’, because there is no formative dialogue (the form of fest is not modified) and therefore no subjective experience, but still an elementary form of agency. This form of agency is presumably the most common form in animals, including humans. For example, scratching an itch without being aware of it is done not completely reflexive, because the effects and appropriateness for the goals of the animal are presumably monitored, and the behaviour may subsequently be varied. However, all is done without subjective experience. Many animals have evolved nervous systems that are sufficiently sophisticated to allow simulations of behaviour before actually performing a specific behaviour. The advantage is then that at least part of the fitness consequences can be simulated in advance as well, and that behavioural changes with a strong (expected) disadvantage can be avoided (e.g., Dennett 1996, chapter 4; Dehaene and Changeux 2000). To keep the explanations simple, this possibility is not explicitly shown in the schemes of Fig. 3, but all forms of agency (Fig. 3b-f) are likely to have such simulation capabilities in animals with appropriate nervous systems. Incorporating simulation in the basic scheme of Fig. 1d works in a simple form as follows (see van Hateren 2014a for a specific scheme and computations). A series of behavioural trajectories are simulated by running virtual versions of the A loop of Fig. 1d, with simulated behavioural variance controlled by simulated fest. The trajectories can be simulated simultaneously (in parallel) by the nervous system for non-symbolic agency (Fig. 3b-d), but 8

presumably require consecutive (serial) evaluation for symbolic agency (Fig. 3e, f; see also Section 3.2). In effect, the simulated trajectories form a population. Because of the way fest modulates variance, the population will gradually get a probability density concentrating in areas of high simulated fitness. Finally, an actual behaviour is produced as a realization of this probability density. Such a behaviour has an increased probability of being beneficial compared with behaviour produced directly, without simulation. Note that this simulation occurs in a virtual version of the A loop as symbolized by the arrows connecting fest and the individual in Fig. 3. This needs to be distinguished from the simulation of dialogue (as in Fig. 2) that corresponds to the arrows connecting fest and Ô or X in Fig. 3c-f. When present, dialogic simulation acts as a component in the simulated A loop actually producing agency. ‘Conscious agency’ as shown in Fig. 3c is equivalent to the right-most diagrams of Fig. 2b. The interaction with the object is subjectively experienced, because it involves modifying fest. However, the modification of fest is fairly straightforward, involving the immediate goals of the animal and the role the object has for those goals. When the goals are less immediate, and a series of sub-goals have to be produced and followed in what could be called ‘presymbolic reasoning’, the agency can be classified as ‘intelligent’ (Fig. 3d). An example may be crows that bend metal wire into hooks in order to obtain food (Weir et al. 2002). Although this appears to involve reasoning in some sense, requiring planning of subtasks in order to reach a somewhat distant goal, there are no indications that it involves symbolic reasoning. A similar type of agency may have been involved when hominins first started to produce stone axes, a few million years ago. Once symbolic language and thought had developed, as in Fig. 2c, symbolic reasoning became possible, leading to ‘symbolic agency’ (Fig. 3e). Symbolic reasoning makes it easy to make plans for distant places and distant times, and to perform complex behaviours, with many sub-goals. It is clear that humans have this form of agency. It is not clear if it exists in other animals at all, but if it does it is presumably only of a limited nature. When symbols are strongly dependent on culture (Fig. 3f), including norms derived from social and cultural life, symbolic reasoning can become rational reasoning, and the agency ‘rational agency’, or free will as defined here (free will proper in the terminology of Schlosser 2014). Although symbolic agency may be seen already as a weak form of free will, I assume that the stronger forms of free will as traditionally understood imply responsibility. The latter must therefore involve rational reasoning as dependent on culturally established symbols and norms. Note that the word ‘rational’ as used here involves reasoning without excluding emotion. Emotion presumably plays an important role for nudging or driving organisms with a modifiable fest into directions that lead to a beneficial ftrue or that prevent straying too far away from a beneficial ftrue, at least probabilistically. It is therefore an intrinsic part of the forms of agency depicted in Fig. 3c-f. Rational agency is likely unique to humans, and leads to moral notions such as reasoned responsibility and reasoned guilt. It is clearly only a part of human agency, because all forms of agency shown in Fig. 3 still coexist in humans. Because it incorporates feedback onto the form of fest, which is part of the A loop of Fig. 1d, rational agency ultimately derives its behavioural freedom and meaning from this loop. However, the intrinsic meaning as embodied in fest in a strongly social and cultural species as the human one cannot be seen as independent of culture. In other words, rational agency modifies intrinsic meaning, and is subjectively experienced, in a way that partly depends on culture. 3 Discussion The theory presented here states that organisms obtain agency through the continual cycling of a feedback loop utilizing stochasticity modulated by a self-estimated fitness fest that is under selection pressure to conform with the actual fitness ftrue. The feedback loop not only produces agency, but also intrinsic meaning in the organism. The transition from basic agency to free will requires consciousness, and a major conjecture made in this article is that consciousness first arose from communicating intrinsic meaning. Although this conjecture will ultimately need to be tested empirically, its a priori plausibility is supported, to varying degrees, by the following considerations. First, it seems reasonable to assume that the transfer of intrinsic meaning (an emergent phenomenon of life) leads to another emergent phenomenon (consciousness). Second, the conjecture puts the cut between non-conscious and conscious life at a plausible position: consciousness predates the human species, but requires at least a sophisticated nervous system. Third, it explains the unity of 9

consciousness (van Hateren 2014b): since an organism has only one ftrue, it should also have only one fest. Modifying this one fest must then provide an experience that is perceived as integrated. Fourth, despite the fact that there is only one fest, there are many different ways by which the form of fest can be modified when an organism interacts with reality (either directly or indirectly through simulation). Therefore there are many different forms of subjective experience (the qualia). Fifth, the conjecture appears to fit well with how consciousness is believed to develop in infants (Mead 1934; Trevarthen and Aiken 2001; Reddy 2003). Subsequent stages in this process are sketched in Fig. 2b and discussed in more detail in van Hateren (2014b). And finally, the conjecture explains why consciousness seems to be associated with external and internal dialogue (Fig 2b; mostly nonverbal, but partly in the form of language in humans, Fig. 2c). Exactly why consciousness is experienced as such, i.e. if that is explained by yet deeper, underlying mechanisms in nature, is not specified by the current theory. Although such an explanation may be found eventually, it seems more likely that the subjective phenomenon is a brute fact of nature, irreducible, just like the stochasticity of systems described by quantum physics appears to be irreducible (no hidden variables). The internalization of dialogue as explained in Fig. 2b and c bears some resemblance to higherorder theories of consciousness (e.g. Carruthers 2005). But the basic phenomenon of consciousness is taken here as first order, arising from communicating intrinsic meaning. The current theory differs from other theories in having a uniquely stochastic core depending on evolutionary fitness. The theory has indeed much in common with the biological naturalism of Searle (e.g. Searle 2007, 2013), in that both consciousness and free will are viewed as, in principle, not different from other biological properties that evolved in evolution. The specific mechanisms proposed here give these general notions a solid basis in terms of function, evolution, and development. With respect to the relationship between neurobiology and free will, Searle (2007, p. 76) notes that hypothesizing that a causal indeterminacy at the psychological level is matched by one at the neurobiological level2 does not necessarily imply randomness at the psychological level: “the indeterminacy at the micro level may (..) explain the indeterminacy of the system, but the randomness at the micro level does not by itself imply randomness at the system level.” (emphasis in original). That is indeed what is happening in the A loop of Fig. 1d: behavioural trajectories are indeterminate to some extent, but not random, whereas the source of this indeterminacy is stochasticity at a lower level (albeit modulated by fest). According to Searle (2013, p. 10344) the five most important (but unexplained) features of consciousness are qualitativeness, ontological subjectivity, unity, intentionality, and intentional causation. The current theory provides explanations for these as, respectively, different ways of modifying the form of fest (producing qualia as mentioned above), the internal and ontological (really existing) nature of fest and its dialogic modifiability, the fact that there is only one fest (see above), the intentional nature of fest (as being about ftrue, see Section 3.4), and the involvement of fest in causation (active causation, see Section 2.1 and in particular the elaborated forms of active causation in Fig. 3e, f). The approach taken here may perhaps seem too mechanistic to readers taking their perspective from the humanities. Although indeed mechanisms have been indicated or conjectured wherever possible, the term ‘mechanism’ is used here in the broadest possible sense as referring to any process or means that can be understood as producing an effect or result, thus including the psychological, social, and cultural processes enabling consciousness and free will. I reserve the term ‘mechanistic’ for conventional systems that explain phenomena primarily in deterministic terms. The causation here has a strongly stochastic flavour, with the modulated stochasticity of Fig. 1c acting continually in the feedback loop of Fig. 1d. The causal structure of the mechanism is therefore non-mechanistic and non-mechanical from the very start. Moreover, the seeds of causal freedom and meaning produced by this mechanism are strongly amplified by the subsequent communicative and social processes as depicted in Figs. 2 and 3. Finally, as stated in the Introduction, the theory does not aspire to replace higher-level approaches, but merely attempts to provide plausible, qualitative explanations for the relationships between the various levels.

2

It is sometimes assumed that neuronal systems are (primarily) deterministic systems. However, in practice neurons and neuronal systems display a considerable level of stochasticity, both at the neuronal and behavioural level (Faisal et al. 2008), and indeed a major effort in neurobiological experiments consists of separating signal from noise.

10

3.1 Determinism, stochasticity, and two-stage models of free will As noted above, the notion of free will developed here is different from the one developed in Dennett (1984), where ‘free’ is basically interpreted as free from external constraints. Internal constraints are viewed there as not restricting freedom, because they are part of the organism itself. In contrast, the current theory provides (some) freedom from both external and internal constraints, because it contains a core mechanism that is significantly stochastic rather than primarily deterministic. As argued in Section 2.1, this produces a form of agency that is free in the sense of not being completely determined. Dennett (1984) argues that randomness (stochasticity) is not the type of freedom worth wanting, because it does not yield control over one's own behaviour, whereas internal deterministic drivers of behaviour do and thus enable responsibility. The current theory gets around this apparent contradiction – that one needs to be determined in order to be free – by viewing behaviour as a process extending over time, in particular as the behavioural trajectories produced by the feedback loop of Fig. 1d and its extensions. In such behavioural trajectories, it is no longer possible to distinguish or separate deterministic and stochastic components. Each factor producing the trajectory – the deterministic fest and the stochastic variation – can only be understood from the complete past of itself and the other factor. This is so because fest drives the stochasticity and the stochasticity drives fest, with changes retained by the organism. Hence it would be meaningless to say that only the deterministic component of a trajectory is important for freedom, or only the stochastic component. Neither component exists, only an entangled entity does. Such an entity cannot function in arguments assuming simpler entities (primarily deterministic or primarily stochastic)3. The A loop of Fig. 1d that produces agency is related to so-called two-stage models of free will, but differs from those in essential points. Two-stage models attempt to solve the problem of combining spontaneity with determined goals by proposing a first stage that generates, randomly, possible behaviours or options, followed by a second stage that makes a choice amongst those options in a fixed, deterministic way. Such models have a long history, dating back at least to William James (reviewed in Doyle 2013; see also the variant proposed by Tse 2013). The problem with such models is that they are too stochastic for free will: the outcome is dominated by the randomly generated options of the first stage – even if the model is looped as mentioned by Doyle (2013), or concatenated as mentioned by Tse (2013). There is no proper way by which the criteria of the second stage can influence the randomness of the first stage, and such models therefore do not lead to agency and intrinsic meaning. The A loop solves this problem by letting fest (a deterministic factor) modulate stochasticity, and by using feedback to accumulate the resulting random behavioural changes into a behavioural trajectory that is simultaneously deterministic and stochastic. In addition, the implicit coupling of fest to ftrue – as imposed by the Darwinian evolutionary mechanism – solves the problem of the likely evolutionary instability of arbitrary goals. On an evolutionary timescale, only high ftrue is stable. 3.2 The neuroscience of free will According to the theory developed here, there are several forms of conscious agency (Fig. 3c-f). Recently, neural correlates of conscious agency have been studied in neuroscience (Libet 1985; Haggard and Eimer 1999; Soon et al. 2008; Fried et al. 2011). This is typically done by letting a subject make a decision and comparing the timing of a neurophysiological signal (EEG, fMRI, or single-neuron recordings) with the timing of the experienced decision as verbally reported by the subject. Experiments typically show that brain activity related to the decision can be detected up to several seconds earlier than the reported moment of decision. For example, in Soong et al. (2008) this 3

For readers who like to think in mathematical terms, the following may help. The transformation proposed here roughly corresponds to the difference between addition and multiplication. If d and s stand for deterministic and stochastic factors, respectively, then d+s could be either primarily deterministic or primarily stochastic, depending on the size of d and s. But in d×s it would be impossible to say which factor is the more (or only) important one. E.g., in 123+5 one might neglect 5, but in 123×5 neglecting 5 would make no sense. The modulated stochastic causation of Fig. 1c is in fact a multiplicative interaction of deterministic and stochastic factors. The final result in Fig. 1d is obviously further complicated by the feedback cycle, the evolutionary drive, the time-varying environment, and the retention of changes in the organism (through its state and learning).

11

earlier brain activity can be used to predict a subject's decision with about 60% success rate, significantly different from the 50% expected in the particular experiment. However, to conclude from these experiments that subjects have no or limited free will, would be strongly overstating the case when viewed from the present perspective (critiques from several other perspectives can be found in Baumeister et al. 2010; see also Schlosser 2014). Free will is a process rather than an instantaneous event, an argument that has been used before against the interpretation that Libet-type experiments are incompatible with free will (e.g., Dennett 2003, chapter 8; Gallagher 2006). One factor is the time required for the internal simulations presumably required for symbolic reasoning. But another factor that is argued here to be crucial and inevitable is that agency must accumulate from continual cycling through the A loop of Fig. 1d and its elaborations. Agency of any kind – symbolic or not – only gradually builds up. Many cycles are typically required to produce a behavioural trajectory that is, statistically, strongly dependent on fest (van Hateren 2014a). A complication here is that animals with sophisticated nervous systems may simulate a series of behavioural trajectories first before actually producing a behaviour (see Section 2.5). Conscious, but non-symbolic agency, such as in Fig. 3c, d, could be fairly fast, because behavioural trajectories could be simulated simultaneously (in parallel) in the nervous system. But the symbolic forms of agency (Fig. 3e, f) are necessarily slow, because they are produced by utilizing components that are internalized versions of the slow dialogue between individuals (right-most diagram in Fig. 2c). Such dialogues depend on physical behaviour (such as gestures and speech) that is serial and several orders of magnitude slower than neural activity (typically seconds rather than milliseconds). Even if no gestures or speech are produced, as in symbolic thought, the process still remains serial. It continues to depend on the neural processes required for accessing language memory, assembling symbolic constructs, and so on. These processes evolved primarily in the context of spoken language, with corresponding speed and timing. A further consequence of using these specialized processes in the A loop is that trajectories can only be simulated (as in ruminations) consecutively, not simultaneously. The A loop (symbolized by the arrows between fest and the individual in Fig. 3) therefore becomes particularly slow when it works by varying these slow forms of reasoning (the loop involving fest and X in Fig. 3e, f). Free will as associated with rational agency (Fig. 3f) then necessarily takes many seconds to accumulate significance (and for many decisions orders of magnitude longer, see e.g. Baumeister et al. 2011). Clearly, neurophysiological experiments on free will cannot easily address the long timescales involved in rational and symbolic reasoning. Typically, they will tap into the faster forms of agency of Fig. 3c, d, which are also consciously perceived and therefore reportable. Because the parameters used by those forms of agency are likely to be at least partly determined (prepared) by the results of earlier rational deliberations, they are a kind of frozen free will. It is therefore not surprising that subjects will generally affirm that decisions executed through the mechanisms of Fig. 3c, d are made freely (i.e., are experienced as willed), although that may be true only indirectly at that very moment, strictly speaking. 3.3 Responsibility and agent causation Kane (1996) defines free will as the power of an agent to be the ultimate creator and sustainer of its own ends and purposes. The scheme of Fig. 3f approaches that definition: by rational reasoning, the individual can modify the form of fest, in other words, the individual can modify the way the world and the individual's interactions with the world are valued by the individual. However, the resulting freedom and responsibility of the individual is clearly not absolute. Not only is rational agency embedded in the other forms of agency of Fig. 3, it is also tied to how the individual's culture defines rationality (through the interaction of X with culture in Fig. 3f). The intrinsic values of the individual, as embodied in the form of fest, originate from three major, entangled factors: nature (as determined by evolution), nurture (through environment, including social and cultural factors), and the individual itself (through several of the mechanisms of Fig. 3, which include the self-forming actions of Kane 1996). In the history of the free will debate, two forms of causation are often distinguished, event causation and agent causation. Event causation is the default, well-known causation of the natural sciences, where, in its basic form, one event causes another. Because event causation seems difficult to 12

reconcile with free will, another form of causation has been postulated, where not events, but the agent itself is the cause of further events. However, agent causation has always been viewed with suspicion from a naturalistic point of view, because it smacks of supernaturalism and dualism (the view that mind is a substance different from and independent of matter). The problem with both event and agent causation is that neither events nor causation are instantaneous. For example, when a billiard ball rolls and collides with a second, steady ball, we might say that the collision is an event that causes the second ball to start rolling. But that makes it sound as if the event and the causation take place at one point in time. Of course, that is not what happens. The balls will only gradually touch each other, with the atoms of the balls smoothly interacting more and more strongly. The balls will deform a little, and the transfer of momentum of one ball to the other takes a little time. In other words, both concepts ‘event’ and ‘causation’ are idealizations that depend on choosing a timescale much longer than the specific micro-physics involved. Consequently, if one assumes that a decision is made at one point in time, it forces the conclusion that only event causation and not agent causation makes sense. However, if a decision is, more realistically, considered to be a process that takes time, such as required by going many times through the A loop of Fig. 1d, the view becomes different. Then the behavioural or simulated behavioural trajectory, taken as a whole, is the cause, and the agent is producing the causation (through its agency and goals) equally much as the stochastic micro-events are. Thus at least part of this causation could be properly designated by a term like ‘agent causation’. 3.4 Intentionality Intentionality is used here in the technical sense as the property of being about something or referring to something (‘aboutness’). It is thus more general than having to do with intentions. For example, speech has intentionality, because words and language are usually understood by the communicating partners to be about an object, an idea, or anything else they want to talk about. In its weakest interpretation as mere aboutness, intentionality is already present in Fig. 1d. The mechanism works because the estimated fitness fest refers to, is about, the true fitness ftrue. It could only evolve because it increases fitness when fest represents ftrue in a sufficiently truthful way. In effect, the organism implicitly utilizes fest as a representation of ftrue. The way objects are consciously experienced, as proposed in Fig. 2b, has a more elaborate intentional aspect because of the internalized dialogue through which this is accomplished. Full symbolic intentionality, i.e. intentionality based on the aboutness of symbols, arises in the iconic and symbolic forms of communication illustrated in Fig. 2c. Symbols used by the communicating partners are about O or X (e.g. Tomasello and Carpenter 2007). Similarly, symbols used in thought (final diagram of Fig. 2c) are intentional, as are the symbols used in symbolic and rational agency (Fig. 3e, f). The current theory implies that meaning, consciousness, and symbolic intentionality originated at quite different times in evolution. Intrinsic meaning is presumably very old, having evolved at or close to the origin of life (van Hateren 2013). However, it is a form of meaning that remains implicit in the organism, and the intentionality it implies is therefore quite different from that of the more narrowly defined explicit meaning of human language. Consciousness arose, in its basic form, more recently, perhaps one or a few hundred million years ago, when animals evolved complex nervous systems and sufficiently social lifestyles to enable formative communication. Only quite recently, perhaps a few million years ago in protolinguistic hominins, communication began to evolve into the advanced forms of Fig. 2c that enable full symbolic intentionality. The meaning embedded in such communication is explicit and highly flexible, although it still depends, ultimately, on the intrinsic meaning as embodied in the fest of the communicating individuals. 3.5 Relationship with phenomenologically inspired approaches The theory presented here converges with several of the key points made by phenomenologically inspired approaches to cognition (e.g., Thompson 2007), but reaches these conclusions via a different path starting from a different position. In phenomenological approaches, meaning (or, more technically, normativity) is generally derived from the autonomy of the organism. The fact that the organism maintains its integrity against external disturbances is seen as normative. For example, 13

Thompson (2007, p. 70) speaks of “the norm of the maintenance of the organism's integrity”, and similar ideas can be found elsewhere4. In autopoietic theory (Varela et al. 1974; Thompson 2007) the normativity is viewed as the consequence of the self-generated functional constraints that enable living organisms to maintain themselves. Although I share the conclusion about the genuine normativity of all life, I think this way of deriving meaning can only work when an additional assumption is made. As also argued by Davies (2009, pp. 86-87), consistently acting as though following a normative goal (here maintaining the self through autopoiesis) does not imply really having such a goal. There are other processes in nature that seem to follow goals, such as water in a river heading for the ocean, or molecules of a gas tending to spread out through a volume. But we know that the end-points of such processes are not genuine goals, essentially because we know how to explain these processes in purely mechanistic terms5. An autopoietic system that is purely mechanistic (i.e., essentially deterministic) has the same problem. Assuming that autopoietic systems as such really establish self-maintenance as normative (i.e., that such systems strive for self-maintenance) depends on the additional, tacit assumption that existing is better than not existing. Such an assumption needs justification: existing or not existing are neutral in the world of physics and chemistry, and explaining why life would place more value on living than on dying requires a separate argument. In this article this separate argument is provided by the special, non-mechanistic scheme6 of Fig. 1d that establishes high fest as a genuine goal. Such a goal implies striving for self-maintenance, because fest is coupled to ftrue. One might argue that perhaps an extension of self-maintenance could be used as the starting point to derive meaning and agency, similarly as done for fitness in the active A loop of Fig. 1d. Indeed, Di Paolo (2005) has introduced an active component by arguing that agency arises when autopoiesis is extended with adaptivity, the ability of organisms to regulate themselves such that they remain viable even under variable environmental conditions. However, the term ‘active’ is used in a more restrictive way here than in the approaches of Di Paolo and, e.g., Kauffman (2000). As used here, it not only requires self-directed behaviour (as in acting on one's own behalf and maintaining oneself), but also some behavioural freedom. Thus a system that is fully driven in a deterministic or random way is not really active – although it may appear to be so, as a sophisticated robot might. It is proposed here that only the fitness-driven entanglement of deterministic and stochastic causation in a feedback loop as in Fig. 1d can produce behavioural freedom, i.e., genuine action. The scheme of Fig. 1d only works well because of the strong drive produced by the combined self-maintaining and reproductive capabilities of life. Self-maintenance (survival) alone would not be powerful enough, because it represents only half of this drive. The driving force of life is its capability of exponential growth in numbers (i.e., high fitness), because without such a capability, environmental fluctuations would inevitably drive all forms of life to extinction, eventually. Such fluctuations are too strong and unpredictable for any fixed, non-evolving organism to survive. Only the fact that the few successful variants at any point in time could quickly multiply made it possible for life to sustain itself on Earth for nearly 4 billion years. High fitness requires not only survival, but also reproduction. If survival (self-maintenance) is too weak, organisms die before reproducing. If reproduction is too weak, organisms will be the last of their kin. Only fitness, and the basic Darwinian scheme, could produce the strong drive required for evolving an internalization of fitness (as fest). A strong drive is required, because internalized fitness is in fact a fairly weak second-order property functioning through second-order statistics (variance), piggybacking on the basic, first-order Darwinian scheme 4

For example in the work of Merleau-Ponty (discussed in de Preester 2003, pp. 201-202), in the work of Damasio with its emphasis on homeostasis as implicitly valuable to the organism, e.g. “the extension and transfer of homeostatic goals to objects and situations that become imbued with biological value” (Damasio and Carvalho 2013, p. 145), and in the work of Kauffman with its emphasis on the autonomy of life, acting on its own behalf, implying “Once there is an autonomous agent, there is a semantics from its privileged point of view” (Kauffman 2000, p. 111). 5 Noting that molecules act ‘as if’ they prefer to be spread out through a volume rather than remain concentrated in a corner of this volume would not add explanatory power beyond the mechanistic explanation. The mechanisms work purely ‘on the spot’, without being forward looking in any sense. 6 In that scheme the modulation of stochasticity can only be understood from its expected future benefits, not from its immediate mechanistic consequences (which are at a particular point in time not distinguishable from those of a non-modulated stochasticity). Thus the mechanism itself is, in some sense, forward looking, although it has evolved through regular, backward looking evolutionary mechanisms (Author 2014a).

14

directly depending on ftrue. This internalization produced the elementary form of agency and the intrinsic meaning as explained above. Nevertheless, even if the actual mechanism of obtaining meaning as presented here is different from the one postulated in a phenomenologically inspired approach such as autopoietic theory, what is common is that such meaning is taken to be real and present, presumably right from the origin of life. The current approach can in fact be used to bootstrap the meaning and agency used in other approaches. Once the mechanisms presented here are in place, it follows that existing is indeed better than not existing, from the point of view of the organism. Consequently, autopoiesis is then indeed to be seen as genuinely meaningful, homeostasis and autonomy indeed as representing biological value, and enhanced adaptivity indeed as increasing agency. The present approach in fact aligns well with several concepts advocated by current approaches taking their inspiration from phenomenology. In particular, the view that life and mind are enacted, embodied, embedded, and situated is consistent with the strong dependence of an organism's fitness on its environment and how that is engaged. As argued here, fest is an intrinsic drive of living organisms that is crucial for explaining life's agency and the emergence of mind. This implies that organisms enact their world, because the agency arising from the fest-stochasticity cycle will affect ftrue and thereby fest. Agency therefore affects meaning, and meaning is internally generated. It also implies that the mind is shaped by and even requires embodiedness, because without a body there would be no ftrue and thus no fest, or at least not a genuine and stable fest. It finally implies that the way organisms are embedded and situated in the specific context of a situation is important, because such situatedness either influences ftrue and thereby fest, or influences aspects of fest anyway, independent of whether these correspond to ftrue or not. 4 Conclusion In this article free will is constructed in a series of steps. Life produces active causation through a feedback loop where behavioural variability is driven by a self-estimated fitness fest, itself driven by the true fitness ftrue on an evolutionary timescale (Fig. 1d). By internalizing ftrue as fest, the organism transforms the ‘as if’ goal-directedness towards high ftrue into the genuine goal-directedness towards high fest. The elementary agency provided by active causation and the genuine goal-directedness of high fest produce intrinsic meaning in the organism, embodied in the form of fest. When animals first developed ways to modify each other's intrinsic meaning through formative communication, consciousness emerged. Elementary forms of consciousness, such as provided by the mammalian mother-infant bond, subsequently facilitated the formation and internalization of more complex forms of consciousness, such as of objects (Fig. 2b). Such complex forms eventually led (Fig. 2c) to the formation of iconic and symbolic systems of dialogue about objects and abstractions (language) and internalized forms of symbolic dialogue (thought). Agency can occur in various forms (Fig. 3), where the most complex form, rational agency (Fig. 3f), can be identified with free will as understood here. The present theory proposes quite specific mechanisms for the origin of agency, meaning, mind, and rational agency, all stated in the language of the natural sciences. It is therefore, at least potentially, amenable to empirical tests based on methods derived from those sciences. It also could be used to identify topics that are likely too complicated to yield to detailed analysis of components and that would be more properly investigated, then, at a phenomenological level. In particular, this occurs for topics directly involving fest, such as the subjective experience, free will, sense of meaning and purpose, and how conscious agency interacts with social and cultural processes. Finally, the theory makes it fairly straightforward to demarcate the parts of reality where meaning, mind, and rationality can be present from the parts where those phenomena are absent.

Acknowledgements I would like to thank the anonymous reviewers of the manuscript for thoughtful comments and suggestions.

15

References Anderson, P. W. (1972). More is different. Science, 177, 393-396. Baumeister, R. F., Masicampo, E. J., & Vohs, K. D. (2011). Do conscious thoughts cause behavior? Annual Review of Psychology, 62, 331-361. Baumeister, R. F. , Mele, A. R., & Vohs, K. D. (2010). Free will and consciousness: How might they work? New York: Oxford University Press. Brembs, B. (2011). Towards a scientific concept of free will as a biological trait: spontaneous actions and decision-making in invertebrates. Proceedings of the Royal Society B, 278, 930-939. Carruthers, P. (2005). Natural theories of consciousness. In P. Carruthers, Consciousness: essays from a higher-order perspective (pp. 36-60). Oxford: Clarendon Press. Chandler, D. (2007). Semiotics: the basics. Oxon: Routledge. Damasio, A., & Carvalho, G. B. (2013). The nature of feelings: evolutionary and neurobiological origins. Nature Reviews Neuroscience, 14, 143-152. Davies, P. S. (2009). Subjects of the world: Darwin's rhetoric and the study of agency in nature. Chicago: University of Chicago Press. Dehaene, S., & Changeux, J. P. (2000). Reward-dependent learning in neuronal networks for planning and decision making. Progress in Brain Research, 126, 217-229. Dennett, D. C. (1984). Elbow room: The varieties of free will worth wanting. Oxford: Clarendon Press. Dennett, D. C. (1996). Kinds of minds: Toward an understanding of consciousness. New York: Basic Books. Dennett, D. C. (2003). Freedom evolves. New York: Penguin Books. De Preester, H. (2003). Meaning: what's the matter? Intrinsic relations and the organism. Theoria et Historia Scientiarum, 7, 195-205. Di Paolo, E. A. (2005). Autopoiesis, adaptivity, teleology, agency. Phenomenology and the Cognitive Sciences, 4, 429–452. Doyle, R. A. (2013). The two-stage model to the problem of free will. In A. Suarez & P. Adams (Eds.), Is science compatible with free will?: Exploring free will and consciousness in the light of quantum physics and neuroscience (Chap. 16). New York: Springer. Faisal, A. A., Selen, L. P. J., & Wolpert, D. M. (2008). Noise in the nervous system. Nature Reviews Neuroscience, 9, 292-303. Fried, I., Mukamel, R., & Kreiman, G. (2011). Internally generated preactivation of single neurons in human medial frontal cortex predicts volition. Neuron, 69, 548-562. Galhardo, R. S., Hastings, P. J., & Rosenberg, S. M. (2007). Mutation as a stress response and the regulation of evolvability. Critical Reviews of Biochemistry and Molecular Biology, 42, 399-435.

16

Gallagher, S. (2006). Where's the action? Epiphenomenalism and the problem of free will. In S. Pockett, W. Banks, & S. Gallagher (Eds.), Does consciousness cause behavior? An investigation of the nature of volition (pp. 109-124). Cambridge, Mass.: MIT Press. Haggard, P., & Eimer, M. (1999). On the relation between brain potentials and the awareness of voluntary movements. Experimental Brain Research, 126, 128–133. Heisenberg, M. (2009). Is free will an illusion? Nature, 459, 164-165. Kane, R. (1996). The significance of free will. Oxford: Oxford University Press. Kauffman, S. A. (2000). Investigations. New York: Oxford University Press. Libet, B. (1985). Unconscious cerebral initiative and the role of conscious will in voluntary action. The Behavioral and Brain Sciences, 8, 529–566. Mead, G. H. (1934). Mind, self, and society. Chicago: University of Chicago Press. Ram, Y., & Hadany, L. (2012). The evolution of stress-induced hypermutation in asexual populations. Evolution, 66, 2315-2328. Reddy, V. (2003). On being the object of attention: implications for self–other consciousness. Trends in Cognitive Sciences, 7, 397-402. Schlosser, M. E. (2014). The neuroscientific study of free will: A diagnosis of the controversy. Synthese, 191, 245-262. Searle, J. (2007). Free will as a problem in neurobiology. In J. Searle, Freedom and neurobiology: Reflections on free will, language, and political power (pp. 37-78). New York: Columbia University Press. Searle, J. (2013). Theory of mind and Darwin's legacy. Proceedings of the National Academy of Sciences of the U.S.A., 110(suppl. 2), 10343-10348. Soon, C. S., Brass, M., Heinze, H. J., & Haynes, J. D. (2008). Unconscious determinants of free decisions in the human brain. Nature Neuroscience, 11, 543–545. Thompson, E. (2007). Mind in life: Biology, phenomenology, and the sciences of mind. Cambridge Mass.: Belknap Press. Tomasello, M., & Carpenter, M. (2007). Shared intentionality. Developmental Science, 10, 121-125. Trevarthen, C., & Aitken, K. J. (2001). Infant intersubjectivity: Research, theory, and clinical applications. Journal of Child Psychology and Psychiatry, 42, 3-48. Tse, P. E. (2013). The neural basis of free will: Criterial causation. Cambridge Mass.: MIT Press. van Hateren, J. H. (2013). A new criterion for demarcating life from non-life. Origins of Life and Evolution of Biospheres, 43, 491-500. van Hateren, J. H. (2014a). Active causation and the origin of meaning. Biological Cybernetics, doi:10.1007/s00422-014-0622-6 van Hateren, J. H. (2014b). Consciousness results when communication modifies the form of selfestimated fitness. arXiv:1401.6761 [q-bio.NC] http://arxiv.org/abs/1401.6761 17

Varela, F. G., Maturana, H. R., & Uribe, R. (1974). Autopoiesis: the organization of living systems, its characterization and a model. BioSystems, 5, 187-196. Vygotsky, L. (1986). Thought and language. Translated from the original 1934 Russian edition. Cambridge Mass.: MIT Press. Weir, A. A. S., Chappell, J., & Kacelnik, A. (2002). Shaping of hooks in New Caledonian crows. Science, 297, 981.

18