PENICILLIN FOR THE M IND? REASON , E DUCATION AND C OGNITIVE S CIENCE Tim van Gelder April 1998

We face a crisis of reason. Average levels of reasoning ability are dismal; powerful forces are working against any improvement; and current educational strategies are failing to correct the problem. All this is happening at a time when reasoning skills are becoming more important than ever before. In the pages that follow I will add substance to these claims. More significantly, I will argue for a particular response. Dealing with the crisis will require educational innovation. Useful It is difficult to innovations will be obtained only through the imagine a more application of insights from cognitive science as they important educational bear on reasoning and learning. Given the economic and objective than the teaching and learning of technological conditions of the near future, we must how to think more incorporate these insights into information technology. effectively than we typically do. Indeed, if In short, insofar as we take the crisis of reason we cannot learn to think seriously, we must use cognitive science in the more rationally and development of computer tools for aiding the effectively, we are, as a species, acquisition of reasoning skills. 1. REASON —WHAT IT IS, AND WHY IT IS VALUABLE

in serious trouble. - Nickerson et al.

To reason is to negotiate the intricate webs of inferential dependence among sentences of a language. It involves activities such as determining whether a claim supports or conflicts with others, drawing out implications, supporting opinions with arguments, and raising or rebutting objections. Over the centuries, reason has been elaborated in some very sophisticated ways. In domains such as statistics and formal logic the rules of good inference have been articulated with great precision. In science and mathematics, where these rules are taken quite seriously, vast edifices of knowledge have been constructed. However, my concern here is not with these kinds of specialised manifestations. Rather, it is with reason as a practical tool in everyday affairs: the norms and habits of thought regulating the beliefs and conduct of ordinary people in the bulk of their personal and working lives. Reason in this sense is what philosophers often call “informal logic” or “argumentation”. It is the kind of thinking we

— 1—

all use—more or less well—when engaging in activities such as spending our incomes, resolving disputes, drawing up policies, or making career choices. Muddleheadedness has always been the sovereign force in human affairs—a force more potent than malevolence or nobility. It lubricates our hurtful impulses and ties our best intentions in knots. It blunts our wisdom, misdirects our compassion, clouds whatever insights into the human condition we manage to acquire. It is the chief artisan of the unintended consequences that constitute human history. - Paul Gross & Norman Levitt

The principal advantage of reason is that it helps clear the path to truth; while the advantage of truth is that it is the foundation of well-being. Poor reasoning diminishes our capacity to satisfy our desires, from everyday needs and fancies through to our grandest ambitions. That reason should play this role is hardly surprising, for reason just is those habits of thought which have been found most effective in helping us improve our beliefs and achieve our ends.

However, reason is more than a useful tool. It is essentially connected with certain things we find intrinsically valuable. One is a robust notion of citizenship. A majority of the world’s population today lives in societies which are, or are in transition to, some form of liberal democratic capitalism. For individuals to be genuine members of these societies, they must be able to make reasoned choices about their ways of living, their leaders, and their purchases and investments. Those without the power of autonomous rational choice may dwell in a liberal democratic society, but they participate in it only the way a dry leaf participates in a flash flood. The philosophical stakes can be raised higher still. Many great thinkers, from Plato to Mill, have esteemed reason as the highest part of the soul. Our capacity and tendency to reason is what distinguishes us most decisively from everything else. Clearly, other animals—and, increasingly, computers—share many of our most impressive and endearing characteristics. Yet only humans are able to adjudicate the extent to which one sentence of natural language is confirmed or confuted by another. Blaise Pascal once claimed that “all our dignity lies in thought.” This overstates the case, but it captures the idea: fully realising one’s distinctively human nature presupposes a certain adroitness in the art of reason. 2. THE DISMAL STATE OF REASON Reason is a skill, one that develops slowly over many years and is exceedingly difficult to master. Everyone is born with the germ of reason, but rarely is it allowed to fully mature. The implications for human well-being are often comic, and too often tragic. One of my favourite examples is the medieval doctrine of armarium uguentum,

— 2—

the theory that a wound should be treated by administering aid to the weapon that caused it: If the wound is large, the weapon with which the patient has been wounded should be anointed daily; otherwise, every two or three days. The weapon should be kept in pure linen and a warm place but not too hot, nor squalid, lest the patient suffer harm. 1 Presumably, this doctrine survived because some lucky patients did in fact recover while the weapons were being treated. Unfortunately for the others, the principles of good causal inference had not been discovered at that time. Anecdotal impressions are underwritten by serious scientific evidence. In recent years systematic studies have confirmed and elaborated the verdict of (un)common sense. One of the most thorough studies of reasoning skills was that reported by Deanna Kuhn in her 1991 book The Skills of Argument. One hundred and sixty people of various ages We consider ourselves distinguished from the and education levels were intensively ape by the power of interviewed to ascertain their ability to reason thought. We do not with regard to issues such as unemployment and remember that it is like the power of walking in crime. She discovered that while most people the one-year-old. We readily adopt strong convictions, a majority think, it is true, but we think so badly that I cannot generate any genuine evidence at all for often feel it would be those convictions. Just as dismal were their better if we did not. abilities to countenance alternative opinions, - Bertrand Russell generate and rebut counterarguments, and assess the bearing of some piece of evidence on their beliefs. This poor performance cannot be attributed to the difficulty or unfamiliarity of the topics. Kuhn found that, if anything, domain-expertise obstructed good reasoning.2 Note also that these deficiencies were uncovered in highly structured interviews where subjects were explicitly asked to produce evidence, rebuttals, etc.. How much worse their spontaneous thinking must be! Kuhn’s results are very much in line with those of related research, of which there is a great deal, often covering much larger samples. Some of the most comprehensive datasets have been gathered by the US Department of Education’s National Assessment of Educational Progress (NAEP). Every four years this program assesses the cognitive skills of thousands of schoolchildren at three grade levels. Some insight into average levels of reasoning ability can be gained by studying the results of the mathematics, science and reading tests. As one expert summarised it, 20 years of NAEP testing “has documented a critical shortage of reasoning skills among our young people.”3 For example, an analytic writing test measures “the

— 3—

ability to provide evidence, reason logically, and make a welldeveloped point.” Fewer than one in a hundred students are able to achieve the “elaborated” level (roughly, what one would hope to see in a college freshman).4 These dry statistics are fleshed out by insights from the laboratory. In careful experiments, psychologists have probed human judgment from many angles. The news is not good. Our minds are naturally prone to a wide range of systematic errors, biases, and illusions.5 Some of these can be corrected with instruction and practice, but others seem to be inbuilt “features” of our cognitive architecture, just as the blind spot on the retina is intrinsic to the design of the human eye.6 Most people go about reasoning quite oblivious to the ways their cognitive deck is stacked against them. It is little wonder they so often end up performing so badly. Can things really be so dismal? How then do we manage to sustain such an advanced civilisation? Fortunately, many people do in fact reason tolerably well. These people are disproportionately responsible for our collective achievements. We have slowly managed to organise our affairs so that the others can at least get by with little skill in argumentation. And is the cup half full or half empty? How much better would life have been, if only people were in general better able to follow the dictates of reason? The low average levels of reasoning ability would be less disturbing if, like average incomes or nutrition levels, they were gradually improving. Unfortunately, the opposite appears to be the case. The broad statistical evidence on this question is mixed. Some relevant measures of intellectual aptitude are falling, but some others—such as average IQ—are showing slight gains. Still, other kinds of evidence contribute to an overall picture of decline. Indeed, complaints that we are “dumbing down” now form a kind of continual background noise in our cultural conversation.7 Educators, in particular, are often heard wailing for lost reasoning skills. Detailed evidence of the “dumbing down” of higher education is catalogued in a report from the U.S.-based National Association of Scholars, “The Dissolution of General Education: 1914-1993.” Its message is that even the most prestigious institutions have substantially reduced their demands in terms of both content and rigour; even the length of the academic year has been shrinking.

— 4—

A dramatic way to reinforce this image of decline is to observe the degeneration of public political discourse over the last century. Neil Postman has made this point with his descriptions of the famous debates between Abraham Lincoln and Stephen Douglas.8 From today’s perspective, these were intellectual feats of extraordinary proportions. Their encounter in Peoria on October When a population 16, 1854, commenced with a three-hour address by becomes distracted by trivia, when cultural life Douglas; the meeting adjourned for a meal break, is redefined as a followed by four more hours of dialectic. The point perpetual round of of interest here is not the mental powers of Lincoln entertainments, when serious public and Douglas. The point, rather, is the mental conversation becomes a powers of the audience. Its members were form of baby-talk, when, in short, a people ordinary educated folk, attending as much for become an audience entertainment as for edification. They had attention and their public spans, tolerance for linguistic complexity, knowledge business a vaudeville act, then a nation finds of political affairs, and capacity to follow extended itself at risk; culturechains of argument, almost unimaginable in any death is a clear possibility. comparable audience today. - Neil Postman

3. CRISIS ? W HAT CRISIS ? For the dismal state of reasoning skills to be counted a crisis, there must be something heightening the level of urgency. Two major factors contribute this urgency. The first is the changing distribution of employment and wealth in the new global economy, placing greater emphasis on intellectual skills. The second is the growing strength of forces working against the cultivation of reason. In short, even as reasoning skills become more vital, they are becoming harder to cultivate and sustain. The trouble with humans is that they are expensive. Over the centuries, a great deal of technological innovation has been driven by the desire to reduce the number of people required to carry out a given task. The result has been increasingly rapid change in the nature of employment. Roughly, we only employ people to do what machines still cannot. The jobs remaining—and the new ones opening up—are those demanding talents not yet possessed by machines. Foremost among them are intellectual skills, including the ability to reason. This has profound implications for individuals seeking good jobs and decent incomes. Technology is gradually increasing the level of intellectual proficiency required in the workplace. If average levels in the workforce stay fixed, there will be greater numbers of people competing for fewer jobs at the lower end, and vice versa. The result: falling wages and unemployment for ever larger numbers of people, while wages escalate for those fortunate enough to be still in the game.

— 5—

This is one of the major economic trends currently altering the distribution of wealth and well-being.9 In an era of economic globalization, this trend dictates that for a country to maintain or increase its standard of living, it must raise average levels of intellectual skills. This is not a new idea; indeed, it was a dominant theme in A Nation at Risk10 some fifteen years ago. In the meantime, the claim that a country’s standing in the global hierarchy of wealth is increasingly determined by thinking skills has become a mantra for pundits, politicians, and university presidents. Global competitiveness is serious business. However, there is a more profound basis for concern than anxiety over relative percapita incomes. The decline of intellectual skills such as reasoning is an issue of social justice, regardless of local or national borders. Increasingly, society is dividing into the haves and the have-nots, depending largely on ability to participate in the new global economy. This polarisation is a matter of income in the first instance, but it also has implications for quality of life and for such intangibles as human dignity. It is a new kind of class division, replacing the old opposition between workers and capitalists with a new one between highly skilled, highly paid participants and the underemployed and underpaid underclass. In the worst case, this division can lead to significant social unrest.11 In Australia, for example, we now have a new national political party serving as the voice of those who realise that the global economy is leaving them behind. Disturbingly—though not surprisingly—these people’s legitimate economic concerns have become distorted by elements of fascism, racism and isolationism. The proper response to these unwelcome developments is to attempt to ensure that everyone has the opportunity to participate in the new economy, and to share in its benefits. Unfortunately, there are powerful and growing forces currently working against any such goal. Again, the major villain here is technology. Even as it transforms the economy, and increases the level of intellectual proficiency demanded in the workplace, technology is making it hard to gain and sustain that proficiency. There is a very general mechanism at work here. New technology is commonly devoted to making our lives easier in one respect or another. Very often it does this by reducing the mental effort required to engage in some activity. Consequently, people get less practice in thinking, and end up less able to do it. Probably the most notorious manifestation of this process is television, which has taken the technology of effort-free entertainment to a high level of perfection. The evils visited by television upon our intellectual health have been rehearsed in many places.12 Here, I will

— 6—

illustrate the general point with a less-noticed, but equally pernicious phenomenon—the photocopier. My mother was a philosophy major at the University of Melbourne some 25 years before I arrived to do the same degree. In those days, there was often only one copy of a book or article, and it was in the library. She went there to read it, and anything she took away had to be in her head or in her laboriously handwritten notes. This forced her to distil the text down to its essentials; and she did this time and time again. Today, almost nobody does it; if something is worth reading, it is worth copying first. In the library at the University of New South Wales students make 15 million copies a year (cost: 400 large trees). Photocopying fosters the illusion that knowledge has been acquired. The mental effort of comprehending and remembering has been postponed, often forever. These days teachers arrange to have class materials copied in advance, to save students the trouble. As a result, students get much less practice digesting a text. They are losing the disposition and ability to do such basic things as identify the main point and the main line of argument. They cannot form a critical response because they never engage with the reasoning in the first place. What other forces are operating to reduce average levels of reason? We should not, I think, underestimate the harmful impact of some contemporary antirationalist ideologies. Indeed, in the past few decades Reason has been under an assault comparable in scale to the nineteenth-century Romantic reaction to Enlightenment ideals. This assault has two major prongs. The first is more superficial and yet may be more consequential. In the world of business management, no idea is too flaky to serve as a hook on which to hang a few bestsellers. The idea that an excess of logic is bad for the bottom line is no exception. Usually these breathless screeds do not explicitly promote the wholesale rejection of rationality. Rather, by shifting all the focus to the alleged virtues of something else entirely, they create a climate in which good old-fashioned reason is derided and neglected. Consider for example the case of Tom Peters, perhaps the most famous management “guru” of them all. Peters followed his blockbuster In Search of Excellence with such unforgettable titles as Thriving on Chaos and The Pursuit of Wow. His main message seems to be that, in a changing world, rational business planning is liable to come unstuck, so try being crazy instead. Peters has long been a fan of what he calls “the technology of foolishness;” he now recommends “the sublime pleasures of modestly organised anarchy.” This would all be merely amusing except that Peters’ books stay on the bestseller lists for years (literally) and untold numbers of managers have been

— 7—

through his “excellence seminars.” One shudders to think of the damage done to perfectly good corporations. However, for audacious antirationalism, even Tom Peters pales beside Edward de Bono. De Bono describes himself as the wealthiest intellectual in history, and he might well be right. De Bono’s license to print money is the idea that, in the modern world, “vertical thinking” (viz., traditional logical thinking) is passe; what you need is “lateral” or creative thinking. He then Imagination, deserted provides some handy methods for enhancing your by reason, begets 13 laterality quotient. Of course, nobody is impossible monsters. opposed to creativity. De Bono is dangerous United with reason, she is the mother of all arts, because he leads people to believe that their and the source of their critical thinking skills are perfectly well-developed wonders. - Francisco Goya as they are, but need to be set aside. The irony is that if you ask the man in the street to freeassociate on “thinking” he’ll say “de Bono.” Through his books, seminars, and consulting, de Bono has probably had more actual impact on how people think, and what they think about thinking, than all the formal logic classes ever taught. The other prong of the ideological assault on Reason is the motley crew of “postmodernists” and others subscribing to the idea that there is something suspicious about the very idea of rationality. They combine various strands of woolly thought to arrive at the view that Reason is optional and even oppressive. One strand is a crude relativism: different cultures, races, and genders have—they claim— different “ways of knowing,” and there is no objective standpoint from which to choose between them. Second, there is the idea that a deep semantic indeterminacy infects all language, and so the implications of any claim are always up for grabs. Third, it is claimed that all forms of discourse are thoroughly and essentially expressions of power; Reason itself is nothing but a convenient way for one group to effect its social dominance. These are the barbarians within the gates. They thrive best within the university, where large territories have already been colonised. Their intellectual shortcomings have been explicated and illustrated in many places and many ways,14 and need not be recounted here. Two points should be noted, however. First, it is patent that in postmodernist discourse relevant standards of clarity, logic, relevance, etc., have been abandoned; and this is taking place within our centers of higher education. We thus have yet another depressing illustration of dismal state of reasoning skills. The second point is more disturbing. Postmodernism exercises a sinister influence on the development of reasoning skills in the population at large. This is because postmodernist ideas have a way

— 8—

of trickling down. It is an unfortunate accident of history that postmodernism has taken strongest hold in Departments of English (as they used to be called). These departments traditionally held primary responsibility for teaching people how to write a decent essay. Until recently, criteria of good writing included making a clear point and sustaining a well-organised argument. In other words, English departments used to teach critical writing and hence critical thinking. But postmodernism throws these values into question. Insofar as postmodernism has crept downwards into the writing classes at colleges and even schools, reasoning has been neglected, if not overtly discouraged. Geraldine de Luca, director of freshman English at Brooklyn College, has confessed that “even the concept of error is starting to seem repugnant to me.”15 Under conditions like this, what hope is there that students will learn how to write a wellreasoned essay? 4. THE FAILURE OF EDUCATION Insofar as we take the crisis of reason seriously, we will want to do something about it. But what? The obvious suggestion is that we need more or better education. Within schools and universities there are basically two ways of teaching reasoning skills. The direct method takes the bull by the horns, focusing directly on basic principles, skills and dispositions of good reasoning. The indirect method is to hope that in learning mathematics, biology, history, etc., students will somehow manage to pick up the art of argumentation. Overwhelmingly, the method we actually use is the indirect one. Only rarely do students receive any explicit instruction and practice reasoning itself. But does the indirect method work? H.L. Mencken, for one, thought not: Certainly everyday observation shows that the average college course produces no visible augmentation in the intellectual equipment and capacity of the student. Not long ago, in fact, an actual demonstration in Pennsylvania demonstrated that students often regress so much during their four years that the average senior is less intelligent, by all known tests, than the average freshman.16 More recent evidence suggests that the indirect method does actually have a slight positive effect. For example, Ernest Pascarella has studied the general effect of higher education on the development of critical thinking skills. He found that in the first year average levels of critical thinking improve almost half a standard deviation (over and above improvements in a control group).17

— 9—

Nevertheless, one thing is clear: the current Seldom has there been combination of direct and indirect instruction is such widespread failing, in the face of contrary pressures, to deliver agreement about a significant social issue adequate rates of improvement in reasoning skills. as there is reflected in One can imagine calling for sweeping changes in the view that education the educational system to enhance the indirect is failing in its most central mission—to acquisition of reasoning skills; indeed, numerous teach students to think. reports and inquiries have done exactly that. But - Deanna Kuhn the prospect of any such changes actually materialising is remote at best. It thus appears that if we really want to improve reasoning skills, it will have to be through a renewed and increased emphasis on direct methods. Before prescribing that everyone rushes off to take courses in reasoning, we should ask whether such courses are of any benefit. Unfortunately, the evidence on this question is sparse and inconclusive. Over the past few decades there have been many different programs developed with the deliberate intention of improving general thinking skills of one form or another. These programs vary greatly in the extent to which they focus on informal reasoning, and most have not been subjected to desirable levels of empirical scrutiny. After a comprehensive review, Raymond Nickerson and colleagues concluded that these programs often make no detectable difference, and even when they do, gains are mostly “modest.”18 Further, the measured gains are on the specific materials used in the programs, and do not transfer to general thinking.19 This is not exactly encouraging stuff. What about the direct teaching of informal reasoning skills in particular? Does it do any good? The most widespread, coherent and deliberate program for directly teaching reasoning skills is probably the array of university-freshman level courses going under titles such as “Informal Logic,” “Critical Thinking,” and “Introductory Logic”. Every year philosophy departments offer these courses to tens, perhaps hundreds of thousands of students around the world. There is strong consensus on their content, as embodied in a number of standard textbooks. The rhetoric surrounding these courses constantly emphasises the importance of good reasoning. But do these “informal logic” courses in fact lead to improvement in reasoning skills? When I arrived at Indiana University as a newly-minted Ph.D, my first teaching responsibility was an informal logic course. The chairman offered this as a “cushy” assignment to ease the transition from graduate student to assistant professor. Over the next four years, that course became my speciality. Yet I gradually came to suspect that, if the goal of teaching is to facilitate learning, and if the

— 10 —

students were supposed to be learning how to reason, then informal logic was actually much more difficult to teach than other courses, including specialised graduate courses in philosophy of mind. My students, it seemed, were hardly any better off at the end of semester than they were at the beginning. Of course, they had picked up some vocabulary and a few tricks to help them through the tests. But their reasoning abilities were mostly still so dismal that it was difficult to believe they were much worse when they first came in. Only later did I realise that my suspicions were in line with the empirical data. There is astonishingly little published evidence on the matter. However, the evidence that is available suggests that courses such as the one I was teaching are largely a waste of time. For example, David and Linda Annis pre-and post-tested students in an introductory logic class using the well-established Watson-Glaser Critical Thinking Appraisal (CTA). The basic result, for our purposes, was that taking the logic course gave no significant advantage on the overall CTA measure.20 A more rigorous study by Richard Nisbett and colleagues found that an informal logic class made no discernible difference to simple logical reasoning.21 These results are consistent with the data from a general study of the development of critical thinking in the first year of university. Pascarella found “only trivial and statistically nonsignificant associations between the number of... logic courses taken during the freshman year and either CTA total or subscale scores at the end of the freshman year.”22 In other words, there is currently no good evidence that informal logic courses are in fact of any real help in improving general reasoning skills. We cannot conclusively infer that such courses never do any good, but the data as it currently stands definitely warrants scepticism. There is an obvious irony in this situation. Informal logic courses constantly emphasise the importance of evaluating the reasons behind one’s beliefs. Yet scores of informal logic teachers every year put a great deal of effort into teaching courses without any good evidence that these courses have their intended effect. Of course, they do have their own informal observations of the performance and progress of students in their classes. But it has been established beyond doubt that intelligent professionals can be utterly deluded about the impact of their own activities, when they base assessments on casual observation. Consider, for example, the efficacy of psychotherapy. Therapist Robin Dawes reports that the upshot of hundreds of empirical studies is that patient improvements have almost nothing to do with the kind of therapy undertaken or the credentials or

— 11 —

experience of the psychotherapist.23 Similarly, a mountain of evidence indicates that, with rare exceptions, investment professionals (stockbrokers, mutual fund managers, etc.) have no capacity to choose or advise wisely. Indeed, they typically perform worse than simple strategies such as choosing randomly or predicting that an interest rate will stay the same.24 Presumably, most psychotherapists and investment professionals believe that their professional expertise produces results that justify their fees. The objective evidence shows otherwise quite conclusively. My point here, of course, is that we currently have no good reason to believe that philosophy professors teaching informal logic classes are doing any better. If standard informal logic courses really do fail to improve reasoning, why is this? Reflecting on my own teaching experiences, I came up with a long list of plausible partial explanations. The three most pertinent are: • Knowledge versus practice. Informal logic courses purport to teach students how to reason, but in fact they mostly teach students about reasoning. The idea is that learning the theory of good reasoning will automatically lead to improved performance. This works about as well as teaching people how to dance by discussing the theory of dance in the classroom. • Teaching Useless Stuff. Much of the content of I challenge anybody here to show me a informal logic courses consists of elementary serious piece of formal logic—propositional logic and truth argumentation in tables, syllogisms and Venn diagrams, perhaps natural languages that has been successfully a little quantificational logic. Unfortunately, evaluated as to its such techniques are almost completely irrelevant validity with the help of formal logic. I regard to everyday reasoning, as I discovered when I this as one of the vowed to use only genuine examples of greatest scandals of everyday reasoning in class and on tests. It was human existence. - Y. Bar-Hillel exceedingly difficult to find any to which formal tools applied. The examples used in the textbooks are, with very rare exceptions, invented by the author. • Empirically uninformed. A considerable amount of empirical research has been done on the kinds of reasoning people ordinarily engage in, and their natural abilities and deficiencies (discussed further below). Typical informal logic courses are utterly oblivious to this mountain of evidence. As a result they spend much time teaching material of marginal relevance to the students’ real needs and problems. It is as if a tennis coach were to discourse on the finer points of doubles play, while failing to notice that some students are still holding the racquet by the wrong end.

— 12 —

The evidence I have been laying out indicates that current educational methods are failing to deliver adequate levels of reasoning skills. Fortunately, it doesn’t establish that reasoning skills can’t be taught. It might simply be that we haven’t yet figured out how to do it effectively. Perhaps it is time to fundamentally rethink how we go about teaching informal reasoning skills. 5. COGNITIVE SCIENCE AND EDUCATIONAL DESIGN When educational methods are failing, there must be a mismatch between what is being offered and what is really needed. To eliminate the mismatch we have to understand what it takes for reasoning skills to grow. We have to understand the nature of thinking and how learning happens. These issues are the domain of cognitive science. Thus, cognitive science ought to be the starting point of any serious program of educational innovation. As John Bruer put it in his Schools for Thought: Cognitive scientists study how our minds work—how we think, remember, and learn. Their studies have profound implications for restructuring schools and improving learning environments. Cognitive science—the science of the mind—can give us an applied science of learning and instruction.25 When education is properly grounded in cognitive science, the results can be impressive. Bruer describes a selection of successful projects in the teaching of elementary arithmetic, geometry, physics, reading, and writing. “Teaching methods based on [cognitive science] research,” he claims, “are the educational equivalents of polio vaccine and penicillin.” Clearly, our informal reasoning skills need a strong shot of this penicillin. But that is much easier said than done. What, more precisely, does contemporary cognitive science have to tell us about reasoning skills? And how can we use that knowledge to develop effective educational methods? 6. COGNITIVE SCIENCE : IMPLICATIONS FOR LEARNING TO REASON Cognitive science provides relevant insights at three distinct levels: the general nature of cognition; the acquisition of cognitive skills; and the shape of reasoning skills in particular. 6.1 The general nature of cognitive processes The picture of mind we begin with has a lot to do with how we end up teaching. In early days, cognitive science was dominated by a conception of thinking as internal symbol manipulation. It was, roughly, a view of mind as a programmable digital computer inside

— 13 —

the skull. Intelligence is a matter of deploying suitable rules and data structures; learning is like “putting tools in the shed,” as William Clancey has put it. On this model of intelligence and knowledge, teaching someone to reason should just be a matter of having them learn and follow the general rules of good inference—i.e., exactly the approach taken in standard informal logic courses. Over the past few decades this crude picture has taken quite a hammering. Counterevidence has come from many directions, including increasing understanding of the neural basis of cognition,26 difficulties in artificial intelligence,27 and philosophical problems with the orthodox computational conception of mind. 28 Many cognitive scientists still subscribe to sophisticated descendants of the original picture, but many others have been pursuing quite different approaches. A number of general themes emerge from the cacophony of contemporary reactions to the standard computational account. One is that much of cognition is best thought of not as inner activity of an isolated Cartesian mind, but as a form of cooperation between brain, body, and physical or social environment. Complex argumentation, for example, usually involves speaking or writing, interlocutors, and external storage devices such as paper or wordprocessors. The role of the brain is not to carry out the argumentation, but rather to facilitate the collective achievement.29 A second theme is that human knowledge cannot in general take the form of explicit rules and symbolic data structures.30 The alternative is that most of what we know is actually stored implicitly, distributed across large numbers of parameters (e.g., synaptic connections).31 And a third theme is that cognitive processes are best thought of not as the algorithmic manipulation of digital symbols, but as state-space evolution in dynamical systems.32 One obvious question is whether these alternative approaches have the resources, even in principle, to account for reasoning skills. In one famous defence of computational orthodoxy, Jerry Fodor and Zenon Pylyshyn argued that cognition in general, and inference in particular, is systematic, and that only “classical” computational models can explain this fact. Lars Niklasson and I have done modeling work showing that connectionist dynamical systems using only non-symbolic, distributed representations are indeed able to perform propositional inference with very high levels of systematicity.33 The question remaining to be settled is the empirical one: which approach delivers the best detailed models of human reasoning, with all its idiosyncrasies and deficiencies?34 Thinking of cognition as embedded, distributed and dynamic suggests a very different general orientation on the problem of teaching reasoning. The aim is not to fill the head with general rules, but to

— 14 —

gradually alter the myriad parameters such that the dynamical processes come to constitute increasingly skilful performances. In order to achieve that, however, we need more detailed understanding of how cognitive skills are acquired in general, and the shape of human reasoning skills in particular. 6.2 Cognitive skill acquisition Concerning the acquisition of skills, cognitive science delivers some fairly clearcut lessons. First, skill acquisition in general typically follows a three-stage trajectory from novice to expert.35 In the first or “cognitive” stage, the novice’s activity consists primarily in attempting to deliberately follow explicit rules describing the domain and actions to be taken. Execution is laborious, slow and clumsy. In the second or “associative” stage, sequences of steps become “chunked” into automatic procedures that can be carried out swiftly and smoothly. In the third, or “autonomous” stage, skilful performance has become maximally automatic and highly tuned to the task at hand. Conscious control is refocussed, as when an expert driver is planning a route across the city even while unconsciously negotiating a busy intersection. The main force behind the transition from beginner to expert and even elite performance is what K. Anders Ericsson calls “deliberate practice.”36 Mere repetition is not enough. According to Ericsson, practice is effective to the extent that it (a) is motivated by the goal of improving performance; (b) is accompanied by coaching to provide direction and rapid feedback; (c) involves continual selfmonitoring; and (d) focuses selectively on improving particular aspects of the overall skill. It is also interesting to note that four hours a day seems to be the optimal quantity of deliberate practice, and that plenty of rest and sleep is required. What about acquisition of higher cognitive skills in particular? John Bruer has integrated the lessons from four decades of cognitive science concerning the fundamental requirements for higher cognitive expertise. 37 The three main elements of this “new synthesis” are domain knowledge, general methods, and metacognition. Good thinking always takes place in some domain or other. It is impossible to think well unless one has at least some knowledge of the domain; and in general, the more the better. Nevertheless, experts do draw upon methods or strategies that can be applied in almost any domain, enabling them to handle a wider range of problems more effectively.38 They also have superior metacognitive capacities. They have better understanding of the nature of thinking in general; an appreciation of their own strengths and limitations as thinkers; and while thinking they actively monitor their own processes and level of performance.

— 15 —

These insights into the nature of cognitive skills have direct implications for skill acquisition and hence teaching. As Bruer put it, “instruction based on elements of the new synthesis is our best educational bet if we want all students... to acquire higher order skills.” (p.79) 6.3 Patterns in Human Reasoning The third major contribution of cognitive science is what it tells us about human reasoning itself—the kinds of reasoning patterns we habitually engage in, the kinds of errors we make, how those errors can be avoided, and what kinds of inference can be learned. Most directly relevant is work done by psychologists such as Deanna Kuhn and David Perkins on the general outline of peoples’ informal or “everyday” reasoning habits. For example, Perkins and colleagues have extensively studied how people justify their stands on issues such as whether violence on TV leads to violence in real life. They have concluded that people base their reasoning on a picture or “model” of the situation, and the most common problem is failure to develop a sufficiently elaborate model. Good critical reasoners, by contrast, actively search for inadequacies in their current model.39 Also relevant is the extensive research done by psychologists on particular kinds of inference. At least three research programs are worth mentioning here. One is the psychology of simple deduction, such as conditional arguments or classical syllogisms.40 A second program focuses on “hypothesis testing” in the context of laboratory problems such as Wason’s four card test, or the “2-4-6” problem.41 Third, there is the voluminous research on statistical reasoning and decision-making conducted by Amos Tversky and Daniel Kahneman, and their numerous followers and critics.42 The most general lesson from these kinds of detailed experimental studies is that human reasoning has strong tendencies to go wrong in surprisingly systematic and predictable ways. Any program for reasoning education must be cognisant of, and counteract, these tendencies.

The mind of man is far from the nature of a clear and equal glass, wherein the beams of things should reflect according to their true incidence; nay, it is rather like an enchanted glass, full of superstition and imposture, if it be not delivered and reduced. - Francis Bacon

Finally, valuable work has been done on the effects of attempts to teach certain specific reasoning skills. Most notable here is the work by Nisbett and colleagues, who have concluded that people can in fact learn to make better use of some quite general inferential rules, such as the “law of large numbers” from statistics, and some pragmatic inference schemas. However, some other general

— 16 —

principles, such as abstract rules of formal logic, appear to firmly resist learning.43 7. A P EDAGOGICAL MODEL The challenge now is to make use of these various kinds of insight in designing truly effective methods for teaching reasoning skills. In doing this, it is useful to distinguish between the pedagogical model, and the implementation of that model in a particular social, economic and technological context. The pedagogical model specifies how instruction would proceed in ideal circumstances. The implementation translates the ideal model into practical methods for real classrooms. The guiding idea of the new pedagogical model is that if reasoning is a cognitive skill, and practice is what drives skill acquisition, then learning to reason must be centred on practice. Of course, this is not exactly a surprising idea. In one form or another it can be traced back all the way to Plato. If the An ounce of practice idea is remarkable, it is only because it conflicts is worth a pound so directly with way reasoning is in fact usually of precept. - English proverb taught. Standard courses emphasise theory over practice in at least three ways. First, as a matter of educational philosophy: the ability to reason is supposed to follow automatically from a sound understanding of the theory of reasoning, and the role of practice is simply to verify that the theory has been properly assimilated. Second, chronologically: in classes and textbooks, students are bombarded with theory long before they ever get to give it a go themselves. And third, in the sheer quantity of time given over to the effort to acquire theoretical understanding rather than actually engaging in reasoning. It is important to understand that a practice-based pedagogical model doesn’t dispense with theory or explicit instruction. Rather, it reverses the traditional priority of theory over practice. The ability to reason is acquired primarily through engaging in reasoning rather than learning about it. Students begin reasoning almost immediately; reasoning takes up most of their time and effort; and theoretical understanding of reasoning emerges out of their growing competence in reasoning, rather than vice versa. The role of theory and explicit instruction is to facilitate improvement through practice. The idea that a grasp of reasoning should be acquired through practice is just the point of departure. The bulk of the pedagogical model is a series of prescriptions concerning the nature of practice required if ordinary reasoning skills are to be substantially enhanced.

— 17 —

These prescriptions can be loosely divided into two categories: what to practice, and how to practice it. What to practice? 1. Activities. The skills of general informal reasoning can be analysed into a hierarchical structure of activities or procedures. This is not the place to lay out this full structure; suffice to say that at the very highest level we should distinguish four progressively more complex activities: • Critical evaluation. Materials are presented, and students must determine if they contain reasoning; if so, what that reasoning is, and how strong it is. • Production. Students must produce good reasoning in support of claims. This involves critically evaluating their own attempts. • Debate. Students take sides on an issue, and attempt to demonstrate the superiority of their position by producing good reasoning to support their case, and critically evaluating the reasoning proposed on the other side. • Inquiry. Students must determine, to the best of their ability, what side they should take on an issue. Objective, rational inquiry can be understood as internalised debate, where the one student must play both sides of the fence. Each of these activities can be broken down into sub-procedures. For example, inquiry, debate and production all invoke the ability to critically evaluate reasoning, and critical evaluation can itself be broken down into a series of distinct activities (around nine, at the first level of analysis). 2. Range of Materials. Reasoning, in the sense relevant here, is general skill. Students must practice that general skill on a wide range of materials, and not just abstract, formal or circumscribed domains such as chess, Latin or arithmetic. Students should build their skills in domains with which they are relatively familiar, but should exercise those skills on materials that are wholly novel or unusually challenging. 3. Empirically Informed. The nature of the activities, and the choice of practice materials, must be guided closely by what we have learned from empirical studies of the patterns and weaknesses of human reasoning. 4. Metacognition. The practice regime must be specifically designed such that students acquire an understanding of their reasoning abilities and limitations, and the habit of actively monitoring their own reasoning processes.

— 18 —

How should practice proceed? 5. Scaffolding. The moves students make in this practice ought to be appropriate ones; otherwise practice just reinforces bad habits and never leads to mastery. Thus the practice must take place within a framework which structures student efforts, like training wheels on a bicycle. Scaffolding is gradually removed as expertise develops. 6. Staging. Effective practice requires that the activities and exercises are appropriately staged, both in order of difficulty and complexity, and in the order of dependence. Practice in critical evaluation must precede practice in debate, and students should try relatively simple texts like newspaper letters before they tackle Hegel’s Phenomenology of Spirit. 7. Instruction. Acquisition of reasoning, like any skill, is accelerated by explicit verbal instruction. That instruction must be targeted to the student’s particular level of competence and current problems. Importantly, in this pedagogical strategy explicit instruction is given concurrently with attempts at the relevant aspect of reasoning. The student’s “hands on” experiences provide a context in which the instruction is seen as relevant and useful, is more easily understood, and can be put into practice right away. 8. Feedback. Reasoning is a particularly difficult skill to master. This is partly because poor performance can seem perfectly adequate to the reasoner, and so she sees no need to improve it. Further, it is not enough to know that their reasoning is inadequate; students need to know in what way it falls short. For these reasons, it is important that they receive regular and rapid feedback on their performance. 9. Interaction. Reasoning with others, whether cooperatively or competitively, has the usual virtue of providing interest and motivation. Additionally, when a student knows her reasoning will be open to critical scrutiny from others, there is incentive to anticipate and pre-empt those criticisms. Over time, the student learns to “internalise the external critic”—i.e., to think critically. This practice-based pedagogical model is solidly grounded in contemporary cognitive science. Consequently, my brash hypothesis is this: if any educational method can make a genuine difference to reasoning skills, then a method based on this model can. Or, more prosaically: Instruction and learning realising the practice-based pedagogical model will lead to substantial gains in reasoning skill. Indeed, we can be more specific. According to John Anderson, “virtually every study of skill acquisition has found a straight-line

— 19 —

function on a log-log plot” of performance time against practice.44 This is the well-known “power law of practice.” Our default hypothesis should be that reasoning abilities also will improve as a power function of practice. This is actually rather encouraging; it means that we should expect a relatively steep “learning curve” in early stages of practice. If informal reasoning turns out to be an apparent exception to the general principle, that would be very interesting in itself, and worthy of detailed exploration. 8. COMPUTER-BASED IMPLEMENTATION So we have a pedagogical model for reasoning skills, and we have a coarse prediction based on that model. How can the model actually be implemented, such that the prediction can be tested? Implementation is, by definition, a practical matter. The contingent realities of one’s actual situation—social, economic, and technological—determine the nature of an implementation and the extent to which it succeeds in realising the model. Three general features of our current situation are overwhelmingly important for any attempt to implement a practice-based pedagogical model for reasoning skills: the expense of human expertise; the relatively low priority of education; and the rapidly advancing nature of information technology. The obvious way to implement the pedagogical model is to have students enroll in courses which demand constant practice in reason and argument over a long period closely supervised by expert instructors. This has actually been tried already. The traditional Oxford-style philosophy degree is a fair approximation, with undergraduate students honing their argument skills over a number of years in intimate tutorials. Does it work? Certainly these degrees have a reputation for turning out accomplished thinkers and debaters. The value of intensive philosophy training gained some support in Kuhn’s study, which included a small sample of philosophy graduate students: The performance of the philosophers is...easily summarized. No variation occurs, with all five philosophers showing perfect performance in generation of genuine evidence, alternative theories, counterarguments, and rebuttals....The performance of the philosophers shows that it is possible to obtain expertise in the reasoning process itself, independent of any particular content to which this reasoning is applied.45 However, there is an obvious problem with any proposal to implement the practice model with direct human instruction. For the supervision (and hence practice) to be effective, tutors must have a relatively high level of expertise. Further, teachers cannot effectively

— 20 —

supervise more than a small handful of students at a time. Human labor is generally expensive, and even more so when it calls upon special skills. It would be costly indeed to use human teachers to implement the practice model on a large scale. This problem might not be insurmountable in a social and economic context in which education was accorded a high priority. However, in most countries today education fares relatively poorly in the competition for resources. In countries like the United States and Australia, there is never a shortage of rhetoric about the value of education and importance of investing heavily in it. But when budgets get drawn up, education usually ends up on a near-starvation diet—just enough money to keep the system alive, but never quite enough to gain any strength. This deplorable situation is unlikely to change anytime soon. We therefore seem to be in a dilemma. The existing approaches to direct teaching of reasoning skills are affordable, but largely inefficacious. The proposed alternative seems promising, but prohibitively expensive. Fortunately, there may be a way out. The third relevant feature of our current situation is the increasing availability and sophistication of information technology. We are entering an era when education will be radically transformed by computers and computer networks. The challenge before us is to harness these changes to the cause of improving reasoning skills. Again, there is nothing new about the basic idea. The dream of using computers to enhance reasoning skills is almost as old as the field of artificial intelligence. There have even been programs aimed at developing the kind of general argumentation skills that are the focus of attention here.46 However, all such work has run straight into a major roadblock: the intractability of informal reasoning. Coaching reasoning requires the ability to understand natural language, and to follow and engage in reasoning. Yet these abilities are the quintessence of distinctively human intelligence. Natural language and informal reasoning are two of the hardest problem domains in artificial intelligence. Hundreds, perhaps thousands of researchers have been attacking these problems for decades, and we are still a long way from recreating anything like human abilities. Thus the ambition of providing a “teacher in a box” is completely unrealistic, at least for the foreseeable future. The most common response has been to narrowly circumscribe the domain and the reasoning activities involved—i.e., to shift to a kind of reasoning “microworld.” Such efforts focus on developing specific skills in a particular domain, with the hope that this will also boost general reasoning skills.

— 21 —

These kinds of intelligent tutors have a very mixed record. Consider CATO, a state-of-the-art package developed at one of the world’s foremost educational research institutes, the Learning Resource Development Center at the University of Pittsburgh. This “intelligent learning environment” aims to assist law students make arguments on the basis of previous cases. CATO “knows” quite a lot about some important cases in the area of trade We have not had a secrets, and has an impressive ability to manipulate thousand failures. We argument structures in that domain. Nevertheless, it have discovered a thousand things that illustrates some of the difficulties facing any attempt don’t work. to enhance general reasoning skills using microworld - Thomas Edison packages. At the top of the list is the problem of generalisation or transfer. When evaluated, CATO was found to be as effective in enhancing basic argument skills in its domain as classes with a human teacher. Unfortunately this improvement did not transfer to a closely related reasoning task. CATO was not evaluated for its impact on general informal reasoning skills. Thus, the little evidence we do have invites pessimism about the ability of CATO to significantly enhance general reasoning skills.47 There is an alternative to the microworlds approach. Instead of attempting to build an intelligence-possessing system with highly restricted scope, we can build an intelligence-enhancing system with wide scope. Instead of recreating intelligence in the computer, we can have the computer magnify intelligence that is already available. In the ordinary classroom situation there are three sources of intelligence. The first is the student herself. Although she may be a novice at argumentation, she has other abilities, presupposed by good reasoning, that are currently beyond the reach of computers—most notably, natural language understanding. Second, there is the intelligence and reasoning ability (such as it is) of her peers. Third, there is the expertise of the instructor. On this alternative approach, the computer takes intelligence from these various sources and leverages it to maximum advantage. This idea is the basic design principle underlying the Reason! package under development at the University of Melbourne. Reason! is a kind of scaffolding whose role is to guide students through the four main activities in the pedagogical model—critical evaluation, production, debate, and inquiry. Each of these activities, when fully broken down, is a complex hierarchy of operations. For example, in the critical evaluation stage, Reason! requires the student to • determine whether the text contains reasoning • identify the major conclusion • identify the stated support

— 22 —

• identify assumptions • diagram the structure of the argument • assess the acceptability of the reasons and assumptions • evaluate the strength of support • evaluate whether the conclusion should be accepted Reason! will also guide the student through various steps required in executing each of these operations. At every stage, it provides help of various forms—a hypertext guide to informal reasoning, and contextsensitive hints and FAQ files. Reason! provides a structured environment for practicing reasoning activities. It is a direct implementation of the basic insight behind the pedagogical model, that mastery of the skills of argument is acquired only through engaging in reasoning. It is also designed with a view to meeting—as far as is feasible—the detailed prescriptions of the pedagogical model. In order to do this, Reason! leverages the available human intelligence in various ways. Reason! itself is a general purpose, domain-independent engine. The engine operates on modules of exercises. For example, one module may consist of a series of reports of scientific studies drawn from newspapers and magazines, together with model answers, hints, etc., provided by the module developer (e.g., the course instructor). The student asks Reason! to load up an exercise, and proceeds to work through the various steps. At various points the student is able to (or may be required to) call up the model answer and compare it with what she has produced. In this way Reason! provides a framework within which the intelligence of the student and the instructor conspire to deliver rapid feedback on the student’s reasoning attempts. Of course, this feedback may be of lower quality than that which would be provided by a live human tutor; however, it has the distinct advantage that a large supply of it is always immediately available. Another way Reason! leverages available human intelligence is by providing a structured framework for interaction with, and evaluation by, peers. For example, the production phase can be an iterative process in which one student lays out an argument, another provides a (highly structured) critical evaluation, the argument is revised, and so on. Reason! is still in early stages of development. The first systematic study of its effectiveness at enhancing reasoning skills is scheduled for the second half of 1998. This study, and subsequent ones, will provide at least some evidence bearing on the major research hypothesis, that instruction and learning based on the practice-based pedagogical model leads to demonstrable gains in reasoning skill. Of course, if no significant effect is found, the problem

— 23 —

might be with Reason! rather than the pedagogical model. Thus, the studies can also be seen as bearing on an auxiliary research hypothesis: Effective computer-based implementations of the practic model can be developed. The idea that this hypothesis is true is the hunch driving the development of the Reason! package. 9. CONCLUSION In the preceding pages I have argued for two main points. The first is that we are confronting what I have been calling a crisis of reason. The second is that the only viable avenue of response is to develop computer tools for instruction in reasoning skills. However, developing such tools is no easy matter. As Thomas Landauer has emphasised, genuinely helpful computer tools are very difficult to design at the best of times.48 These difficulties are magnified when the tools must operate in the domain of human intelligence. Success in this project will require an intensive research and development program over many years. Currently, there are some efforts in this direction scattered around the world. These are valuable but insufficient. We must be prepared to double and redouble our efforts. While teaching informal reasoning at Indiana University, I often used to wonder what kind of improvements we might see if only students were as committed to thinking as the athletes at Human history that great sporting institution were to sport. becomes more and These days, as a research fellow, I wonder what more a race between education and kind of progress we could make if we invested in catastrophe. our cognitive health even half the resources we - H.G. Wells invest in our physical health. Making that investment is, to be sure, a kind of gamble, given the scale of the problem and the theoretical and technical challenges involved. However it is a gamble we must take, since—if the argument presented here is correct—we really have no other option.

— 24 —

N OTES 1 2 3

4 5 6 7 8 9

10

11 12 13 14

15 16 17

18 19 20 21 22 23 24

25 26 27 28 29 30

(Beckher, 1622) (Kuhn, 1991) Chapter 9. Gordon Anrig, president of the Educational Testing Service, quoted in (Bruer, 1993), p.5-6. Bruer’s Chapter 1 contains an alarming review of the NAEP data. (Singal, 1991). A useful review is (Baron, 1994). (Piatelli-Palmarini, 1994). See, for example, the essays in (Washburn, 1996). See (Postman, 1985), chapter 4. These issues are discussed in (Rifkin, 1995); on this point, see particularly chapter 12. A Nation at Risk: The Imperative for Educational Reform was a specially commissioned report to the US Department of Education by The National Commission on Excellence in Education, drafted in the form of an open letter to the nation. (Rifkin, 1995), chapters 14, 15. E.g., (Mander, 1978). (de Bono, 1991) is a good representative of the de Bono vision. (Bricmont & Sokal, 1997; Gross & Levitt, 1994; Sokal, 1996; Windschuttle, 1994). Quoted in (Washburn, 1996). (Mencken, 1956), p.127. (Pascarella, 1989). Note that more than half the improvement in critical thinking skills acquired at university happens in the first year (Pascarella & Terenzini, 1991). For an even more pessimistic assessment, see (Follman, 1987). (Bruer, 1993), pp.64-5. (Annis & Annis, 1979). (Nisbett, Fong, Lehman, & Cheng, 1987). (Pascarella & Terenzini, 1991), p.139. (Dawes, 1994), Chapters 2 and 4. A mountain of good evidence supports these rather astounding facts. Good popular introductions include (Dreman, 1982; Malkiel, 1995). (Bruer, 1993) p.2. (Churchland & Sejnowski, 1992). (Dreyfus, 1992). See, e.g., (Clark, 1989). (Clark, 1997). (Horgan & Tienson, 1996).

— 25 —

31

32 33 34 35 36 37 38 39

40 41 42 43 44 45

46 47 48

(Churchland, 1988). On the notion of distributed representation, see (van Gelder, 1991; van Gelder, 1992). (Port & van Gelder, 1995; van Gelder, forthcoming). (Niklasson & van Gelder, 1994). (van Gelder & Niklasson, 1994) See (Anderson, 1990), Chapter 9. (Ericsson & Charness, 1994). See (Bruer, 1993), Chapter 3. (Sternberg, 1985); (Resnick, 1987). See (Perkins, 1989; Perkins, Allen, & Hafner, 1983; Perkins, Faraday, & Bushey, 1991). See, e.g., (Johnson-Laird & Byrne, 1991). Essays in (Newstead & Evans, 1995) provide reviews of this field. For a recent review, see (Goldstein & Hogarth, 1997). (Nisbett, et al., 1987). (Anderson, 1990), p.261 (Kuhn, 1991), p.258, 262. It would be rash to conclude from this evidence alone that philosophy training caused the superior performance of the philosophy students. See, e.g., (Carbonell, 1970). See (Aleven & Ashley, 1997). (Landauer, 1995).

— 26 —

REFERENCES (1983) A Nation at Risk: The Imperative for Educational Reform. Washington DC: U.S. Government Printing Office. Aleven, V., & Ashley, K. D. (1997) Teaching case-based argumentation through a model and examples: Empirical evaluation of an intelligent learning environment. Proceedings of the Eighth World Conference on Artificial Intelligence in Education. Anderson, J. R. (1990) Cognitive Psychology and Its Implications (3rd ed.). New York: W.H. Freeman. Annis, D., & Annis, L. (1979) Does philosophy improve critical thinking? Teaching Philosophy, 3, 145-152. Bar-Hillel, Y., & others (1969) Formal logic and natural languages: A symposium. Foundations of Language, 5, 256-284. Baron, J. (1994) Thinking and Deciding (2nd ed.). Cambridge: Cambridge University Press. Beckher, D. (1622) Medicus Microcosmus. Bricmont, J., & Sokal, A. (1997) Impostures Intellectuales. Paris: Editions Odile Jacob. Bruer, J. T. (1993) Schools for Thought: A Science of Learning in the Classroom. Cambridge MA: MIT Press. Carbonell, J. R. (1970) AI in CAI: An artificial intelligence approach to computer-assisted instruction. IEEE Transactions on Man-Machine Systems, 11, 190-202. Churchland, P. M. (1988) A Neurocomputational Perspective. Cambridge MA: MIT Press. Churchland, P. S., & Sejnowski, T. J. (1992) The Computational Brain. Cambridge MA: Bradford/MIT Press. Clark, A. (1989) Microcognition: Philosophy, Cognitive Science, and Parallel Distributed Processing. Cambridge MA: MIT Press. Clark, A. (1997) Being There: Putting Brain, Body and World Together Again. Cambridge MA: MIT Press. Dawes, R. M. (1994) House of Cards. New York: The Free Press. de Bono, E. (1991) I Am Right, You Are Wrong. Harmondsworth: Penguin. Dreman, D. N. (1982) The New Contrarian Investment Strategy. New York: Random House. Dreyfus, H. L. (1992) What Computers Still Can't Do: A Critique of Artificial Reason. Cambridge MA: The MIT Press. Ericsson, K. A., & Charness, N. (1994) Expert performance. American Psychologist, 49, 725-747.

Follman, J. (1987) Teaching of critical thinking—promises! promises! Informal Logic, 9, 133-140. Goldstein, W. M., & Hogarth, R. M. (Ed.). (1997). Research on judgment and decision making. Cambridge: Cambridge University Press. Gross, P. R., & Levitt, N. (1994) Higher Superstition: The Academic Left and its Quarrels with Science. Baltimore: Johns Hopkins University Press. Horgan, T. E., & Tienson, J. (1996) Connectionism and the Philosophy of Psychology. Cambridge MA: MIT Press. Johnson-Laird, P. N., & Byrne, R. M. J. (1991) Deduction. Hillsdale NJ: Erlbaum. Kuhn, D. (1991) The Skills of Argument. Cambridge: Cambridge University Press. Landauer, T. K. (1995) The Trouble with Computers: Usefulness, Usability, and Productivity. Cambridge MA: MIT Press. Malkiel, B. G. (1995) A Random Walk Down Wall Street (6th ed.). New York: W.W. Norton & Co. Mander, J. (1978) Four arguments for the elimination of television. New York: Morrow. Mencken, H. L. (1956) Minority Report: H.L. Mencken's Notebooks. New York: Knopf. Newstead, S. E., & Evans, J. S. B. T. (Ed.). (1995). Perspectives on Thinking and Reasoning: Essays in Honour of Peter Wason. Hillsdale NJ: Erlbaum. Niklasson, L., & van Gelder, T. (1994) On being systematically connectionist. Mind and Language, 9, 288-302. Nisbett, R. E., Fong, G. T., Lehman, D. R., & Cheng, P. W. (1987) Teaching reasoning. Science, 238, 625-631. Pascarella, E. (1989) The development of critical thinking: Does college make a difference. Journal of College Student Development, 30, 19-26. Pascarella, E. T., & Terenzini, P. T. (1991) How College Affects Students: Findings and Insights from Twenty Years of Research. San Francisco: Jossey-Bass. Perkins, D. N. (1989) Reasoning as it is and could be: An empirical perspective. In D. M. Topping, D. C. Crowell, & V. N. Kobayashi ed., Thinking Across Cultures: The Third International Conference on Thinking. Hillsdale NJ: Erlbaum, 175-94.

— 28 —

Perkins, D. N., Allen, R., & Hafner, J. (1983) Difficulties in everyday reasoning. In W. Maxwell & J. Bruner ed., Thinking: The Expanding Frontier. Philadelphia PA: The Franklin Institute Press, 177-189. Perkins, D. N., Faraday, M., & Bushey, B. (1991) Everyday reasoning and the roots of intelligence. In J. F. Voss, D. N. Perkins, & J. W. Segal ed., Informal Reasoning and Education. Hillsdale NJ: Erlbaum, 83-105. Piatelli-Palmarini, M. (1994) Inevitable Illusions: How Mistakes of Reason Rule Our Minds. New York: John Wiley & Sons, Inc. Port, R., & van Gelder, T. J. (1995) Mind as Motion: Explorations in the Dynamics of Cognition. Cambridge MA: MIT Press. Postman, N. (1985) Amusing Ourselves to Death. New York: Viking Penguin. Resnick, L. B. (1987) Instruction and the cultivation of thinking. In E. D. Corte, J. Lodewijks, R. Paramentier, & P. Span ed., Learning and Instruction. Oxford: Pergamon Press, Rifkin, J. (1995) The End of Work. New York: Putnam. Singal, D. J. (1991) The other crisis in American education. Atlantic Monthly. Sokal, A. (1996) A physicist experiments with cultural studies. Lingua Franca, May/June, 62-64. Sternberg, R. J. (1985) Instrumental and componential approaches to the nature and training of intelligence. In S. F. Chipman, J. W. Segal, & R. W. Glaser ed., Thinking and Learning Skills, Volume 2: Research and Open Questions. Hillsdale NJ: Erlbaum, van Gelder, T., & Niklasson, L. (1994) Classicalism and cognitive architecture. In Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society. Hillsdale NJ: Erlbaum, van Gelder, T. J. (1991) What is the 'D' in 'PDP'? An overview of the concept of distribution. In S. Stich, D. Rumelhart, & W. Ramsey ed., Philosophy and Connectionist Theory. Hillsdale NJ: Lawrence Erlbaum Associates, 33-59. van Gelder, T. J. (1992) Defining "distributed representation". Connection Science, 4, 175-191. van Gelder, T. J. (forthcoming) The dynamical hypothesis in cognitive science. Behavioral and Brain Sciences. Washburn, K. (Ed.). (1996). Dumbing Down: Essays on the Strip Mining of American Culture. New York: W.W. Norton. Windschuttle, K. (1994) The Killing of History: How a Discipline is Being Murdered By Literary Critics and Social Theorists. Paddington, N.S.W.: Macleay.

— 29 —