Psychology and Aging 2000, Vol. 15, No. 4, 648-656

Copyright 2000 by the American Psychological Association, Inc. 0882-7974/00/S5.00 DOT: 10.1037//D8B2-7974.15.4.648

The Effects of Learning a New Algorithm on Asymptotic Accuracy and Execution Speed in Old Age: A Reanalysis Paul Verhaeghen

Reinhold Kliegl

Syracuse University

University of Potsdam

Time-accuracy curves were derived for 16 younger and 19 older persons who participated in a study on training in the method of loci (Baltes & Kliegl, 1992). The effects of instruction were to immediately and permanently boost asymptotic performance and initially slow down the rate of approach to the asymptote. After extensive practice, rate of approach returned to the initial fast level. Age differences were found in both asymptotic performance and rate of approach. The effects of instruction and practice, however, were similar in younger and older adults, but older adults needed 1 session of instruction more than younger adults did before the intervention showed its full effect.

One of the most replicated findings in the field of cognitive

(the onset time). From the onset time on, performance rises in a

aging is the age-related decline in recall from episodic memory

negatively accelerating curve, that is, the performance curve rises

(for a meta-analytic overview, see Verhaeghen, Marcoen, & Goos-

rather steeply from chance level and then gradually becomes less

sens, 1993; Verhaeghen & Salthouse, 1997). It is well known that

and less steep, flattening toward a horizontal asymptote. The third

measures assumed to indicate basic speed of processing are im-

parameter describing the function is the rate of approach, that is,

portant mediators between age and episodic memory (for an over-

the rate at which performance goes from chance level to the

view, see Salthouse, 1996, and Verhaeghen & Salthouse, 1997).

asymptote from the onset time on. More formally, the equation

Research leading to this conclusion has mostly used correlational

used to fit the data in the studies described above (assuming that

analysis. Recently, however, the influence of basic speed on higher

chance level is zero) is

order cognition has also been examined with an experimental method. In these studies (Kliegl, Krampe, Mayr, & Liebscher,

p = c {1 — exp[(a — t)/b]} for t > a: and

1998; Kliegl, Mayr, & Krampe, 1994; Mayr, Kliegl, & Krampe, 1996; Verhaeghen, 1999; Verhaeghen, Kliegl, & Mayr, 1997;

p = 0 for I < a; with a a 0; fc > 0 and e £ 1.

(1)

Verhaeghen, Vandenbroucke, & Dierckx, 1998), processing time is indirectly manipulated through the availability of the external

In this equation, p stands for performance or accuracy and t for the

resource "presentation time," and accuracy is mapped out as a

time available; a is the onset time, b is the rate of approach, and c

function of time. A mathematical function relating time to accu-

represents the asymptote. Note that the b parameter as included in

racy (a time-accuracy function) is then fitted to the data, and the

this equation is scaled such that higher b values indicate slower

parameters describing this function

rates of approach, or, in other words, curves that are less steep.

are used for subsequent

analysis.

One important advantage of using the time-accuracy methodol-

Researchers studying the relation between presentation or pro-

ogy is that it allows for a distinction between dynamic effects (i.e.,

cessing time and cognitive performance in young adults have

effects on rate and onset) and asymptotic effects of aging on

demonstrated that this relation is typically nonlinear and well

cognitive performance. One obvious consequence of age-related

described by a delayed exponential equation (e.g., Dosher, 1976;

slowing might be that aging effects the dynamics only of the curve.

Lohman, 1989; McClelland, 1979; McElree & Griffith,

In this case, older adults would simply need more time for perfor-

1995;

Wickelgren, 1977). This equation is governed by three parameters. Performance remains at chance level until a certain point in time

mance to rise above chance level; or they might need more time to reach their asymptotic level, which would then be equal to that of young adults; or both onset time and rate differences would be present. If only dynamic differences are present, then age differ-

The original research was conducted in the context of the Expertise and Cognitive Aging Project at the Max Planck Institute for Human Development and Education, Berlin, Germany, codirected by Reinhold Kliegl and Paul B. Baltes. Coirespondence concerning this article should be addressed to Paul Verhaeghen, Department of Psychology, 430 Huntington Hall, Syracuse University, Syracuse, New York 13244-2340; or to Reinhold Kliegl, Department of Psychology, University of Potsdam, Post Office Box 60 15 53, D 14415 Potsdam, Germany. Electronic mail may be sent to [email protected] or to [email protected].

ences can be remediated by allowing for more processing time. Salthouse (1996) called this the limited-time mechanism for cognitive slowing. Aging, however, might also affect the level of performance ultimately reachable, or even the quality of the final outcome. Salthouse (1996), for instance, has argued for a simultaneity mechanism of cognitive slowing, that is, one consequence of slowing could be that products of earlier information processing may be lost by the time that later processing is completed. If earlier products are needed for accurate performance, then such a mech-

LEARNING A NEW ALGORITHM

anism might affect the ultimate, asymptotic performance of the system. Another advantage of the time-accuracy methodology is that performance of an individual is now captured in three distinct parameters that each can be assigned meaning, depending on the task. Take the onset parameter. For episodic memory, it makes sense to posit that a word stimulus must be read before it can be in any way remembered and, conversely, that it will stand a minimal chance of being remembered as soon as it has been accessed in the lexicon. Consequently, the onset parameter can be interpreted as the time needed to recognize the word and access the lexicon. For the rate of approach parameter, Kliegl (1995) offered an explanation in terms of elaboration rate. The assumption is that participants studying a list of words generate elaborations (i.e., particular associations) for each of the stimuli and that the probability of recall is directly proportional to the number of elaborations that can be generated (e.g., Craik & Tulving, 1975; James, 1890; Kausler, 1991, p. 351; note that sheer repetition does not necessarily lead to better recall [e.g., Glenberg, Smith, & Green, 1977], therefore this component of elaboration seems a necessary ingredient of effective encoding for free recall). Elaborations are assumed to be sampled out of a finite store of associations. Hence, increased processing time will allow for sampling of more elaborations and consequently to better recall. Kliegl (1995) assumed that after usage, the elaborations are put back into the store. This increases the likelihood of sampling elaborations already retrieved as time increases and will lead to diminishing returns with increased processing time, or in other words, to a time-accuracy curve flattening to an asymptote. The negative exponential is compatible with such a stochastic replacement model of learning (Restle & Greeno, 1970). The asymptote is generally taken to reflect the maximum strength of activation (McClelland, 1979), or the carrying capacity of the system (van Geert, 1993), under the particular circumstances of the task. Previous time-accuracy research has demonstrated that in rather simple tasks, such as figural scanning, word scanning, figural reasoning, and cued recognition (Kliegl et al., 1994; Mayr et al., 1996), older adults are merely slower in getting performance off the measurement floor or in reaching the asymptote than young adults. This suggests that a simple limited-time slowing mechanism is sufficient to explain aging effects in such tasks. However, for recall, age differences in asymptotic performance have been found, as well as age differences in the rate of approach parameter (Kliegl et al., 1998; Verhaeghen et al., 1998, Experiment 1). This suggests that the elaboration processes slow down with age and that the carrying capacity of the system for episodic memory decreases. There is, however, a possible confound here, in the sense that strategic processes at encoding and retrieval are possibly important determinants of episodic recall, and people do differ in the strategies they bring to such tasks (e.g., Verhaeghen & Marcoen, 1996). Behavioral slowing and loss of ultimate accuracy can only rightfully be traced to mental slowing and to a loss of carrying capacity when the correspondence axiom (Cerella, 1990) holds, that is, when we can be certain that young and older adults apply the same processes (in the same sequence) to a task. If strategy differences exist, then behavioral slowing and age-related accuracy differences might be the result of usage of less-than-optimal strategies by older adults and not of mental slowing or loss of

649

carrying capacity per se. Consequently, more conclusive evidence for the existence of age-related differences in speed of encoding processes and in final accuracy should come from research where strategy differences between younger and older adults have been minimized. A line of research that seems particularly promising for our goal of strategy equalization is the intervention research aimed at eliciting cognitive and memory plasticity by teaching older adults effective cognitive and mnemonic strategies (e.g., Baltes & Labouvie, 1973, Baltes & Schaie, 1976; Baltes & Willis, 1982; Denney, 1979; Kliegl & Baltes, 1987; Labouvie-Vief, 1976). It has been demonstrated that the performance of older adults on memory tasks can indeed be boosted considerably by simple instruction and some practice with an efficient mnemonic, such as the method of loci. In a meta-analysis, Verhaeghen, Marcoen, and Goossens (1992) found the effect to be about three quarters of a standard deviation. This demonstration of memory plasticity, in turn, implies that older adults are typically applying suboptimal strategies to the kind of memory tasks psychologists subject them to in their laboratories, but so, apparently, are younger adults. In a follow-up meta-analysis, Verhaeghen and Marcoen (1996) noted that in all memory-training studies (with only one exception) in which both a group of younger and older adults were trained, the effects of training in the older adults were found to be smaller than those in younger adults. Consequently, younger adults actually benefit more from memory training than older adults. Thus, it seems that when younger and older adults are performing identical (and highly effective) strategies, age differences do not disappear; on the contrary, they are exacerbated. This observation still leaves us in the dark as to the locus of the effect: Older adults might be disproportionally slower than younger adults when using effective mnemonics (and this could be due to increased onset times, slower rates of approach, or both), their level of asymptotic performance might increase less after training, or both. Another possibility is that older adults need more practice with the strategy than younger adults and that true equalization of strategy use will only occur after extended practice. In this article, we apply the time-accuracy methodology to a long-term data set that examined plasticity in episodic memory functioning (Baltes & Kliegl, 1992; Kliegl, Smith, & Baltes, Kliegl, Smith, & Baltes, 1989, 1990). In this study, a group of 16 younger and 19 older adults were trained in the method of loci and practiced the method during fifteen 1-hr sessions. The method of loci is an imagery-based mnemonic that is particularly suited for serial recall of words. Briefly, in this mnemonic, participants are instructed to associate each of the words to be remembered with a particular location (a "locus") in an ordered sequence of locations, using interactive imagery for the association (see, e.g., Bellezza, 1987, for more details). At retrieval, the user mentally revisits each location in the correct order and uses the location as a retrieval cue for the image from which the original word is decoded. In this study, a series of 30 Berlin landmarks provided the loci. A total of 38 pretest, instruction, practice, and assessment sessions were administered over a period of 1 year and 4 months. Given the extremely long duration of this program, we assume that the technique will be highly overleamed, and younger and older adults will be thoroughly equated on strategy use at the end of training and practice. Data are available for one pretest occasion and nine posttest occasions. At each of these points during training, serial

650

VERHAEOHEN AND KLIEGL

recall of a list of 30 words was measured at 6 (pretest to sixth posttest) or 12 (seventh posttest to final posttest) presentation times per word, ranging between 0.84 s and 20.00 s. In previous articles reporting on results from this study, data were averaged across these presentation times. In this article, we reexamined the data from these different presentation times to derive timeaccuracy functions for each individual at each of the testing occasions. Previous analysis on this data set revealed sizable treatment gain for both young and older adults. The age differences present at pretest were magnified over the course of training and practice sessions, especially at the longer presentation times, suggesting asymptotic age differences. The first goal, then, of this article is to assess the effects of training and practice in the method of loci on the parameters of the time-accuracy functions. More specifically, we were interested in capturing the differences between the effects of instruction versus continued practice on dynamic and asymptotic effects on the time-accuracy function for serial recall. One interesting aspect of the previous findings, as seen in Figure 2 in Kliegl et al. (1990) and Figure 1 in Baltes and Kliegl (1992), is that the curves describing performance as a function of testing occasions are quite smooth, rising steeply at first but then gradually flattening toward a horizontal asymptote. At first sight, this seems to suggest that the transition from an untrained to a trained and then a well-practiced state is quite undramatic, and it closely follows the general rule of the power law of practice (Newell & Rosenhloom, 1981). A closer investigation of the correlation matrix, however, revealed that the correlation between pretest and the first posttest was quite small, but correlations were high between the different posttest occasions (Kliegl et al., 1990). This then suggests no smooth transition, but indicates that different mechanisms are indeed at play in pretest and posttest, making for a drastic reordering of participants on the performance continuum. No such dramatic change occurs between posttest occasions, suggesting that settling into a well-practiced state is indeed a smooth transition. We hoped to see this qualitative shift from learning to continued practice reflected in the parameters of the underlying time-accuracy functions. Given that the method of loci prescribes a specific strategy, the meaning of the parameters of the time-accuracy function changes after posttest. As stated above, the onset time reflects the time needed for minimal recall of the stimulus. When people are applying the method of loci correctly, encoding of a word entails retrieval of the appropriate locus, reading the word, and forming a combined mental image of the locus and the word. Thus, the onset time at and after posttest should not only reflect perceptual processes needed for word reading, but also the time needed for retrieval of the locus during the encoding phase of the experiment. We remain agnostic as to whether these processes occur in parallel; we assume, however, that an increase hi the onset parameter after training would signify that either the locus retrieval and word reading processes are executed in a serial fashion, or else that queuing is occurring somewhere in the parallel stream of processing (Liu, 1996). A decrease in onset time over the course of practice can mean at least two things: either retrieval of the locus and word reading become increasingly parallel in their execution, or the retrieval process is executed in an increasingly faster fashion. The rate-of-approach parameter would still reflect the time needed for elaboration, which, in this case, means forming and elaborating on an interactive image. Forming interactive images is

not a strategy that most adults spontaneously apply to serial recall (Verhaeghen & Marcoen, 1994). Consequently, we expect that, after posttest, the rate parameter will decrease with continued practice, because the process of generating an interactive image will become more efficient. For the transition from pretest to posttest, we might expect the rate of approach to go up, because this transition entails a change from strategies that are spontaneously applied to the task and are presumably partially automatized to a new, deliberately controlled way of doing the task. Given that the method provides an excellent cueing structure for both encoding and retrieval, we would expect asymptotic accuracy to go up considerably after training. If the sequence of locations is well learned at the first posttest, then we should expect little change in asymptote after the first posttest. The second goal of our reanalysis was to examine age differences in the parameters and how they change across training sessions. On the basis of previous research on adult age differences in time-accuracy functions for recall from episodic memory (Kliegl et al., 1998; Verhaeghen et al., 1998), we expect initial age differences in rate of approach and asymptotic performance but not onset time. One question of particular interest concerns potential age differences in the nature of the transitions and the possibility of interactions between age and testing occasions. It is our assumption that comparison of the final sessions of practice, when the mnemonic is highly overlearned in both age groups, should give us a good estimate of age differences in memory functioning when both groups are using an identical highly effective strategy. Although some literature on aging and skill acquisition exists (for overviews, see Baron & Cerella, 1993, and Salthouse, 1991), our article is the first to investigate age differences at the level of the parameters of an underlying performance model, rather than at the level of observed latencies or observed accuracy.

Method Detailed information on participants, procedure, and testing materials can be found in the original articles describing this study (Baltes & Kliegl, T992; Kliegl et al., 1989, 1990). Here, we briefly reiterate the main points concerning the method.

Participants Participants were 16 young adults, ranging in age between 20 and 30 years, and 19 older adults, ranging in age between 66 and 88 years old. All participants scored above average on the Hamburg-WechslerIntelligenztest fur Erwachsens test (i.e., German version of the WAIS; Hardesty & Lauber, 1956; young IQ, M = 118, SD = 6; old 1Q, M = 125, SD = S). All were in good self-reported health. Younger and older adults did not differ significantly

on intelligence and self-reported

health

measures.

Design The entire experimental schedule consisted of thirty-eight 1-hr sessions, distributed over a period of 1 year and 4 months. After initial assessment, two phases can be distinguished: a training phase (Sessions 5 and 6) and a practice phase (Sessions 7 to 37). The last session was a debriefing session. Materials.

Lists of 30 words were used for the assessment of serial

recall. All words were nouns denoting concrete objects. Words used for the assessment lists appeared only once in the experiment.

651

LEARNING A NEW ALGORITHM

Training conditions.

In each of the training and practice sessions,

scheduled at approximately two sessions per week, 4 lists of 30 words were administered. Across all sessions of training and practice, each participant performed 4,380 trials of interactive imagery, linking a word to a location. During the training phase (Sessions 5 and 6), the method of loci was explained. Words were presented at time intervals comfortable to each individual, that is, without time pressure. During the 15 sessions of practice (Sessions 8, 9, 11, 12, 14, 15, 21, 22, 23, 28, 29, 30, 33, 34, and 35), presentation of words was experimenter paced. Different presentation times between 0.84 s and 20.00 s per word were used (6 different times up until Session 25, 12 after Session 25; see next paragraph for precise values). Participants were moved to shorter presentation times when they passed a criterion (50% or 80%) correct. When performance was below criterion on two successive occasions, longer presentation times were used again.

Performance Assessment Assessment of serial recall occurred in Sessions 3 (pretest), 7 (immediate posttest), 10,13, 16,17, 25, 26/27. 31/32, and 36/37 (practice). There were two assessment formats reflective of the change in rates of presentation times after Session 23. At the first seven occasions, 6 lists of 30 words were administered with presentation times of 20.00 s, 15.00 s, 10.00 s, 5.00s, 3.00s, and l.OOs. At the final three occasions, assessment extended over two sessions. In the first of these sessions, four lists were administered at 20 s, 15 s, 11.84 s, and 8.44 s; in the second occasion, eight lists were administered with presentation times of 6.33 s, 4.75 s, 3.56 s, 2.67 s, 2.00 s, 1.50 s, 1.14 s, and 0.84 s. We decided to drop the shortest presentation times from the analysis, because it is unlikely that the method of loci can be applied at very fast presentation rates. (This is an assumption that seems inherent in research using this mnemonic. Studies using the method of loci with college-age adults have used presentation times of 5.00 s and longer; Bellezza & Reddy, 1978; Carlson, Kincaid, Lance & Hodgson, 1976; De Beni & Comoldi, 1985, 1988; Kemp & van der Krogt, 1985; Lee & Edwards, 1981; Roediger, 1980; Ross & Lawrence, 1968. At shorter times, it may become difficult to both retrieve the locus and start forming an image.) Therefore, only scores based on the five presentation times of 20.00 s, 15.00 s, 10.00 s, 5.00 s, and 3.00 s were used for the time-accuracy analysis. Starting with assessment 26 and 27, mean scores for 11.84 s and 8.44 s were used as an approximation for the 10-s score. Likewise, a score for the 5.00-s rate was based on the mean of scores of the 6.33-s and 4.74-s rate and for the 3.00-s rate on the mean of scores for lists presented at 3.56 s and 2.67 s per word.

<

0

5

10

15

20

25

30

35

40

Session Figure /. The three parameters of the time-accuracy functions for younger and older adults as a function of training in the method of loci (error bars denote standard error of the mean). The first data point (Session 3) represents pretraining performance; all other data points are postinstruction.

sion interaction on the parameters. As explained in the introductory section, we were interested in capturing the differences between the effects of instruction versus continued practice on dynamic and asymptotic effects on the time-accuracy function for

Results

serial recall. Therefore, two series of ANOVAs were conducted, one on the contrast between the pretest and immediate posttest, and

For each individual, the three parameters of the time-accuracy

one on the immediate posttest to final test portion of the learning

function for each testing occasion were estimated by fitting Equation 1 to the data, with proportion recalled as the dependent variable and presentation time (in seconds) as the independent variable. The 10 time-accuracy functions for each participant (one

Effects

of Age and Training-Practice on Onset Time

for each occasion based on five lists with different presentation

For parameter a, the onset time, neither the main effects nor the

times) were fitted simultaneously, using the Statistical Package for

interaction effect reached significance (all Fs < 1, indicating that

the Social Sciences nonlinear regression module. The parameter

the effects

estimates of these individual time-accuracy curves were then av-

contrast. In the analysis of the nine testing occasions between

eraged over individuals within each age group to yield average

immediate posttest and final test, a main effect was found for

are very small) in the pretest-to-immediate-posttest

estimates for younger and older adults for each of the occasions.

testing occasion, F(8,264) = 3.67, MSE = 0.2Q,p< .001, but not

These estimates are depicted in Figure 1. The models fit the data

for age, F(l, 33) = 0.58, MSE = 0.59, m. As can be seen in

quite well; the average pseudo—R2 value for the fit of the individ-

Figure 1, the onset parameter tended to decrease over testing

ual curves was .83 for the young (SD = 0.07) and .89 for the old

occasions. The main effect of testing occasion, however, was

(SD = 0.05).

qualified

by an Age

X Testing Occasion

interaction,

F(8,

An analysis of variance (ANOVA) was conducted to determine

264) = 2.80, MSE = 0.20, p < .01. (Note that this interaction was

the effects of age, testing occasion, and the Age X Testing Occa-

a cross-over Interaction. Hence, no transformation is necessary to

652

VERHAEGHEN AND KLIEGL

account for the effects of general slowing in the onset parameter.)

highly reliable, with older participants being much slower to reach

Regression analysis shows that the decrease in onset time over

asymptote than young adults.

testing occasions was entirely due to changes within the older sample. When we regressed the average onset time on testing occasion number within each age group, linear regression did not fit the data of the young group (ff 2 = .00; regression coefficient

Effects of Age and Training-Practice on Asymptotic Level of Performance

—.00); for the old group, however, there was a reliable decrease of

A problem arose with the analysis of parameter c. Verhaeghen

the onset parameter across testing occasions (J?2 = .58; regression

et al. (1997) suggested to analyze the effects of the asymptotic

coefficient = -.02). This implies that retrieving the locus and

parameter (c) by investigating the effect of c on time demand

identifying the stimulus took a constant amount of time in younger

needed to reach a given level of performance. In other words, not

adults, regardless of the stage in practice. For older adults, these

the parameter c itself is being analyzed, but its impact on the time

processes sped up with about 20 rns for each additional 1-hr

needed to reach a given criterion level of recall. This criterion level

session of practice.

necessarily needs to be smaller than the lowest asymptote reached by any participant in any session—otherwise the time needed would be indeterminate for participants with an asymptote lower

Effects of Age and Training—Practice on Rate of Approach

than the criterion level. As the lowest level was very low indeed (several people had asymptotes lower than . 1 at one or more of the

In accordance with the principles outlined in Verhaeghen et al.

early sessions), the Verhaeghen et al. (1997) technique was not

(1997), analyses on parameter b, which is the rate of approach,

applicable to the present data set. Likewise, an alternative tech-

were conducted on log-transformed scores. After

logarithmic

nique suggested by Verhaeghen et al. (1998; i.e., analyzing the

transformation, testing for an Age X Condition interaction be-

asymptote after logit transformation) could not be applied because

comes a test for deviations from a multiplicative model of aging.

many participants attained perfect asymptotes in the later sessions,

That is, we tested whether the young over old ratio of rates

therefore logit (c) would be indeterminate. Also, no matter how the

remained constant across testing occasions. This model is in ac-

data are transformed, this ceiling effect in later sessions makes

cordance with extant models of age-related slowing (e.g., Cerella,

conclusions from any type of ANOVA tentative at best.

1990), which state that central processing in older adults is slowed

Because none of the more sophisticated approaches could be

by a constant proportion as compared to processing in younger

applied, we decided to conduct a standard ANOVA on untrans-

adults. Given our assumption that perceptual factors would impact

formed c parameters. For the pretest-to-immediate-posttest con-

the onset time parameter and not the rate parameter, and given that

trast, a reliable main effect was found for testing occasion, F(l,

no motor response is executed during the encoding stage of this

33) = 71.25, MSE = 0.08, p < .001, and age, F(l, 33) = 7.06,

memory task, we assume that peripheral input-output processes

MSE = 0.06, p < .05, but not for the Age X Testing Occasion

play no role in the rate parameter. Hence, this parameter should

interaction, F(l, 33) = 2.16, MSE = 0.08, ns. This signifies that

thus conform to a strictly proportional model.

older adults had lower asymptotes than younger adults and that

For the pretest to immediate posttest contrast, only the effect of testing occasion was significant, F(l, 33) = 6.78, MSB = 18.86,

there was a marked positive effect of training on asymptotic accuracy.

p < .05. Neither the age main effect, F(l, 33) < 1, MSE = 13.80,

For the posttest-to-final-test portion of the learning curve, only

ns, nor the Age X Testing Occasion interaction, F(l, 33) = 1.39,

the age main effect was found to be significant, F(l, 33) = 12.48,

MSE = 18.86, ns, reached significance. Consequently, one effect

MSE

of instruction in the method of loci appeared in the form of

264) <

slowing. This slowing is equivalent across age groups, and

interaction, F(8, 264) = 1.05, MSE = 0.04, ns, reached signifi-

= 0.18, p

< .001.

Neither the testing occasion,

F(8,

1, MSE = 0.04, ns, nor the Age X Testing Occasion

younger and older adults do not differ in the rate of approach before and immediately after training.

1

For the posttest to final test sessions, a reliable main effect was found for age, F(l, 33) = 12.79, MSE = 3.52, p < .01. The Age X Testing Occasion interaction, F(8, 264) = 4.13, MSE = \M,p< .001, was also reliable, but not the testing-occasion main effect, F(8, 264) = 1.33, MSE = 1.84, ns. However, as can be seen in

One of the reviewers suggested that the Age X Session interaction in

the posttest-to-final-test segment may falsify general slowing theory, because the age relation of the parameters changes over sessions (see Cerella, 1990, and Myerson, Hale, Wagstaff, Poon, & Smith, 1990, for theories describing older adults' performance as a simple function of performance

speed increased again, apparently at the same rate in both age

of younger adults; and Salthouse, 1996, for a more mediator-oriented version of this view). We wish to point out, however, that these general slowing perspectives do not contain theoretical statements about the form of the change in age differences after instruction in a mnemonic; for instance, the theory makes no claim about whether older adults should learn a technique equally fast or slower than younger adults. This implies that there is technically no clear general slowing null hypothesis to falsify. Contrary to the reviewer, we think our data suggest that correspondence (i.e., the two groups of participants are applying the same processing steps in the same order; Cerella, 1990) might indeed be established, but only after the second posttest session, that is, after the older participants have reached the peak of mnemonic-induced slowing. From that point onward, the age difference in rate of processing appears to remain stable, that is, it

groups. The age difference in the posttest to final test segment was

conforms to the expectations of general slowing theory.

Figure 1, the slowing effect of instruction was not complete in older adults at the first posttest. Rather, older adults continued to slow down in the second posttest session.1 Therefore, we reanalyzed the data, conducting an ANOVA on the segment of the curve representing the second posttest to the final posttest. In this analysis, reliable effects were found for age, F(l,

33) = 35.06,

MSE = 2.19, p < .001, and testing occasion, f(7, 231) = 3.16, MSE = 0.75, p < .01, but not for the Age X Testing Occasion interaction, F(7, 231) = 1.40, MSE = 0.75, ns. Thus, after slowing down at the first and (for older adults) second posttest, elaboration

653

LEARNING A NEW ALGORITHM

14 12 O

10 Q. Q.

8 6

DC

4 Younger Adults Older Adults

2

0.0

0.2

0.4

0.6

0.8

1.0

Asymptote Figure 2.

Rate of approach as a function of asymptotic performance and

training session for younger and older adults. The two points in the left-hand portion of the graph represent performance in Session 3 (pretest); the arrow indicates the direction of time; successive sessions are connected with lines.

cance. However, as can be seen in Figure 1, the immediate effects of training in the old were not restricted to the first posttest. Rather, asymptotic performance continued to rise in the next session. Restricting the analysis to the portion of the learning curve from the second to the final posttest did not substantially alter the results of the ANOVA [age effect, F(l, 33) = 9.87, MSE = 0.18, p < .01; testing occasion effect, F(7, 231) < 1, MSE = 0.03, ns; Age x Testing Occasion interaction, F(7, 231) = 1.06, MSE = 0.18, ns]. Thus, older adults appear to have lower asymptotic levels of performance across continued practice with the method of loci than younger adults. [There seems to be a trend toward convergence of the two age groups in the asymptote parameters over sessions. To assess the size of this effect, we calculated the percentage of total variance associated with the interaction effect, using Levin's (1997) Equations 25 and 3, and we found this to be .06 (i.e., w2 = .00) for the portion of the curve going from posttest to final test portion of the curve and .0002 (i.e., cu2 = .00) for the portion extending from the second to the final posttest. Hence, this apparent convergence is associated with an extremely small effect size (Cohen, 1988, labeled w2 = .01 as a "small" effect)]. Graphical Analysis Figures 2 and 3 depict alternative ways of looking at the data concerning some of the parameters. Both of these figures reprise the data contained in Figure 1, but they represent them in different ways, emphasizing different aspects of our data. Figure 2 depicts two phase portraits, one for each age group. A phase portrait is a graph depicting change in a system over time; the system is depicted in an n-dimensional space (called state space) and is defined by n descriptive variables. Here, we show a twodimensional plot relating the rate of approach parameter to asymptotic performance. As can be seen in the graph, the two phase

portraits are remarkably similar. At first, a sizeable increase occurs in the asymptotic parameter because of instruction in the method of loci, while at the same time the rate of approach goes up (meaning that the research participants are getting slower at executing encoding operations). When the peak of slowing and of the rise in asymptotic performance is reached, the curve drops almost vertically, meaning that the asymptote stays constant while the rate of approach goes down. The lines are almost parallel for the two age groups, showing a high qualitative resemblance of the effects of instruction and continued practice. At the same time, the diagram also illustrates two important differences between the age groups, namely, first, that the starting point of the curves (i.e., the point depicting performance at pretest) is different; and second, that older adults need one session more to reach their peak of asymptotic performance. The similarity in shape combined with a different starting point Leads to the nonconvergence of age differences in the course of memory training. Figure 3 depicts the change over testing occasions of the youngold difference in the rate parameter in a Brinley plot. In such a plot (Brinley, 1965), average data of older adults are depicted as a function of average data of young adults. The difference between a traditional Brinley plot and Figure 3 is that Figure 3 contains an ordered sequence of points obtained on the same participants. The curve describing the change over time of the rate parameter seems to curl back onto itself. This suggests that, after the initial shock of applying the new mnemonic, participants return to their initial speed of encoding processes (and thus to the same initial age difference). This in turn suggests that the speed at which encoding processes are executed may well be a given within the cognitive system, that is independent of the algorithm used, at least after a sufficient amount of practice with the algorithm has been provided. In that sense, the basic speed of processing of the system works as a point attractor (e.g., Kelso, 1995), that is, after destabilization, the system returns to its initial settings, much as a spring that is

r CO

4

< 12

I g

a "o

a (0

rr

1

°

6 4

H

2 n J

0

2

4

6

8

10 12 14

. Rate of Approach (Younger Adults) Figure 3.

Phase portrait in young-old space of the change over training

in the method of loci of the rate-of-approach parameter for the timeaccuracy functions. The arrow indicates the direction of time; successive sessions are connected with lines.

654

VERHAEGHEN AND KLIEGL

stretched and then released will eventually return to its initial

due to the fact that no data points were sampled close to the onset time (minimum presentation time used for estimating the curves was 3 s). Nevertheless, the pattern is compatible with the hypoth-

Discussion

esis that after instruction in the method of loci participants not only read the word but also retrieved the locus, which they need to

In this study, older and younger adults were trained in the

generate the mental image. The decrease of the onset time over the

method of loci, and they received extensive practice with the

course of practice, up to the initial pretest level, suggests that over

method over a period of many months. The results were analyzed

practice, retrieval of the locus becomes more and more automatic.

at the level of parameters of the time-accuracy function, that is, the

For young adults, this transition may have occurred during the

function relating proportion of recall to the time each item has

instruction sessions, which were sufficient to establish the skill at

been presented.

a very high level. The reader should note that the finding that the

Our first goal was to assess the effects of training and practice

onset times for young and older adults do not differ at pretest is

in the method of loci on the parameters of the time-accuracy

consistent with all previous time-accuracy research on recall and

function. Training in an effective mnemonic, such as the method of

recognition of words from episodic memory (Verhaeghen, 1999;

loci, entails learning a new algorithm for encoding the to-be-

Verhaeghen et al., 1998). This result suggests that the lexical

remembered materials. As such, this intervention initially destabi-

access process has an identical time course for younger and older

lizes the system. This can be seen in the effects of training on the

adults— a result consistent with lexical access time-course esti-

parameters of the function: It slowed the participants down after

mates from spread-of-activation studies (for an overview, see

posttest (viz., the rate parameter is larger after training, and for the

Duchek & Balota, 1993) and at least some of the data on reading

older adults, the onset time increases as well), while at the same

times (Wingfield & Stine-Morrow, 2000).

time asymptotic performance is boosted. After this initial shock,

Our finding of a dramatic shift from an untrained to a trained

the main effect of continued practice is to speed up processing

state, followed by a gradual relaxation of the system into a more

(viz., the rate of approach goes down, and for the older adults, the

efficient way of executing the algorithm is in accordance with the

onset time decreases), while the asymptotic level of performance

correlational analysis on the same data set reported in Baltes and

remains relatively stable. Thus, two distinct mechanisms seem to

Kliegl (1992). This analysis showed that instruction disrupts the

be at work in instruction and practice, which have their impact on

rank order of participants, which then stayed quite consistent

different aspects of time-related performance. The method of loci

during continued practice in both age groups. We suspect that the

provides participants with a framework that allows both for effi-

maximum capacity of the system may well be a function of both

cient encoding of the individual words and for correct serial order

the learner and the task, therefore the transition from the individ-

at recall. This effectively increases the maximum capacity of the

ual's preferred strategy at pretest to the imposed strategy at post-

system for serial recall. However, this increase comes at the cost of

test may boost performance of certain individuals more than that of

slowing down the encoding processes, as evidenced by an in-

others. This is convergent with some evidence gathered by Ver-

creased rate of approach. Maybe the reason for this initial slowing

haeghen and Marcoen (1996). In that study, older adults who at

is that the assemblage of the components of a newly learned

pretest indicated that they linked words in a list using some

algorithm is typically an effortful process, occurring under delib-

associative technique had more trouble with applying the method

erate control (Duncan, 1986; Norman & Shallice, 1986). In the

of loci correctly. Likewise, in that study, the ability to recall paired

method of loci, some compilation of various components is indeed

associates, as measured with an independent test at pretest, had a

required: retrieving the locus, forming an image of the locus,

larger effect on posttest than on pretest performance. Thus, the

forming an image of the word, and combining these in an inter-

method of loci might be better suited to certain individuals than to

active fashion. After initial destabilization, the system speeds up

others, and this reorders participants. Once the method has been

again. Likely candidates for this speed-up include faster execution

learned, however, practice mostly has the effect of honing the

of the compilation process itself, increased automaticity in the

component skills, and there is no reason why this should disturb

execution of one or more of the components, maybe allowing for

the rank order of participants.

parallel processing, or the acquisition of a library of stock images

For the rate-of-approach parameter, significant age differences

(or of stock solutions for finding an interactive image) that can be

were found in the practice portion of the learning curve. Conse-

applied at particular loci. Note that this double effect of increased

quently, older adults are slower than young adults (at least during

asymptotic performance and first elevated and then decreasing rate

the period of extended practice with the mnemonic). This is in

of approach makes for a deceptively smooth learning curve at the

accordance with the bulk of the literature on aging and slowing

level of mean performance, in that the abrupt change in asymptotic

(e.g., Cerella, 1990; Salthouse, 1991, 1996) and with previous

performance is masked by the abrupt task-related slowing that

time-accuracy research that examined age differences in recall

occurs.

from episodic memory (Kliegl et al., 1998; Verhaeghen et al.,

An interesting finding is that the onset time increases for older

1998). This result points at the existence of a limited-time mech-

adults (but not for younger adults) immediately after training (i.e.,

anism (Salthouse, 1996) for encoding of information both when

during the first and second posttest) and then decreases quite

individuals apply their usual encoding strategies and when the

steadily (at a rate of about 20 ms per session). Estimates of the

method of loci is applied.

onset parameter should be interpreted cautiously, however. For

There was an age main effect in asymptotic performance, both

instance, we found that in the last few sessions, onset times were

in the pretest-to-posttest contrast and in the practice portion of the

effectively lower for older than for younger adults. This might

learning curve. Older adults are ultimately less accurate than

indicate a failure in precision of estimating the parameter, maybe

younger adults. Kliegl et al. (1998) and Verhaeghen et al. (1998)

LEARNING A NEW ALGORITHM have previously demonstrated such asymptotic age differences in recall from episodic memory, and the current research extends this finding to recall when both age groups are equated in their use of a highly efficient strategy. Thus, this is strong evidence for the robustness of an asymptotic age difference in recall from episodic memory. This is not an obvious result. It signifies that even when given an unlimited amount of time, and even after months of practice, older adults do not reach the same level of recall as younger adults when applying a mnemonic device that provides an excellent and consistent cueing device at both encoding and retrieval. Stated in Salthouse's (1996) terms, this implies that a simple limited-time mechanism of slowing is insufficient to explain adult age differences in recall from episodic memory. Of course, in the present experiment no special instructions occurred after the initial instruction sessions. New practice regimes or special coaching directed at increasing asymptotic levels late in the experiment may lead to further shifts of the asymptote. With regard to the change in age differences over training and practice, no Age X Testing Occasion interaction was found for either the rate of approach or the asymptote. Thus, within the limits of the statistical power of this analysis, the effects of instruction and practice on these parameters were identical for younger and older adults. The phase portrait depicted in Figure 2 offers a vivid description of this similarity: The change over time of the system described by the rate of approach and the asymptote is indeed remarkably similar in younger and older adults, as illustrated by the parallemess of the traces. This suggests a strong qualitative resemblance in the processes underlying performance changes as a function of instruction and practice in younger and older adults. (But note that older adults reach the peak in the graph only at the second testing occasion, showing that although the system described by the rate and asymptotic parameters is indeed similar in older and younger adults, the older system moves slower through state space.) There is a third, and quite exciting, illustration of the young-old parallelism. The trace of the rate parameter in Brinley space as depicted in Figure 3 reveals that, at least at the group level, the original age difference acts as an attractor state. In other words, over the course of training and practice, the rate of approach returns to its original setting (and hence to its original age difference), much as a spring returns to its original shape after contraction or expansion. This finding suggests that the speed of elaboration of encoding into episodic memory, after a strategy has become automatic, might well be a constant that is intraindividually stable. Summarized, it appears that the instruction in the method of loci has its primary effect in a lasting increase of the asymptotic level of recall and in temporary slowing of elaboration processes. In older adults, we observed the additional effect of increasing the minimum time needed for minimal recall. Older adults were found to be slower than younger adults and to have a lower level of asymptotic recall. These differences were largely preserved over training and practice after an initial warming-up period for older adults.

References Bates, P. B., & Kliegl, R. (1992). Further testing of limits of cognitive plasticity: Negative age differences in a mnemonic skill are robust. Developmental Psychology. 28, 121-125.

655

Bates, P. B., & Labouvie, G. V. (1973). Adult development of intellectual performance: Description, explanation, and modification. In C. Eisdorfer & M. P. Lawton (Eds.), The psychology of adult development and aging (pp. 157-219). Washington, DC: American Psychological Association. Bakes. P. B., & Schaie, K. W. (1976). On the plasticity of intelligence in adulthood and old age. American Psychologist, 31, 720-725. Bates, P. B., & Willis, S. L. (1982). Plasticity and enhancement of intellectual functioning in old age: Penn State's Adult Development and Enrichment Project (ADEPT). In F. I. M. Craik & E. E. Trehub (Eds.), Aging and cognitive processes (pp. 353-389). New York: Plenum. Baron, A.. & Cerella, J. (1993). Laboratory tests of the disuse account of cognitive decline. In J. Cerella, W. Hoyer, J. Rybash, & M. Commons (Eds.), Adult information processing: Limits on loss (pp. 175-203). San Diego, CA: Academic Press. Bellezza, F. S. (1987). Mnemonic devices and memory schemas. In M. A. McDaniel & M. Pressley (Eds.), Imagery and related mnemonic processes: Theories, individual differences, and Lipplications (pp. 34-55). New York: Springer-Verlag. Bellezza, F. S., & Reddy, B. G. (1978). Mnemonic devices and natural memory. Bulletin of the Psychonomic Society, 11, 277-280. Brinley, J. F. (1965). Cognitive sets, speed and accuracy of performance in the elderly. In A. T. Welford & J. E. Birren (Eds.), Behavior, aging and the nervous system (pp. 114-149). Springfield, IL: Thomas. Carlson, R. F., Kincaid, J. P., Lance, S., & Hodgson, T. (1976). Spontaneous use of mnemonics and grade point average. Journal of Psychology, 92, 117-122. Cerella, J. (1990). Aging and information processing rate. In J. E. Birren & K. W. Schaie (Eds,), Handbook of the psychology of aging (3rd. ed., pp. 201-221). San Diego, CA: Academic Press. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104, 268-294. De Beni, R., & Cornoldi, C. (1985). Effects of the mnemotechnique of loci in the memorization of concrete words. Acta Psychologica, 60, 11-24. De Beni, R., & Comoldi, C. (1988). Does the repeated use of loci create interference? Perceptual and Motor Skills, 67, 415-418. Denney, N. W. (1979). Problem solving in later adulthood: Intervention research. In P. B. Baltes & O. G. Brim (Eds.), Life-span development and behavior (Vol. 2, pp. 37-66). New York: Academic Press. Dosher, B. A. (1976). The retrieval of sentences from memory: A speedaccuracy study. Cognitive Psychology, 8, 291—310. Duchek, J. M., & Balota, D. A. (1993). Sparing activation processes in older adults. In J. Cerella, J. Rybash, W. Hoyer, & M. L. Commons (Eds.), Adult information processing: Limits on loss (pp. 384-422). San Diego, CA: Academic Press. Duncan, J. (1986). Consistent and varied training in the theory of automatic and controlled information processing. Cognition, 23, 279-284. Glenberg, A. M., Smith, S. M., & Green, C. (1977). Type I rehearsal: Maintenance and more. Journal of Verbal Learning and Verbal Behavior, 16, 339-352. Hardesty, A., & Lauber, H. (1956). Hamburg-Wechster-lntelligenztestfiir Erwachsene (HAWIE) [Hamburg Wechsler intelligence test for adults]. Bern, Switzerland: Huber. James, W. D. (1890). Principles of psychology (Vol. 1). New York: Holt. Kausler, D. H. (1991). Experimental psychology, cognition, and human aging (2nd ed.). New York: Springer-Yerlag. Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press. Kemp, S., & van der Krogt, C. D. (1985). Effect of visibility of the loci on recall using the method of loci. Bulletin of the Psychonomic Society, 23, 202-204. Kliegl, R. (1995). From presentation time to processing time: A psycho-

656

VERHAEGHEN AND KLTEGL

physics approach to episodic memory. In W, Schneider & F. E. Weinert

automatic control of behavior. In R. J. Davidson, G. E. Schwartz, & D.

(Eds.), Memory performance and competencies: Issues in growth and development (pp. 89-110). Mahwah, NJ: Erlbaum.

search and theory (Vol. 4, pp. 1-18). New York: Plenum.

Shapiro (Eds.), Consciousness and self-regulation:

Advances in re-

Kliegl, R-, & Baltes, P, B. (1987). Theory-guided analysis of mechanisms of development and aging through testing-the-limits and research on expertise. In C. Schooler & K. W. Schaie (Eds.), Cognitive functioning

Roediger, R L. (1980). The effectiveness of four mnemonics in ordering

and social structures over the life course (pp. 95-119). Norwood, NJ: Ablex.

Ross, J., & Lawrence, K. A. (1968). Some observations on memory

Kliegl, R., Krampe, R., Mayr, U,, & Liebscher, T. (1998). Testing the limits and multinomial modeling of age differences in proactive Interference: Evidence for a context encoding deficit. Manuscript submitted for publication.

Salthouse, T. A. (1991). Theoretical perspectives on cognitive aging.

Kliegl, R., Mayr, U., & Krampe, R. T. (1994). Time-accuracy functions for determining process and person differences: An application to cognitive aging. Cognitive Psychology, 26, 134-164. Kliegl, R., Smith, J., & Baltes, P. B. (1989). Testing-the-limits and the study of adult age differences in cognitive plasticity of a mnemonic skill. Developmental Psychology, 25, 247-256. Kliegl, R., Smith, J., & Baltes, P. B. (1990). On the locus and process of magnification of age differences during mnemonic training. Developmental Psychology, 26, 894-904. Labouvie-Vief, G. (1976). Toward optimizing cognitive competence in later life. Educational Gerontology, 1, 75-92. Lee, S. -S., & Edwards, P. (1981). Retrieval of abstract nouns as a function of mnemonic instructions and reading ability. Journal of Mental Imagery, 5, 117-126. Levin, J. R. (1997). Overcoming feelings of powerlessness in "aging" researchers: A primer on statistical power in analysis of variance designs. Psychology and Aging, 12, 84-106. Liu, Y. (1996). Queueing network modeling of elementary mental processes. Psychological Review, 103, 116-136. Lohman, D. F. (1989). Estimating individual differences in information processing using speed-accuracy models. In R. Kanfer, P. L. Ackerman, & R. Cudeck (Eds.), Abilities, motivation, and methodology (pp. 119-

recall. Journal of Experimental Psychology: Human Learning and Memory, 6, 558-567. artifice. Psychonomic Science, 13, 107-108. Hillsdale, NJ: Erlbaum. Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition. Psychological Review, 103, 403-428.. van Geert, P. (1993). A dynamic systems model of cognitive growth: Competition and support under limited resource conditions. In L. B. Smith & E. Thelen (Eds.), A dynamic systems approach to developmentApplications (pp. 265-331). Cambridge, MA: Bradford. Verhaeghen, P. (1999). The effects of age-related slowing and working memory on asymptotic recognition performance. Aging, Neuropsychology, and Cognition, 6, 201-213. Verhaeghen, P., Kliegl, R., & Mayr, U. (1997). Sequential and coordinative complexity in time-accuracy function for mental arithmetic. Psychology and Aging, 12, 555-564. Verhaeghen, P., & Marcoen, A. (1994). Production deficiency hypothesis revisited: Adult age differences in strategy use as a function of processing resources. Aging and Cognition, I, 323-338. Verhaeghen, P., & Marcoen, A. (1996). On the mechanisms of plasticity in young and older adults after instruction in the method of loci: Evidence for an amplification model. Psychology and Aging, 11, 164—178. Verhaeghen, P., Marcoen, A., & Goossens, L. (1992). Improving memory performance in the aged through mnemonic training: A meta-analytic study. Psychology and Aging, 7, 242-251. Verhaeghen, P., Marcoen, A., & Goossens, L. (1993). Facts and fiction about memory aging: A quantitative integration of research findings. Journal of Gerontology: Psychological Sciences, 48, P157-P171.

163). Hillsdale, NJ: Erlbaum. Mayr, U., Kliegl, R., & Krampe, R. (1996). Sequential and coordinative

Verhaeghen, P., & Salthouse, T. A. (1997). Meta-analyses of age-cognition

processing dynamics in iigural transformation across the life span.

relations in adulthood: Estimates of linear and non-linear age effects and

Cognition, 59, 61-90. McClelland, J. L. (1979). On the time relations of mental processes: An examination of systems of processes in cascade. Psychological Review,

Verhaeghen, P., Vandenbroucke, A., & Dierckx, V. (1998). Growing

86, 287-330. McElree, B., & Griffith, T. (1995). Syntactic and thematic processing in

structural models. Psychological Bulletin, 122, 231-249. slower and less accurate: The effects of age on time-accuracy functions for recall from episodic memory. Experimental Aging Research, 24, 3-19.

sentence comprehension: Evidence for a temporal dissociation. Journal of Experimental Psychology: teaming, Memory, and Cognition, 27,

Wickelgren, W. A. (1977). Speed-accuracy tradeoff and information pro-

134-157. Myerson, J., Hale, S., Wagstaff, D., Poon, L. W., & Smith, G. A. (1990). The information-loss model: A mathematical theory of age-related cog-

Wingfield, A., & Stine-Morrow, E. A. L. (2000). Language and speech. In

cessing dynamics. Acta Psychologica, 41, 67-85. F. I. M. Craik & T. A. Salthouse (Eds.), The handbook of aging and cognition (2nd ed., pp. 359-416). Mahwah, NJ: Erlbaum.

nitive slowing. Psychological Review, 97, 475-487. Newell, A., & Rosenbloom, P. S. (1981). Mechanisms of skill acquisition and the law of practice. In J. R. Anderson (Ed.), Cognitive skills and their acquisition (pp. 1-56). Hillsdale, NJ: Erlbaum. Norman, D. A., & Shallice. T. (1986). Attention to action: Willed and

Received July 21, 1998 Revision received March 27, 2000

Accepted March 28, 2000 •

Verhaeghen, P., & Kliegl, R.

racy (a time-accuracy function) is then fitted to the data, and the parameters describing this function are used for ... used to fit the data in the studies described above (assuming that chance level is zero) is p = c {1 — exp[(a —t)/b]} for t > a: and ...... Plasticity and enhancement of intellectual functioning in old age: Penn State's ...

913KB Sizes 1 Downloads 128 Views

Recommend Documents

r i p o.pdf
Ripo goes big in buenosaires woostercollective. ... Topten x ripo ilove graffiti de. Page 1 of 1. r i p o.pdf. r i p o.pdf. Open. Extract. Open with ... Displaying r i p o.pdf.

P !P !P !P !P -
Cisco. Cadence. Juniper Networks. Yahoo! Oracle. Burbank. Diridon/Arena. BART. Bay Trail. Existing. Planned. Ridge Trail. Connector Trail !P !P Planned BART Stations. Planned Silicon Valley Rapid. Transit BART Stations. SOUTH BAY LOOP TRAIL k Major E

S P E C I A L R E P O R T FAO/WFP CROP AND FOOD SECURITY ...
Dec 17, 2014 - otherwise indicated, material may be copied, downloaded and printed ...... In the dry season the animals are free to roam both day and night, moving around the fallow ..... wage rates for manual labour are lowest in Forest Guinea, the

S P E C I A L R E P O R T FAO/WFP CROP AND FOOD SECURITY ...
Dec 17, 2014 - Frequent food security monitoring activities must continue as the situation is very .... crop maintenance/growth (weeding, fencing, application of chemicals, etc.) ...... Figure 12: Guinea - Dashboard for Ebola Model (illustration).

C O M P A N Y R E S E A R C H | C o m p a n y U p d a te OUTPERFORM
Financial statement - Quarterly (Btm). 3Q15. 4Q15. 1Q16 ..... เตชะหัสดิน, CFA. Banking, Consumer Finance, Insurance 662-659-7000 ext. ... Renewable Energy.

Sólymos, P., Lele, S. R. & Bayne, E.
National Wildlife Research Centre, Ottawa, ... Foundation for Statistical Computing, Vienna, Austria. ... Third International Partners in Flight Conference (eds.

NEET-Answerkey-2017-Code-P-Q-R-S.pdf
May 7, 2017 - 39 4 4 2 1 84 2 3 3 3 129 2 4 4 2 174 3 3 1 B*. 40 4 2 3 1 85 2 1 1 3 130 4 2 ... Displaying NEET-Answerkey-2017-Code-P-Q-R-S.pdf. Page 1 of 1.

P R HOLAMBE_ Website_Faculty ECT Profile.pdf
Light Fidelity (Li-Fi): In Mobile Communication and. Ubiquitous Computing Applications", “Springer Book” “Advances in Computing. Applications”. Professional ...

BID NO. 1151 - AWARD LETTER - P & R PAPER SUPPLY CO.pdf ...
Page 3 of 12. Page 3 of 12. BID NO. 1151 - AWARD LETTER - P & R PAPER SUPPLY CO.pdf. BID NO. 1151 - AWARD LETTER - P & R PAPER SUPPLY CO.pdf.

xـ{p/ âh{ـp/T z اR qـ_tـ yة ~tـ urـلsкـ{¨_ اR qـ_tـ qـutـ ~إ ...
FICCI – Federation of Indian chambaers of commerce and industry –. xـ{pàz ~ •o z Tté X ોc .... tـ кોy - NASSCOM – National Association of Software and services ... wـ кો – BALCO – Bharat Aluminium Company Limited tـ fé{yé

P&P 2012 (scratch).pdf
BELTRAN PEREZ, SANTIAGO ZARAGOZA M SENIORS 17,5 9 NP - - - NP . SANZ POVAR, JORGE ZARAGOZA M JUNIORS 5,8 2 D - - - D . ASIN GARCIA, ANDRES LA PEÑAZA M BENJAMINE 26,5 14 NP - - - NP . PINO MIRANDA, FRANCISCO VALLE BENASQUE M INFANTILES 22,8 12 NP - -

C2L P&P sponsor benefits.pdf
Promotional materials would read: Penguins & Princesses brought to you by (YOUR NAME) ... Name/logo will appear in C2L enews for January & March.

P&P 2012 (hcp).pdf
35 RICKETTS BARILLAS, GUSTAVO RANILLAS URBAN M SENIORS 18,6 10 18,7 67 - - - 67 . BELTRAN PEREZ, SANTIAGO ZARAGOZA M SENIORS 17,5 9 ...

p. 372
earth's surface? 9. Particles of charge +86 µC, +48 µC, and -90 µC are placed in a line. The center one (the +48 µC charge) is. 1.5 m from each of the others.

GO(P)
The Secretar5r, Kerala Public Service Commission (with C.L). The Registrar, University ... 'fhe Secretary, Kerala State Electricity l3oard (with C.l,). 'lhe Managing ...

/P/ghi
Jul 31, 2013 - phone or a handheld computer suitably adapted to implement said method. ... mobile phones involves a user pressing a key on a keypad.

p ni"
s.2016. To: Chief, Education Program Supervisors. Education Program ... of the SRC all School Heads are advised to revise their School Report Card according ...

p. 372
What is the magnitude of the electric force of attraction between an iron nucleus (q = +26e) and its innermost electron if the distance between them is 1.0x10-12 ...

(S1Ds r ri r r, 1
Jan 5, 2016 - CSDO HOSTING OF 2016 REGIONAL SPORTS COMPETITIONS (STCAA). Date: ... Email: [email protected] I Website: ..... render services during weekends shall be given Service Credits and Compensatory.

Page 1 Page 2 FūR PIPE SIZES STAMP P IPE SIZE & RÄTING # “FDR ...
BLINDS} , PADDLE SPACERS, AND PADDLE BLANKS. (MÅ INTENANCE EL INDS } IN ACCIJRIJANCE W I TH API. STÅŅI]ARI) 59 Ở ÅRE : MATER I ÅLÅŠŤM ...