Paraprofessional-Led Phonological Awareness Training With Youngsters at Risk for Reading and Behavioral Concerns K AT H L E E N L . L A N E , T O D D F L E T C H E R , E R I K W. C A R T E R , C A R L O S D E J U D , AND JENNIFER DELORENZO

ABSTRACT

T

his study examined the efficacy of a paraprofessionalled supplemental early intervention for first-grade students with poor early literacy skills and behavioral concerns. The goal was to determine if (a) the relatively brief intervention was effective in improving phonological skills, and (b) improvements in academic skills would be accompanied by behavioral and social improvements. The results indicated that the students in the treatment condition experienced significant, lasting increases in phonological awareness and moderate improvement in word attack skills. However, significant collateral effects on social and behavioral performance were not observed. Limitations and directions for future investigation are offered.

S

TUDENTS WITH ANTISOCIAL BEHAVIOR ARE

characterized by persistent violations of social norms and expectations (Walker, Ramsey, & Gresham, 2004). These students frequently engage in aggressive, coercive, and noncompliant behaviors that not only alienate their typically developing peers but also serve to disrupt their instructional environment and impair the students’ relationships with teachers and other adults. Although most often noted for their behavioral concerns, students with antisocial behaviors typically struggle academically, as evidenced by their low rates of academic engaged time, poor academic performance, and limited skill sets (Anderson, Kutash, & Duchnowski, 2001;

266

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October, Pages 266–276

Mattison, Hooper, & Glassberg, 2002; Van Acker & Talbott, 1999). Specifically, these students often lack the requisite skills (e.g., phonemic awareness skills) necessary to meet the academic challenges upon initial school entry. Consequently, these youngsters struggle not only in the social and behavioral arenas but in academic areas as well—particularly in the area of reading (Anderson et al., 2001; Hinshaw, 1992, Lane, 1999, 2004). Deficits in reading are especially troublesome given that reading skills are the gateway to many other forms of learning (Lyon, 1996). Without intervention, behavioral and academic deficits are likely to broaden over time (Lane, Carter, Pierson, & Glaeser, in press; Nelson, Benner, Lane, & Smith, 2004; Walker et al., 2004). For example, aggression is likely to expand in scope from verbal noncompliance and caustic comments during primary grades to more serious forms of physical aggression as students enter the secondary grades. Similarly, deficits in early literacy skills such as phonemic awareness and decoding often develop into problems with fluency and comprehension. Thus, as students’ skill deficits expand over time, those deficits become increasingly less amenable to intervention efforts (Adams, Treiman, & Pressley, 2000; Kazdin, 1993; O’Shaughnessy, Lane, Gresham, & Beebe-Frankenberger, 2003). Consequently, it is not surprising that students with antisocial behavior patterns—in the absence of evidence-based interventions—experience academic underachievement, school dropout, and maladaptive relation-

ships with peers and adults. In extreme cases, students with antisocial behavior patterns may develop learning and behavior patterns that warrant the need for services under the label of emotional disturbance (ED) as defined by the Individuals with Disabilities Education Improvement Act (IDEIA; 2004). This finding is of particular concern given that students with ED experience highly negative outcomes relative to general education students and other students with high-incidence disabilities (e.g., learning disabilities; LD). For example, students with ED are more likely than their peers to be retained at a given grade level, fail to complete school, lack selfdetermination skills, and experience negative postsecondary outcomes (Bullis & Cheney, 1999; Carter, Lane, Pierson, & Glaeser, 2006; Wagner, Cameto, & Newman, 2003). The necessity of identifying early, effective interventions for these students is clear. Although a number of empirically validated tools and procedures—such as the Student Risk Screening Scale (SRSS; Drummond, 1994) and the Systematic Screening for Behavior Disorders (SSBD; Walker & Severson, 1992)—have been developed to identify students with antisocial behaviors who may benefit from intervention, the research and teaching communities are still debating how best to intervene with these youngsters. Should intervention efforts focus on behavioral deficits (e.g., externalizing behavior, such as aggression and noncompliance), academic deficits (e.g., poor early literacy skills), or both? Although few in number, some studies conducted to date have indicated that improving academic deficits can not only improve the academic performance of these students but also result in collateral effects on behavior (Ayllon & Roberts, 1974; Coie & Krehbiel, 1984; Lane, Fletcher, DeLorenzo, & McLaughlin, 2005; Lane, O’Shaughnessy, Lambros, Gresham, & Beebe-Frankenberger, 2001; Lane, Wehby, Menzies, Gregg, Doukas, & Munton, 2002). For example, Ayllon and Roberts (1974) conducted an academic intervention using an A-B-A-B design with 10 teacher-nominated, fifth-grade students with disruptive behavior patterns. The intervention, which was conducted by the teacher, focused on providing reinforcement for increased reading accuracy. The results revealed increased academic gains and decreased levels of disruptive classroom behavior. Coie and Krehbiel (1984) reported similar findings in their study of 40 socially rejected African American fourth graders. Students were randomly assigned to one of four conditions: academic skills training, social skills training, a combined training, or control. The academic intervention consisted of academic tutoring in reading and math two times per week, whereas the social skills intervention involved a validated social skills curriculum that was taught one time per week. Both interventions were conducted over a 6-month period. One key finding was that students who participated in the academic skills training condition experienced increased reading achievement, higher levels of task engagement, improved sociometric status, and lower levels of disruptive behavior. However, this study was limited by unequal intervention times and a social skills train-

ing condition that was not specific to the students’ acquisition deficits (Lane, 1999). More recently, Lane et al. (2001) conducted a supplemental early literacy intervention with seven first-grade students identified by their teacher as having poor early literacy skills, externalizing behaviors, and hyperactivity levels that placed them at risk for emotional and behavioral disorders. Students participated in Phonological Awareness Training for Reading (PATR; Torgesen & Bryant, 1994a) for three 30-min sessions per week for 10 weeks; training was conducted by a research assistant in a small-group format during the course of the traditional school day. Students increased their word attack and oral reading skills, with some gains maintained into follow-up phases, which were conducted 2 weeks following completion of the intervention. Although students showed progress, they did not improve at a rate proportionate to typically responding students (e.g., Fuchs, Fuchs, Hamlett, Walz, & Germann, 1993). For some students, these increases in academic skills were associated with decreases in disruptive classroom behavior and improved playground behavior. Because the intervention was implemented by a university research assistant, it remains unclear whether similar outcomes would have been obtained if the intervention had been implemented by school personnel. To address this limitation, Lane, Wehby, et al. (2002) conducted an academic intervention with seven first-grade students who had not been responsive to a schoolwide primary intervention plan. Using the SRSS and teacher reports of academic performance, the researchers identified students who evidenced a high risk for antisocial behavior and were in the bottom third of their class in early literacy skills. In addition to the districtwide literacy plan, students received a supplementary intervention (i.e., John Shefelbine’s Phonics Chapter Books; Shefelbine, 1998). Students met in small groups for 30-min sessions, 3 to 4 days a week over a 9-week period (15 hours total). All intervention sessions were led by the school site literacy coordinator and were implemented with a high degree of treatment fidelity. The results indicated that despite initial variability, students showed improvements in both decoding skills and frequency of disruptive behavior. Furthermore, the five students who completed the intervention demonstrated improvements in nonsense word fluency scores into follow-up phases, whereas disruptive classroom behaviors were absent. All but one student displayed lower levels of negative social interaction on the playground as well. Similar outcomes also were noted in a downward extension of Lane et al. (2005), this time conducted at the kindergarten level. Lane et al. conducted a supplemental academic intervention with three kindergarten students who were nominated by their teacher as having externalizing behaviors and limited early literacy skills. These youngsters participated in a 30-min PATR lesson 3 days a week over a 10-week period. Lessons were conducted by the classroom teacher during the regularly scheduled literacy block as supplemental curricular support for these students. Modest improvements in early lit-

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

267

eracy skills (i.e., onset fluency and letter naming fluency) and lower levels of disruptive classroom behavior were noted for participants. Two students also showed improvements in playground behavior. However, social validity data suggested that the teacher was not entirely satisfied with the intervention. Although she judged the intervention procedures to be acceptable and the outcomes positive, she did not view the intervention as an appropriate method of managing students’ externalizing behaviors. Although these studies collectively offered preliminary support to indicate that improved academic performance in early literacy skills is associated with decreases in disruptive behavior, certain methodological features warrant attention. First, most studies were conducted using single-case methodology, restricting the generalizability of the research findings. Moreover, investigations using group methodologies (e.g., Lane, 1999) found that improved academic skills were not associated with collateral improvements in behavior. However, behavioral performance was assessed using the SRSS, which may not have been as sensitive as direct observation in detecting changes in student behavior. In a study examining the effects of a prereading intervention on the reading and social behavior of 63 kindergarten students who were at-risk for ED and reading problems, Nelson, Stage, Epstein, and Pierce (2005) reported nominal behavioral improvements for students in the treatment group, with greater improvements demonstrated by students in the control group. Second, none of the previously mentioned studies evaluated the effectiveness of interventions involving paraprofessionals as interventionists. Given that teachers and other certified support staff (e.g., counselors, reading specialists) are often challenged by the task of providing more focused interventions to students in need of additional support, it is important to explore additional avenues for addressing this need. Moreover, schools are relying increasingly on paraprofessionals to provide academic and social support to students with disabilities (French, 2003; Giangreco, Edelman, Broer, & Doyle, 2001; White, 2004). This trend has been accompanied by debate about the proper role of paraprofessionals in educational contexts, with researchers recommending that paraprofessionals serve in supplemental—rather than primary— instructional roles (Ashbaker & Morgan, 2006; Carter & Kennedy, 2006; Giangreco & Broer, 2005; Giangreco, Halvorsen, Doyle, & Broer, 2004). Paraprofessionals have been employed successfully in this capacity in academic interventions with students with LD (e.g., O’Shaughnessy & Swanson, 2000; Vadasy, Sanders, Jenkins, & Peyton, 2002) and students struggling with reading (e.g., Miller, 2003; Savage & Carless, 2005). However, academic treatment outcome studies have not been conducted using paraprofessionals with students who have or are at risk for antisocial behavior or other emotional and behavioral problems. This absence of empirical research is especially concerning given that more than one third (38.4%) of paraprofessionals have reported serving students with emotional disturbance (Carlson, Brauen, Klein,

268

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

Schroll, & Willig, 2002) and paraprofessionals frequently assist in delivering academic interventions with students exhibiting behavioral challenges (Carter, Blakeslee, Brickham, & Pelsue, 2006; Maag, Vasa, & Reid, 1998). The question remains whether or not paraprofessional-led interventions can increase the academic skills and decrease the disruptive behavior patterns that characterize this population of students. We extended this line of inquiry by conducting a schoolbased academic study with first-grade students who were at risk for antisocial behavior, using paraprofessionals as interventionists. This study examined the efficacy of an evidencebased, supplemental early literacy program administered by a paraprofessional in improving both the academic and behavioral performance of students with limited early literacy skills and antisocial behavior.

METHOD Participants Participants were 24 first-grade students (18 boys, 6 girls) with poor early reading skills who were at risk for emotional and behavioral disorders based on teacher nominations (nomination procedures to follow). Twelve (50.00%) of the students were European American, 3 (12.50%) were African American, and 9 (37.50%) were Hispanic. Four students were receiving special education services (1 for learning disability, 3 for speech and language impairments), and none of the students had a clinical diagnosis according to the Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision (DSM–IV–TR; American Psychiatric Association, 2000), as reported in cumulative file records. Students ranged in age from 5.96 to 7.21 years of age (M = 6.59, SD = 0.31) at the onset of the study. Initially, 25 students were randomly assigned to intervention and control groups, with students receiving special education services dispersed between the two groups (intervention or control; see nomination procedures and inclusion criteria). However, one student moved from the school at the onset of the study; therefore, data on this student are not reported. Demographic information on intervention and control groups is displayed in Table 1. A chi-square analysis revealed no differences in gender by group (intervention vs. control), χ2(1, N = 24) = 1.40, p > .05. Similarly, t tests revealed no significant differences between groups with regard to (a) intellectual ability, as measured by a Wechsler Intelligence Scale for Children, third edition (WISC-III; Wechsler, 1991) screening; (b) attendance, as measured by the number of school absences; or (c) age. Participant Responsibilities. Three first-grade general education teachers and one paraprofessional at a suburban elementary school in a southwestern state were invited to participate. All three teachers were fully certified, with 2 to 10 years of teaching experience. The paraprofessional had been

TABLE 1. Demographic Characteristics of Participants by Group Treatmenta Measure

n

%

Age

Controlb

M

SD

6.70

0.27

n

%

All participantsc

M

SD

6.45

0.31

n

%

Ethnicity European

9

69.23

3

27.27

12

50.00

American African American Hispanic

1 3

7.69 23.08

2 6

18.18 54.55

3 9

12.50 37.50

2 11

15.38 84.62

4 7

36.36 63.64

6 18

25.00 75.00

Gender Female Male WISC-III Schools attended One Two Three Unknown Days absent

7 4 0 2

84.92

17.12

1.36

0.50

53.85 30.77 0.0 15.38

5 1 2 3 10.42

7.54

88.82

10.91

1.63

0.92

45.45 9.09 18.18 27.27

12 5 2 5 5.33

M

SD

6.59

0.31

86.71

14.44

1.47

0.70

7.93

6.26

50.00 20.83 8.33 20.83

2.84

Note. WISC-III = Wechsler Intelligence Scale for Children, 3rd ed. (Wechsler, 1991) screening measure includes Block Design and Vocabulary subtests to obtain an estimate of intellectual functioning. Schools attended refers to the number of elementary schools the students attended, with two or more schools attended considered a risk factor (Walker, Block-Pedego, Todis, & Severson, 2001). a n = 13. b n = 11. c N = 24.

employed for 5 years at the school. She had a high school diploma, had taken some community college coursework, and reported participating in several district-provided professional development opportunities. The school was participating in a series of studies with the University of Arizona across kindergarten through second-grade levels designed to improve the early literacy skills of general education students who were struggling in the areas of reading and behavioral performance. Teacher responsibilities for this study included (a) identifying potential student participants using a systematic screening procedure, (b) participating in assessment procedures at three time points, (c) attending a 2-hour training to learn the intervention procedures, (d) allowing students to participate in the intervention during the instructional day, (e) evaluating social validity at the conclusion of the study, and (f) conducting the intervention with the delayedtreatment control group in the spring of the same academic year. Teachers participated in the training activities so that they would be familiar with the intervention procedures conducted by the paraprofessional. Paraprofessional responsibilities included (a) participating in an initial training session to learn the intervention and review behavior management strategies, as well as 30-min trainings on a weekly basis over the course of the study; (b) conducting the intervention with two groups as part of her regular duties; (c) allowing univer-

sity research students to observe the sessions to collect treatment integrity data; and (d) evaluating social validity at the conclusion of the study. All four adults consented. Nomination Procedures and Inclusion Criteria. A 1-hour meeting was held with the participating teachers at the end of the first semester to identify potential student participants using a five-step, modified version of the SSBD (see Note 1). In brief, teachers were asked to nominate up to 12 first-grade students in each class who demonstrated externalizing or internalizing behaviors and who performed below average in reading ability. Teachers were given definitions, examples, and non-examples of externalizing (i.e., behavior problems directed toward the external social environment) and internalizing (i.e., behavior problems inwardly directed, away from the external social environment and representing a problem with self) behaviors based on the definitions provided in the SSBD. Teachers reviewed the behavior patterns of all students in their classes and then nominated up to 12 students who best fit either behavioral description. Teachers then rank-ordered the students from 1 (most exemplifies) to 12 (least exemplifies). Next, teachers grouped all students in their classes into two groups: average to high reading ability compared to classmates (Group 1) or below-average reading ability relative to classmates (Group 2). Up to 10 of the ranked

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

269

12 students in each classroom who best met the description of externalizing or internalizing behavior patterns and who fell into Group 2 in terms of reading ability were invited to participate. Teachers sent home parent letters and consent forms to the parents of these teacher-nominated students. All parents consented, and children were assented by project staff. Intervention Procedures Student participants (n = 25) were randomly assigned to either an intervention (n = 13) or a delayed-intervention control (n = 12) condition. However, one of the students in the control condition moved out of the school district at the onset of the study. Students in the intervention condition were then randomly assigned to three different groups (4 students in each group), each of which participated in the PATR structured, supplemental reading curriculum during the fall semester. This program was conducted as a supplement to the literacy plan that included a balanced approach to literacy instruction (e.g., exposure to core literature and explicit instruction in literacy skills). Each intervention group met 3 days a week for 30-min sessions over a 10-week period, for a total of 15 hours of PATR intervention. Students in the delayed-intervention control group were scheduled to receive the intervention in the spring of the same school year. Students in the control group remained in the general education setting while the remaining students participated in the intervention sessions. Students in the control condition participated only in the district literacy plan, which was taught by certified teachers. Phonological Awareness Training for Reading. The Phonological Awareness Training for Reading (PATR; Torgesen & Bryant, 1994a) is a supplementary early reading curriculum designed to promote awareness of words’ sound structure by helping students learn how spoken language is represented by letters. PATR contains four activities: (a) rhyming, (b) blending, (c) segmenting, and (d) reading and spelling. To ensure fidelity of the intervention across groups, 30 lessons containing these activities were prepared to accompany PATR (see Lane et al., 2001; O’Shaughnessy, 1999). After students participated in rhyming activities, they learned how to blend sounds to form words. Next, students learned how to segment the sounds that constitute words (e.g., /sh/ /i/ /p/). Moreover, segmenting was taught by identifying words with the same first sounds, then the same last sounds, and finally the same middle sounds. Students also were taught to identify the sound positions in a given word (e.g., the word is hat; which square says the /t/ sound?). The final lessons introduced letters to represent the sounds that constitute words. These activities focused on applying students’ phonological awareness skills to reading and spelling tasks. This program has met with demonstrated success for students with LD, students at risk for reading problems, and students with comorbid con-

270

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

cerns in the areas of reading and behavior (Torgesen & Bryant, 1994b). Please refer to the PATR technical manual for additional details regarding the scope and sequence of the curriculum as well as validation studies. Setting. Each intervention group met in a private room adjoining two of the three participating classrooms. The paraprofessional who worked with all three teachers implemented the entire intervention with all three groups. Lessons were conducted outside of the core reading block to ensure that students participated in the full literacy program. Training. A 2-hour training session was held with the teachers 2 months prior to school starting. A separate 2-hour training session was held with the paraprofessional because she was unable to attend the initial training session. Topics included an overview of reading development, components of effective reading programs, behavior management strategies, and practices with the lessons and accompanying materials. Sample lessons were modeled and then practiced. Corrective feedback was provided at the end of each practice activity. Project staff (i.e., master’s level research assistants) also participated in a separate training to learn the intervention strategies. Furthermore, these research assistants had themselves implemented the PATR curriculum lessons in prior studies. The research assistants provided ongoing training to the paraprofessional over the course of the school year by conducting 30-min meetings held weekly. During these meetings, treatment integrity data were discussed and feedback provided on both intervention implementation and behavior management strategies used. Treatment Integrity. Behavioral checklists containing intervention components were completed by the research assistants on a weekly basis to measure implementation fidelity for each group. The presence or absence of each intervention component was coded. At the end of the intervention phase, both session and component integrity data were computed. Nine (30%) to 10 (33.33%) lessons for each group were observed to monitor treatment integrity. Mean scores were quite high, with overall mean session scores of 95.60% (SD = 7.38%; range = 71.43%–100%) and mean component scores of 95.85% (SD = 0.17%; range = 76.67%–100%), with Component 8 (maintained appropriate, quick pace for lesson) having the lowest mean score. Dependent Measures and Data Collection Data on students’ academic and sociobehavioral performances were collected at three time points: preintervention, postintervention (i.e., immediately following the conclusion of the intervention), and 4-week follow-up. Student performance was evaluated from multiple perspectives using the following psychometrically sound approaches, which included stan-

dardized measures, curriculum-based measures, and direct observations. Test of Phonological Awareness. The Test of Phonological Awareness (TOPA; Torgesen & Bryant, 1994a) is a 20-item, nationally normed, standardized test designed to assess a student’s awareness of individual sounds within words. The test contains two subtests. In the first subtest (10 items), students are asked to identify which of three pictorially represented words ends in the same sound as a stimulus word. In the second subtest (10 items), students are asked to determine which of four words ends in a different last sound from the other three words. For the purposes of this investigation, the total standard score was determined. Test–retest reliability coefficients over an 8-week interval ranged from 0.69 to 0.77 (see Note 2). Nonsense Word Fluency. Nonsense word fluency (NWF) was assessed using the corresponding subtest of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Kaminski & Good, 1996), a standardized instrument designed to assess fluency of essential indicators of early literacy skills. The NWF subtest contains 20 one-min probes that may be administered individually to early-elementary-age students on a frequent basis to monitor progress. Prior to administering either of the curriculum-based measures, research assistants participated in a 2-hour training led by the first author and based on direct instruction principles. Research assistants learned how to administer and score the NWF probes. Once 90% accuracy was obtained, preintervention data were collected. At each of the three time points, six NWF probes were obtained and their average computed to obtain an estimate of performance. Direct Observations. Two measures were used to monitor student behavior in the classroom and on the playground. Total Disruptive Behavior (TDB), the classroom measure, refers to a set of behaviors that impede instruction and disrupt the learning environment. Examples include touching others’ property; being out of seat without permission; hitting, biting, or choking others; audible noises unrelated to instruction (e.g., yelling, cursing, criticizing); and noncompliance with teacher instructions. Negative Social Interaction (NSI), the playground measure, refers to a set of behaviors that disrupt ongoing play activities and involve any instance of physical (e.g., rough bodily contact, rough contact with objects or materials, physical pestering) or verbal aggression (e.g., bossy statements; name calling; statements of rejection; possessive statements; accusations; unkind statements; aggressive threats, taunts, or argumentative behavior). This definition is a modified version of the social engagement category included in the SSBD. Observation sessions were 10 min in length, conducted during recess and lunchtime activities.

Using duration recording procedures, TDB and NSI were assessed by starting a stopwatch when the target behaviors were occurring and stopping the watch when the behaviors ceased. Both TDB and NSI were converted to a percentage by dividing the elapsed time by 600 s (the length of the observation session) and multiplying by 100. Interobserver agreement (IOA) estimates were collected during 10% of the observation sessions across all three data collection time points (i.e., preintervention, postintervention, and follow-up). IOA scores ranged from 75% to 100% on both TDB and NSI measures, with mean scores exceeding 90%. Social Validity Intervention Rating Profile–15. The Intervention Rating Profile–15 (IRP-15; Martens, Witt, Elliott, & Darveaux, 1985) is designed to assess teachers’ perceptions of treatment acceptability. Teachers rate each of 15 statements pertaining to intervention procedures and outcomes (e.g., I liked the procedures used in this intervention) on a 6-point Likert-type scale (1 = strongly disagree to 6 = strongly agree). Total scores could range from 15 to 90, with higher scores indicating greater acceptability. Internal consistency reliability ranged from .88 to .98. Children’s Intervention Rating Profile. The Children’s Intervention Rating Profile (CIRP; Witt & Elliott, 1983) is a parallel, empirically validated instrument used to assess students’ perceptions of treatment acceptability. Students rate each of 7 items on a 6-point Likert-type scale (1 = I do not agree to 6 = I agree). After reflecting the negatively worded items, raw scores range from 7 to 42, with higher scores indicating greater acceptability. The CIRP was validated on more than 1,000 students in Grades 5 through 10 (coefficient α = 0.86). However, for the purposes of this study, the wording was altered slightly to facilitate the younger age group (Lane, 1999). Internal consistency reliabilities ranged from .75 to .89. Experimental Design and Statistical Analysis Due to student attrition between postintervention and followup data collection periods, data were analyzed in two phases. First, a series of analyses of covariance (ANCOVAs) using preintervention scores and participant IQs as covariates were used to identify significant differences between intervention (n = 13) and control (n = 11) groups on all outcome measures at postintervention. Second, follow-up data were analyzed separately for each group (intervention, n = 9; control, n = 10) using repeated-measures t tests comparing data from postintervention and follow-up time points to determine if changes were sustained 4 weeks after the intervention concluded. For all comparisons, the alpha level was set at .05. Effect sizes were calculated using a pooled standard deviation and the correlation between groups in the denominator (Lipsey, 1998).

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

271

RESULTS Changes in Student Performance Academic Performance. The results of an ANCOVA using standard scores from the TOPA indicated a significant effect for group, F(1, 19) = 6.58, p = .019, η2 = .257. Participants in the treatment group scored significantly higher at posttest (adjusted M = 84.63) than did participants in the control group (adjusted M = 76.40; ES = .71). Group means, standard deviations, and effect sizes are displayed in Table 2. However, the results of an ANCOVA using scores from the NWF did not reveal a significant effect for group, F(1, 20) = 1.75 , p = .200, η2 = .081. Participants in the treatment group, however, scored higher at posttest (adjusted M = 59.06) than did participants in the control group (adjusted M = 48.88), suggesting a moderate improvement in decoding skills (ES = .40). Social Performance. The results of an ANCOVA on NSI scores were not significant, F(1, 20) = 1.59, ns, η2 = .073, ES = −.04. Adjusted postintervention means of participants in the intervention group (adjusted M = 0.33) were slightly higher than control group means (adjusted M = 0.19). The negative effect size suggests that the intervention group demonstrated movement in the correct direction (i.e., decrease in negative social interactions). However, the difference in performance between the intervention and control groups was negligible. Behavioral Performance. The results of an ANCOVA on TDB scores were not significant, F(1, 20) = 0.86, ns, η2 =

.041. Adjusted postintervention means of participants in the intervention group (adjusted M = 3.53) were slightly higher than control group means (adjusted M = 2.04). Effect size calculations actually suggested a slight (.22) increase in total disruptive behavior for the intervention group relative to the control group. Effect Maintenance A series of paired t tests on each of the dependent measures was conducted between postintervention and follow-up time points to determine whether students’ performance was maintained over time. For the intervention group, significant increases between the two time points were not found on any measure except for Nonsense Word Fluency, t(10) = 2.83, p = .018, ES = −.11. The intervention group did not experience any significant changes on any of the dependent measures between time points. Thus, although academic progress was modest, the results were sustained for 4 weeks following the conclusion of the intervention condition. Treatment Acceptability Frequency tables and mean scores of IRP-15 and CIRP items and composite scores were examined to assess teacher, paraprofessional, and students’ perceptions of social validity. Overall, teachers rated the intervention as moderately favorable, as evidenced by a mean IRP-15 score of 56.0 (SD = 9.2; range = 44–65). Although the teachers indicated that they liked the procedures and felt that the intervention was beneficial for the students, they did not feel that the intervention—

TABLE 2. Group Means and Effect Sizes for Dependent Measures by Time Point Preintervention

Postintervention

Variable

M

SD

M

Adj. M

SD

TOPA Treatment Control

72.17 73.64

10.67 10.58

82.62* 77.00

84.63 76.40

12.33 8.50

NWF Treatment Control

21.21 23.54

19.09 26.26

57.65 50.54

59.06 48.88

28.66 21.65

NSI Treatment Control

4.50 4.12

4.86 4.22

0.35 0.17

0.33 0.19

0.42 0.25

TDB Treatment Control

5.52 5.80

7.30 5.56

3.28 2.33

3.53 2.04

4.43 4.14

Follow-up Pre–post ES

M

SD

84.78 83.30

15.56 17.26

64.12 60.56

38.56 22.28

1.78 0.52

5.15 1.46

7.96 6.15

9.12 9.45

Postfollow-up ES −.28

.71

−.11

.40

−.04

.41

.22

.11

Note. TOPA = Test of Phonological Awareness (Torgesen & Bryant, 1994a); NWF = Nonsense Word Fluency subtest of the Dynamic Indicators of Basic Early Literacy Skills (Kaminski & Good, 1996); NSI = Negative Social Interaction; TDB = Total Disruptive Behavior. *p < .02 between preintervention and postintervention.

272

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

as an academic intervention—was a good way to address the students’ behavioral concerns. The paraprofessional who implemented the intervention rated the intervention as moderately favorable, with a total IRP-15 score of 51. The paraprofessional stated, The intervention program was beneficial for students with medium to mild behavior problems. It was not as successful for those with extreme behavior (more violent) problems, and it took time away from other students in the group. Suggest that extreme (violent) behaviors be [handled] more one-on-one and not in a group situation. Research assistants’ treatment integrity ratings and field notes indicated that the pacing of instruction was often less than desirable, as the paraprofessional frequently stopped instruction to redirect students who were misbehaving. The mean CIRP score of 38.4 (SD = 3.0; range = 32–42) suggested that students judged the intervention favorably. The ratings suggested that students who received the intervention liked participating in the groups and felt that they had learned information that would help them do better in school.

DISCUSSION Students with emotional and behavioral disorders are recognized for academic, social, and behavioral characteristics that negatively influence their educational experiences. Although by no means definitive, preliminary research has provided evidence to suggest that increases in early literacy skills are associated with improved behavioral performance (Ayllon & Roberts, 1974; Coie & Krehbiel, 1984; Lane et al., 2001; Lane, Wehby et al., 2002). However, these earlier investigations have had some features that have limited generalizability, such as unequal intervention times (e.g., Coie & Krehbiel), limited outcome measures (Lane, 1999), problems with attrition (Lane, Gresham, & O’Shaughnessy, 2002) , and implementation by university research assistants rather than school personnel (Lane et al., 2001). The current study was conducted to extend and improve this line of inquiry by examining the effects and feasibility of a paraprofessional-led, supplemental early literacy program study with first-grade students with antisocial behaviors and poor early literacy skills. Students in the intervention condition demonstrated significant improvements in phonological skills, as measured by a standardized achievement test, with gains being sustained once the intervention had concluded. These outcomes are consistent with earlier investigations (Lane, 1999). Whereas students experienced moderate growth (ES = .40) on curriculumbased measures of word attack skills (NWF), significant increases were not detected, as in previous investigations (Lane et al., 2001; Lane, Wehby et al., 2002). The standardized test (TOPA) directly parallels the PATR program, closely

approximating the training conditions. The nonsense word fluency subtest, in contrast, can be considered a measure of near transfer. This may account for the differences in student performance. That is, it may be that students had only begun to generalize the skills acquired during the training condition, which also may explain why students actually showed significant decreases in word attack skills once the intervention concluded. Direct observations of social behavior on the playground suggested decreases in negative social interactions. This finding runs contrary to previous, research assistant–led interventions that used the same intervention procedures and worked with the same population of students (e.g., Lane et al., 2001). Similarly, direct observation of students’ total disruptive behavior in the classroom indicated that students did not show significantly lower levels of disruptive behavior in the classroom. Effect size data actually suggested that disruptive behavior was increasing slightly. As with the negative social interaction scores, there were no significant changes between the postintervention and follow-up time points, suggesting that mean levels of performance were sustained for the 4 weeks following the conclusion of the intervention. In sum, whereas students experienced improved early literacy skills, the overall results did not suggest clear collateral effects on behavior—a finding consistent with previous research by Nelson et al. (2005). It may be that paraprofessionals, though able to meet the academic task demands associated with small-group interventions, need additional training or support to manage student behavior. In a study conducted by Giangreco and Broer (2005), paraprofessionals report spending approximately one half of their time providing instruction and one fifth of their time providing behavioral support to students. Although teachers may informally address behavior management strategies in regular meetings with paraprofessionals (French, 2001), adequate and focused preservice, on-the-job, and professional development training opportunities are often not available to paraprofessionals (e.g., Riggs & Mueller, 2001). To most effectively use paraprofessionals in delivering supplemental interventions, schools must make concerted efforts to ensure that paraprofessionals are appropriately and effectively trained to use evidencebased practices. To do this, many schools will have to reconsider their current approaches to training, supervision, and professional development. Limitations and Future Directions This study shows limitations on four key features: brevity of the intervention, small sample size, student attrition, and limited information about the interventionist. First, this intervention consisted of 15 hours of supplemental support over a 10-week period. Although the intervention dosage actually exceeds many of the reading interventions conducted with students with, and at risk for, emotional and behavioral disorders (see the review by Lane, 2004), it may be that aca-

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

273

demic interventions of this type need to last longer to maximize impact. Furthermore, although the intervention did contain features necessary to draw validity inferences about intervention outcome (e.g., treatment integrity, social validity, and maintenance data), the maintenance data were rather limited, given that follow-up data were gathered only 4 weeks following the conclusion of the study and that only 19 of the 24 students had follow-up data. Second, this study involved relatively few participants for a group design study, thereby limiting statistical power to detect change. Although changes in several of the outcome measures were not statistically significant, effect size calculations did suggest that the intervention group made progress relative to the control group. The small sample size may have lacked the statistical power to identify significant growth, even though student performance was moving in the desired direction. Fuchs et al. (1993) have called for the inclusion of more incremental measures, rather than strict reliance on statistical testing. For example, it is possible for students to be making progress—just not at a level commensurate with their peers or at a level that achieves statistical significance. Third, concerns surrounding the small sample size are compounded by high rates of student attrition. Five students were lost between pre- and postintervention after moving out of the district or leaving the public school system to pursue home schooling. Attrition is a major limitation of this study, as it prevented the original analysis (repeated measures across three time points) from being conducted. However, high transiency rates are typical of this population (Walker et al., 2004). Despite the loss of participants, the results indicated that although many of the changes between preintervention and postintervention time points were not statistically significant, the gains that did occur were sustained—a finding that also parallels earlier work (Lane, 1999). Fourth, limited information about the paraprofessional was gathered (e.g., training in behavior management techniques). Future studies would be wise to examine the intervention outcomes in relationship to participant characteristics. It may be that paraprofessionals with greater expertise in applied behavioral analysis principles (e.g., reinforcement, behavioral momentum) could have more effectively managed the groups. This may have translated into more generalized academic gains, which could have then influenced behavioral and social performance. Despite these limitations, this study provided partial evidence to suggest that an early literacy intervention implemented by a paraprofessional can result in improvements in early literacy skills. However, these improvements were observed only on near transfer measures and were associated with only minimal decreases in disruptive behavior in the classroom. These outcomes, although modest, are promising on two fronts. First, it is encouraging to see that a relatively brief intervention aimed at improving phonemic awareness skills improved phonological skills. Although the social and

274

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

behavioral changes were not statistically significant, movement was in the desired direction. Second, it is also encouraging to see that paraprofessional-led, small-group instruction was moderately successful, particularly in light of the current trend of increasing reliance on paraprofessionals to provide services and supports to students with and without disabilities (Giangreco et al., 2001). However, inspection of the treatment integrity data suggests that the paraprofessional conducting these groups appeared to struggle with pacing and behavioral management strategies. Lack of skills in this area may have negatively influenced treatment outcomes. Future investigations would be wise to include additional paraprofessionals as interventionists to determine if paraprofessionals, as a whole, have sufficient training in behavior management strategies to facilitate desired pacing and successfully instruct this difficult-to-teach population. It is possible that additional weekly training in behavior management as well as the academic intervention could have ameliorated some of the less desirable aspects of this study. It may be that with proper training, paraprofessionals may constitute an effective resource for supporting students with limited early literacy skills and antisocial behavior patterns. Summary Because both behavioral and academic concerns broaden over time and become increasingly resistant to intervention efforts (Lane, 2004), it is clear that attention needs to be devoted to studying how to better serve students with, and at risk for, antisocial behavior in the general education setting. Too often, schools tend to “wait and see” (Foorman, Francis, Shaywitz, Shaywitz, & Fletcher, 1997). Rather than intervening at the earliest point of concern, schools tend to subscribe to a model in which they wait until these concerns expand to such a degree that they necessitate special education services (IDEIA, 2004). Instead, the field of emotional and behavioral disorders is encouraging continued exploration of proactive strategies for preventing the development of antisocial behavior by preventing academic deficits (Walker & Severson, I 1992). KATHLEEN L. LANE, PhD, is an assistant professor in the Department of Special Education at Peabody College of Vanderbilt University and an investigator in the Vanderbilt Kennedy Center. Her research interests include conducting academic and behavioral intervention for students with and at risk for emotional and behavioral disorders within the context of multi-tiered models of support. TODD FLETCHER, PhD, is an associate professor in the Department of Special Education, Rehabilitation, and School Psychology at the University of Arizona, Tucson. His research interests include the implications of legislation and policy for English language learners, assessment and instruction of language minority students with special educational needs, and special education reform in Latin America, specifically in Mexico and Chile. ERIK W. CARTER, PhD, is an assistant professor of special education in the Department of Rehabilitation Psychology and Special Education at the University of Wisconsin–Madison. His research addresses peer rela-

tionships, access to the general curriculum, and secondary transition services. CARLOS DEJUD, EdS, is a doctoral student in the Department of Special Education, Rehabilitation, and School Psychology at the University of Arizona. His research interests include nondiscriminatory assessment of culturally and linguistically diverse students and school-based interventions for children with emotional and behavioral disorders. JENNIFER DELORENZO earned her master’s degree at the University of Arizona, where she conducted school-based research with children with emotional disorders. Currently she is an oncology sales specialist in chemotherapy. Address: Kathleen Lane, Vanderbilt University, Peabody College, Department of Special Education, Nashville, TN 37203; e-mail: [email protected] NOTES 1. Details about the modified version of the SSBD used in this study are available from the first author on request. 2. At the time of this study, the Comprehensive Test of Phonological Processing (CTOPP; R. K. Wagner, Torgesen, & Rashotte, 1999) was not yet available. REFERENCES Adams, M. J., Treiman, R., & Pressley, M. (2000). Reading, writing, and literacy. In I. E. Sigel & K. A. Renninger (Eds.), Handbook of child psychology: Vol. 4. Child psychology in practice (pp. 275–355). New York: Wiley. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author. Anderson, J. A., Kutash, K., & Duchnowski, A. J. (2001). A comparison of the academic progress of students with EBD and students with LD. Journal of Emotional and Behavioral Disorders, 9, 106–115. Ashbaker, B. Y., & Morgan, J. (2006). Paraprofessionals in the classroom. Boston: Allyn & Bacon. Ayllon, T., & Roberts, M. (1974). Eliminating discipline problems by strengthening academic performance. Journal of Applied Behavior Analysis, 7, 71–76. Bullis, M., & Cheney, D. (1999). Vocational and transition interventions for adolescents and young adults with emotional or behavioral disorders. Focus on Exceptional Children, 31, 1–24. Carlson, E., Brauen, M., Klein, S., Schroll, K., & Willig, S. (2002). Study of personnel needs in special education: Key findings. Rockville, MD: Westat. Carter, E. W., Blakeslee, L., Brickham, D., & Pelsue, D. (2006). [Paraprofessional knowledge and tasks: Comparisons across the grade span]. Unpublished raw data. Carter, E. W., & Kennedy, C. H. (2006). Promoting access to the general curriculum using peer support strategies. Research and Practice for Persons with Severe Disabilities, 31, 284-292. Carter, E. W., Lane, K. L., Pierson, M., & Glaeser, B. (2006). Selfdetermination skills and opportunities of transition-age youth with emotional disturbance and learning disabilities. Exceptional Children, 72, 333–346. Coie, J., & Krehbiel, G. (1984). Effects of academic tutoring on the social status of low-achieving, socially rejected children. Child Development, 55, 1465–1478. Drummond, T. (1994). The student risk screening scale. Grants Pass, OR: Josephine County Mental Health Program. Foorman, B. R., Francis, D. J., Shaywitz, S. E., Shaywitz, B. A., & Fletcher, J. M. (1997). The case for early reading intervention. In B. Blachman (Ed.), Foundations of reading acquisition and dyslexia: Implications for early intervention (pp. 243–264). Mahwah, NJ: Erlbaum. French, N. K. (2001). Supervising paraprofessionals: A survey of teacher practices. The Journal of Special Education, 35, 41–53. French, N. K. (2003). Paraeducators in special education programs. Focus on Exceptional Children, 36, 1–16.

Fuchs, L. S., Fuchs, D., Hamlet, C. L., Walz, L., & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27–48. Giangreco, M. F., & Broer, S. M. (2005). Questionable utilization of paraprofessionals in inclusive schools: Are we addressing symptoms or causes? Focus on Autism and Other Developmental Disabilities, 20, 10–26. Giangreco, M. F., Edelman, S. W., Broer, S. M., & Doyle, M. B. (2001). Paraprofessional support of students with disabilities: Literature from the past decade. Exceptional Children, 68, 45–63. Giangreco, M. F., Halvorsen, A., Doyle, M. B., & Broer, S. M. (2004). Alternatives to overreliance on paraprofessionals in inclusive schools. Journal of Special Education Leadership, 17, 82–90. Hinshaw, S. P. (1992). Externalizing behavior problems and academic underachievement in childhood and adolescence: Causal relationships and underlying mechanisms. Psychological Bulletin, 111, 127–155. Kaminski, R. A., & Good, R. H. (1996). Toward a technology for assessing basic early literacy skills. School Psychology, 25, 215–227. Kazdin, A. (1993). Treatment of conduct disorders: Progress and directions in psychotherapy research. Development and Psychopathology, 5, 277– 310. Lane, K. L. (1999). Young students at risk for antisocial behavior: The utility of academic and social skills interventions. Journal of Emotional and Behavioral Disorders, 7, 211–223. Lane, K. L. (2004). Academic instruction and tutoring interventions for students with emotional/behavioral disorders: 1990 to present. In R. B. Rutherford, M. M. Quinn, & S. R. Mathur (Eds.), Handbook of research in emotional and behavioral disorders (pp. 462–486). New York: Guilford Press. Lane, K. L., Carter, E. W., Pierson, M.R., & Glaeser, B.C. (2006). Academic, social, and behavioral characteristics of high school students with emotional disturbance and learning disabilities. Journal of Emotional and Behavioral Disorders, 14, 108–117. Lane, K. L., Fletcher, T., DeLorenzo, J., & McLaughlin, V. (2005). Improving early literacy skills of young children at-risk for antisocial behavior: Collateral effects on behavior. Manuscript submitted for publication. Lane, K. L., Gresham, F. M., & O’Shaughnessy, T. E. (2002). Serving students with or at-risk for emotional and behavior disorders: Future challenges. Education and Treatment of Children, 25, 507–521. Lane, K. L., O’Shaughnessy, T., Lambros, K. M., Gresham, F. M., & BeebeFrankenberger, M. E. (2001). The efficacy of phonological awareness training with first-grade students who have behavior problems and reading difficulties. Journal of Emotional and Behavioral Disorders, 9, 219– 231. Lane, K. L., Wehby, J. H., Menzies, H. M., Gregg, R. M., Doukas, G. L., & Munton, S. M. (2002). Early literacy instruction for first-grade students at-risk for antisocial behavior. Education and Treatment of Children, 25, 438–458. Lipsey, M. W. (1998). Design sensitivity: Statistical power for applied experimental research. In L. Bickman & D. J. Rog (Eds.), Handbook of applied social research methods (pp. 39–68). Thousand Oaks, CA: Sage. Lyon, G. R. (1996). Learning disabilities. In E. Marsh & R. Barkley (Eds.), Child psychopathology (pp. 390–434). New York: Guilford Press. Maag, J. W., Vasa, S. F., & Reid, R. (1998). Frequency of interventions used by paraeducators with students with emotional and behavioral disorders. Psychological Reports, 82, 1121–1122. Martens, B. K., Witt, J. C., Elliott, S. N., & Darveaux, D. (1985). Teacher judgments concerning the acceptability of school based interventions. Professional Psychology: Research and Practice, 16, 191–198. Mattison, R. E., Hooper, S. R., & Glassberg, L. A. (2002). Three-year course of learning disorders in special education students classified as behavioral disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 1454–1461. Miller, S. D. (2003). Partners-in-reading: Using classroom assistants to pro-

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007

275

vide tutorial assistance to struggling first-grade readers. Journal of Education for Students Placed at Risk, 8, 333–349. Nelson, J. R., Benner, G. J., Lane, K., & Smith, B. W. (2004). An investigation of the academic achievement of K–12 students with emotional and behavioral disorders in public school settings. Exceptional Children, 71, 59–73. Nelson, J. R., Stage, S. A., Epstein, M. H., & Pierce, C. D. (2005). Effects of a prereading intervention on the literacy and social skills of children. Exceptional Children, 72, 29–45. O’Shaughnessy, T. E. (1999). Phonological awareness training. (Available from the Department of Counseling and Psychological Services, Georgia State University, University Plaza, Atlanta, GA 30306) O’Shaughnessy, T. E., Lane, K. L., Gresham, F. M., & Beebe-Frankenberger, M. E. (2003). Children placed at risk for learning and behavioral difficulties: Implementing a schoolwide system of early identification and prevention. Remedial and Special Education, 24, 27–35. O’Shaughnessy, T. E., & Swanson, L. (2000). A comparison of two reading interventions for children with reading disabilities. Journal of Learning Disabilities, 33, 257–277. Riggs, C. G., & Mueller, P. H. (2001). Employment and utilization of paraeducators in inclusive settings. The Journal of Special Education, 35, 54–62. Savage, R., & Carless, S. (2005). Learning support assistants can deliver effective reading interventions for “at risk” children. Educational Research, 47, 45–61. Shefelbine, J. (1998). Phonics chapter books 1–6: Teachers guide. New York: Scholastic.

Torgesen, J. K., & Bryant, B. R. (1994a). Phonological awareness training for reading. Austin, TX: PRO-ED. Torgesen, J. K., & Bryant, B. R. (1994b). Test of phonological awareness. Austin, TX: PRO-ED. Vadasy, P. F., Sanders, E. A., Jenkins, J. R., & Peyton, J. A. (2002). Timing and intensity of tutoring: A closer look at the conditions for effective early literacy tutoring. Learning Disabilities Research & Practice, 17, 227–241. Van Acker, R., & Talbott, E. (1999). The school context and risk for aggression: Implications for school-based prevention and intervention efforts. Preventing School Failure, 44, 12–20. Wagner, M., Cameto, R., & Newman, L. (2003). Youth with disabilities: A changing population. Menlo Park, CA: SRI International. Wagner, R. K., Torgesen, J. K., & Rashotte, C. A. (1999). Comprehensive test of phonological processing. Austin, TX: PRO-ED. Walker, H. M., Ramsey, E., & Gresham, F. M. (2004). Antisocial behavior in school: Evidence-based practices (2nd ed.). Belmont, CA: Wadsworth. Walker, H. M., & Severson, H. (1992). Systematic screening for behavior disorders: Technical manual. Longmont, CO: Sopris West. Wechsler, D. (1991). Wechsler intelligence scale for children–Third edition. San Antonio, TX: Harcourt Brace Jovanovich. White, R. (2004). The recruitment of paraeducators into the special education profession: A review of progress, select evaluation outcomes, and new initiatives. Remedial and Special Education, 25, 214–218. Witt, J. C., & Elliott, S. N. (1983, August). Assessing the acceptability of behavioral interventions. Paper presented at the annual meeting of the American Psychological Association, Anaheim, CA.

Call for Manuscripts Topics in Early Childhood Special Education TECSE publishes 5 types of manuscripts: • • • • •

reports of original research literature reviews conceptual statements position papers program descriptions

The journal is published quarterly: three topical issues and one nontopical issue. Topical

issues address an identified problem, trend, or subject of concern and importance to early intervention. TECSE accepts articles for review on a continual basis through its online submission site; therefore, authors do not have to make or submit multiple copies.

Complete author guidelines may be obtained from the online submission site:

https://mc.manuscriptcentral.com/tecse Go to the gray Resources box and select the Instructions & Forms link.

276

R E M E D I A L

A N D

S P E C I A L

E D U C A T I O N

Volume 28, Number 5, September/October 2007