Achieve3000 and 1:1 – A Year in Review A review and analysis of the implementation of ipads and Achieve3000 at R. Middle School for the 2013-2014 school year. Introduction As the educational computing strategist over the past few years at a middle school, I have helped setup, train teachers, troubleshoot issues, and monitor progress of several online math and language arts programs. I have become familiar with Scholastic’s Read180, System44 and FasttMath, iLearn’s iPass Math, Renaissance’s Accelerated Reader and more. These programs (and others like them) have become an integral part of school’s curriculum repertoire and they provide a significant amount of data that can help teacher’s identify students ability, progress and gaps in learning. At R middle school this year, we have implemented an online language arts program called Achieve3000 where students read articles and complete activities and quizzes in order to improve their reading and writing abilities. Students are pretested by the program and assigned a starting Lexile score. The program recommends that students accomplish a goal of 40 articles and activities per school year in order for them to achieve the best results. R middle school is composed majorily of students who speak English as a second language and free (or reduced) lunch recipients. We are categorized as a Title1 school. According to last year’s CRT’s, only ____% are proficient in Reading and _____% exceed standards. The majority of our students need to “catch up” to where they should be for their grade level in reading fluency and comprehension. (Math results at R are similar). Instruction is differentited for the students as the reading level of their articles and activities corresponds to their lexile score. Students are assigned a lexile score from the pretest at the beginning of the year and it is modified each month throughout the school year as they complete new assessments. As a middle school, our students should have a beginning lexile score of about 600 for 6th graders, 700 for 7th graders, and 800 for 8th graders. The goal of the program is for students to achieve a lexile score of at least 1300 by the time they graduate high school.
Quotes from the Achieve3000.com website
The average starting lexile of our 6th graders in October was 485 (near 5th grade level); seventh graders scored an average of 316 (close to 3rd grade level)* and 8th graders started at 527 (slightly above 5th grade level). These pretest lexile scores correspond with the students’ CRT scores. On average the students are one to two years behind where they should be. We chose the Achieve3000 program for the purpose of helping our students close the achievement gap and catch up to where they need to be.
Quote from the Achieve3000.com website * There were reports from our 7th and 8th grade science teachers that several students “guessed” their way through the pretest – resulting in much lower starting lexiles than they actually had. Unfortunately, this will skew reported results. Ideally all results would be perfectly accurate - we look at these results with the understanding that some students did not start out at an accurate level. However the reporting system of Achieve3000 allows for reports run by using the monthly lexile assessment adjustment and not the Pretest level. When paramaters of the reports are changed, note will be made. The administration of the pretest is extremely important in creating a starting point for the year and ideas of improvement for pretest administration are being discussed for next year.
The use of Achieve3000 at R** middle school was distributed in the following way: Students were to use Achieve3000 once a week in their science class and twice per week in their Social Studies classes. 8th graders also used the program once per week in their Health class (a one semester course). And the program was used in Fundamentals ELA classes which are classes for students who scored low on their Reading CRTs and need a remedial class instead of an elective course. This distribution of usage led to an uneven amount of usage per grade level. Not including students in fundamentals classes, 6th graders were assigned only 1 article per week (as they do not have a Social Studies class). 7th graders were assigned 2 articles, and 8th graders were assigned 2 articles for one semester, then 3 articles per week during the semester that they also had health. Results Reports from Achieve3000 show the following results for the school year 2013-2014: 6th graders started the year with a lexile of 457 and ended with a lexile of 519 – an increase of 62 lexile points. 7th graders started the year with a lexile score of 575 and ended with 594 – an increase of only 19 lexile points. 8th graders started the year with a lexile of 679 and ended with 726 – an increase of 47 lexile points. Achieve3000 states that a growth in lexile levels from 50-70 is average for the school year. However, we are not looking for average, we are looking for our students to catch up to where they should be for their grade level. It appears from these results that the 6th graders achieved an average growth outcome, the seventh graders performed dismally this year, and the 8th graders increased in lexile scores steadily each month, with a sharp downturn in May – the month the posttest was administered. However, we need to dig deeper into the data provided to understand these results more clearly – as they are “just an average” of each grade level.
R. Middle School Lexile Growth Report 2014-2015 - All Students 900 800 700 600 500 6th
400 300
7th
200
8th
100 0
Chart 1
We will begin delving deeper into the 6th grade results. 6th graders were only assigned to use the program once per week. Most of the students did not fulfill the goal of completing 40 Achieve3000 lessons for the school year – 33 students did. We will look at the results between students who accomplished the 40-lesson goal and those who didn’t. (see Chart 2)
6th Grade Sub-Group Results Students who did not complete 40 Students who completed 40 (33 students) 537 464 364
September
354
May
Chart 2
The results in Chart 2 indicate that students who did not complete 40 lessons did better in the program than those who did. Students without 40 lessons gained an average of 73 lexile points. Students who did complete the 40 lessons lost an average of 10 lexile points. Allowing a small margin of error from inaccuracies in the pre or post test, I would judge the results as having no improvement in their lexile level for the current school year. This does not look good for the effectiveness of the program we have chosen. But who are these students? It turns out that these students are Ms. H’s resource students who do not attend a regular education reading class. They have been pulled from that and put into a class with Ms. H where they use the remedial online program Scholastic’s System44. While it is disappointing that on average they had no growth in lexile by using the Achieve3000 program, most of these students are dealing with learning disabilities and we will not evaluate the effectiveness of the program for the whole school by their results. Perhaps it is not the program best used for these students. From the 6th grade students who used the program once a week in their science class, we see an increase of 73 lexile points, which is slightly more than we would expect to see from students who were not using the program (according to Achieve3000). From the results of chart 1 we see that on average 7th graders only received an increase of 19 lexile points! Not only does this reflect poorly on the Achieve3000 program, but also R. Middle School as a whole – as students should achieve at least a 50 point lexile increase throughout the course of any given school year no matter what curriculum programs are used. Hopefully more specific Achieve3000 administrative reports can shed some light on what has happened here.
The results of a comparison between 7th graders who achieved the 40 lessons and those who did not, affords some interesting findings (see Chart 3). Again, there were few students in the 7th grade who completed 40 lessons and the same as 6th grade, they were the resource students using remedial programs such as System 44 and READ180. However, unlike the 6th graders, these students showed a significant lexile score improvement of 40 points. The big concern is why did those who did not completed 40 lessons only achieve an increase of 14 lexiles? First of all, these students should have easily completed 40 lessons as they were supposed to do two lessons a week – one in Social Studies and one in Science. According to the Achieve3000 usage reports, 7th graders completed 0.6 of a lesson each week. In order to have reached the usage goal, students should have completed at least 0.8 of a lesson each week. The 7th grade science and social studies teachers did not incorporate the program into their lessons as often as they were instructed to. Another reason for the lack of appropriate lexile level growth for the 7th graders may have been due to administration of the pretest. The pre-test was administered in science classes and the 7th and 8th grade teachers reported that several of the students merely “guessed” their way through the test and were assigned a lexile score way below where the student was actually at. The teachers took it upon themselves to adjust their pretest level and set it at a level they thought was more accurate for those students. It is possible that this raised the beginning lexile score and therefore, the small lexile gain for 7th graders may have actually been larger.
7th Grade Sub-Group Results Students who did not complete 40 Students who completed 40 (45 students) 617
603
442 302
September
May
Chart 3 Rest of paper is still under construction….
8th grade…
Recommendations
Start at the beginning of the school year Emphasize importance of pretest and reassign test if results are below
