Measuring the Power of Learning.™

2015–16 CAASPP Post-Test Workshops: Connecting Assessments to Instruction Note: Selected Slides Only Selected slides taken from the May 19, 2016 Webcast Complete powerpoint and recorded webinar posted at http://www.caaspp.org/training/caaspp/index.html

Measuring the Power of Learning.™

Simplified Data Flow TOMS (Test Assignments and Settings)

Test Se4ngs & Basic Student Data

Test Delivery System Student Responses

CALPADS

Student Demographic Data

ORS Test Results

Scoring System

TOMS (Reporting)

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

2

Measuring the Power of Learning.™

Partial Results Available Now •  Test results are available three weeks after a student completes both parts—CAT and PT—of a content area. •  Test results are added nightly. Use Caution: The results available in this reporting system are partial and may not be a good representation of your school’s or LEA’s final aggregate results. These partial results are not appropriate for public release. Final data will be released publicly by the California Department of Education (CDE) by August 15, 2016. 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

3

Measuring the Power of Learning.™

What’s New in 2015–16 •  Additional features in the Online Reporting System (ORS) –  Access to the previous year’s results –  Assessment Target Reports –  Access to ORS test results by additional users

•  Access to Student Score Reports in the Test Operations Management System (TOMS) –  Redesigned Student Score Reports –  Student Score Reports in Spanish

•  Earlier release of score results 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

4

Measuring the Power of Learning.™

Available Reports Report Initial Student Test Results Partial Aggregate Data Partial Assessment Target Reports Student Score Reports PDF/paper Final Student Data File

LEA

School

Parent

P

P

ORS

ORS

P

P

ORS

ORS

P

P

ORS

ORS

P

P

P

TOMS†/Paper††

TOMS†/Paper

Paper

P TOMS

PDFs of the Student Score Reports will be available in TOMS. †† LEAs must forward or mail the copy of the CAASPP Student Score Report to each student’s parent/guardian within 20 working days of its delivery to the LEA. †

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

5

Measuring the Power of Learning.™

Test Results in the Online Reporting System (ORS)

CAASPP Portal: http://www.caaspp.org/ 6

Measuring the Power of Learning.™

Important Reminder The results available in this reporting system are partial and may not be a good representation of your school’s or LEA’s final aggregate results. As a real-time system, these results will change as additional data are received and relevant appeals and rescores are processed. Your school’s or LEA’s final aggregate results may be higher or lower than the partial results posted to this system and, therefore, are not appropriate for public release. Final data will be released publicly by the CDE in September.

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

7

Measuring the Power of Learning.™

Users with Access to Score Reports User Role

Score Report Access Level

LEA CAASPP coordinators

All students within the LEA

Test site coordinators

All students within their school or entity

Test administrators Test examiners Test administrators/test examiners

Students assigned to them via a roster by their test site coordinator

Educators

Students assigned to them via a roster by their test site coordinator

For additional information about user roles available within TOMS, refer to the 2015–16 User Role Guidance document on the CAASPP Portal. 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

8

Measuring the Power of Learning.™

Features and Reports in ORS •  •  •  •  •  •  • 

Home Page Dashboard Subject Detail Report Claim-Level Detail Report Assessment Target Report Student Listing Report Student Detail Report Manage Rosters Accessing the ORS Quick Start Guide is available at http://www.caaspp.org/administration/reporting/index.html

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

9

Measuring the Power of Learning.™

Home Page Dashboard

1

2

10

Measuring the Power of Learning.™

Home Page Dashboard: Select Test, Administration, and Enrollment Status Select Test

1 Select Administration

• 2014–15 data are final • 2015–16 are partial

Enrollment Status View

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

11

Measuring the Power of Learning.™

Home Page Dashboard: Select Grade and Subject 2

Test results are available for students who have completed both parts— CAT and PT.

Students are in the process of completing tests—results are not yet available. 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

12

Measuring the Power of Learning.™

Report Name

Subject Detail Report Legend

Show/Hide Comparison Data Updated Demographic Subgroups ExploraCon Menu Time Stamp

Show/Hide Columns

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

13

Measuring the Power of Learning.™

Available Demographic Subgroup •  • 

Same subgroups as reported to the CDE CAASPP public Web reporting site Based on data received from the California Longitudinal Pupil Achievement Data System (CALPADS) at the time of the demographic snapshot taken at the end of the selected testing window

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

14

Measuring the Power of Learning.™

Exploration Menu

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

15

Measuring the Power of Learning.™

Claim-Level Report Detail

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

16

Measuring the Power of Learning.™

Claims by Subject ELA/Literacy •  Reading •  Writing •  Listening •  Research/Inquiry

Mathematics •  Concepts and Procedures •  Problem Solving and Modeling & Data Analysis •  Communicating Reasoning

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

17

Measuring the Power of Learning.™

Assessment Target Reports Example: Mathematics Target Report

Assessment Target Report Quick Start Guide available at http://www.caaspp.org/administration/reporting/index.html 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

18

Measuring the Power of Learning.™

Assessment Target Reports (cont.) •  Indicator of strengths and weaknesses relative to the test performances as a whole of the group you are viewing •  Strengths and weaknesses do not imply proficiency or that a particular content standard has been met •  Show how a group of students performed on a target compared to their overall performance on the assessment

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

19

Measuring the Power of Learning.™

Assessment Targets (cont.) Example of an ELA Assessment Target Grade 3—Reading Claim #1 Assessment Target (Literary Text) WORD MEANINGS: Determine intended meanings of words, including words with multiple meanings (academic/tier 2 words), based on context, word relationships, word structure (e.g., common roots, affixes), or use of reference materials (e.g., beginning dictionary), with primary focus on determining meaning based on context and the academic (tier 2) vocabulary common to complex texts in all disciplines. 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

20

Measuring the Power of Learning.™

Assessment Targets (cont.) •  • 

• 

Performance on a particular Assessment Target is provided in an Assessment Target Report. The Assessment Target Reports –  are provided at the group level (e.g., classroom, LEA, roster, etc.); –  do not imply proficiency or that a particular content standard has been met; –  provide information regarding a group’s strengths and weaknesses relative to the test performance as a whole; and –  are based on statistical significance tests. For example, a group of eighth grade students might have exceeded expectations in mathematics, but their performance on “investigating patterns of association in bivariate data” might be relatively lower than their overall performance. –  An educator might make this an area of focus for these students. 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

21

Measuring the Power of Learning.™

Assessment Targets (cont.) •  Assessment targets reports are produced for all claims in ELA but only for Claim 1 for Mathematics

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

22

Measuring the Power of Learning.™

Assessment Target Performance Indicators

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

23

Measuring the Power of Learning.™

Considerations for Using Assessment Targets • 

•  •  • 

• 

Assessment target performance indicators (strengths and weaknesses) are only reported at the aggregate level (e.g., roster, grade, school, or LEA). Assessment targets are not appropriate for individual students. ELA and mathematics target reports are available for targets with at least 10 unique items. Assessment target information is more reliable for larger group sizes. –  Larger group sizes will also ensure more unique items contribute to the overall target summary. Assessment targets are group specific. –  Each group has its own unique overall total test performance level. –  Different groups will yield different overall performances. –  Performance comparisons across groups might not be appropriate. 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

24

Measuring the Power of Learning.™

Considerations for Using Assessment Targets (cont.) • 

Assessment target reports –  Should serve as a starting point in overall investigations of students’ strengths and weaknesses –  Are one of many sources of information that should be used to evaluate student performance

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

25

Measuring the Power of Learning.™

Student Listing Report

Sort

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

26

Student Detail Report Report Name

Measuring the Power of Learning.™

Legend for Claim Achievement Category

Summary

AddiConal details of the student’s performance

Performance on Claims

Average LEA and school scale scores for comparison Note: State-level scale score averages will not be available un3l formally released by the CDE.

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

27

Measuring the Power of Learning.™

Manage Rosters •  Create customized groupings of students within a school –  Example: School-level users can create a report that lists all of the students within a specific grade or in a particular classroom.

•  Use in combination with user roles (e.g., TA, TE, TA/TE, educators) to provide teachers with access to test results for their students

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

28

Measuring the Power of Learning.™

Student Score Report (SSR)

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

29

Measuring the Power of Learning.™

Redesigned Student Score Reports 1

5 6

2 3 7

8

4 30

Measuring the Power of Learning.™

Overall Score and Progress Elements of the Student Score Report

1

Overall score and progress for each content area

Additional information about the student’s overall score

31

Measuring the Power of Learning.™

Early Assessment Program (EAP) Grade Eleven Only Elements of the Student Score Report—Front Page

1

EAP College Readiness

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

32

Measuring the Power of Learning.™

Test Results from Multiple Years Elements of the Student Score Report—Front Page

2

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

33

Measuring the Power of Learning.™

Claim Score Information Elements of the Student Score Report—Front Page

3

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

34

Measuring the Power of Learning.™

Link to Test Score Guide Elements of the Student Score Report—Front Page

http://www.testscoreguide.org/ca/ An online resource to help parents/ guardians better understand their child’s CAASPP Student Score Report

4 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

35

Measuring the Power of Learning.™

Student Address Elements of the Student Score Report—Back Page

5

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

36

Measuring the Power of Learning.™

Letter from the State Superintendent of Public Instruction Tom Torlakson Elements of the Student Score Report—Back Page

6

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

37

Measuring the Power of Learning.™

Information About CAASPP Elements of the Student Score Report—Back Page

7

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

38

Measuring the Power of Learning.™

Option for Spanish Student Score Report

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

39

Measuring the Power of Learning.™

TOMS Dashboard

TOMS Student Score Report Downloadable Files Quick Start Guide available at http://www.caaspp.org/administration/reporting/index.html 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

40

Measuring the Power of Learning.™

Timeline for Reporting

41

Measuring the Power of Learning.™

When Will Scores Appear in ORS? • 

Partial scores and aggregate reporting: –  Online summative scores for a specific content area will be in ORS within three weeks of a student having completed the CAT and PT for that content area. –  Paper-pencil testing scores will be in ORS within five weeks of a student’s answer document being released for scoring.

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

42

Measuring the Power of Learning.™

When Will Initial Student Score Reports Appear in TOMS? •  Initial Student Score Reports will be provided as downloadable PDF files in TOMS one week after the student’s last score is reported in ORS. •  Initial Student Score Reports are created during the test administration window as the student completes the last assigned test.

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

43

Measuring the Power of Learning.™

Avoiding Scoring Delays • 

• 

ETS may place a temporary hold on scoring a student’s results to investigate and/or clarify information provided. For example: –  An SSID has multiple students’ test results. –  A student has more than one answer document or multiple test results for the same test. –  A student answer document was returned incomplete. –  The student record was not found in TOMS. If a temporary hold is placed on a student’s score, it will delay the reporting of results in ORS as well as the Student Score Reports and data files. 2015–16 Post-Test Workshops: Connecting Assessments to Instruction

44

Measuring the Power of Learning.™

Sample Reporting Timelines

2015–16 Post-Test Workshops: Connecting Assessments to Instruction

45