Comparing Estimates of Teacher Value-Added Based on Criterion- and Norm-Referenced Tests

Issue/Topic: Teaching Quality--Evaluation and Effectiveness
Author(s): Stuit, David; Berends, Mark; Austin, Megan; Gerdeman, R.
Organization(s): Institute of Education Sciences; REL Midwest
Publication: REL Midwest
Published On: 1/1/2014

Recent changes to state laws on accountability have prompted school districts to design teacher performance evaluation systems that incorporate student achievement (growth) as a major component. Some states and districts are considering teacher value-added models as part of teacher evaluations.

This study aimed to provide new information on the degree to which value-added estimates of teachers differ by the assessment used to measure their students' achievement growth.

  • Single-year estimates from the state assessment (ISTEP+) and norm-referenced test (MAP) were moderately correlated (correlation coefficients of 0.44 to 0.65).

  • On average, 33.3 percent of estimates ranked in the same quintile on both tests in the same school year.

  • No teachers had estimates above the sample average with 95 percent confidence on one test and below the sample average with 95 percent confidence on the other test.

Policy Implications/Recommendations:
  • Designers of performance evaluation systems need to understand the factors that can affect the validity and reliability of value-added results or other measures based on student assessment data used to evaluate teacher performance.

  • Policymakers need to consider measurement error when interpreting estimates of teacher value-added, particularly single-year estimates.

  • Incorporating confidence intervals for value-added estimates reduces the likelihood that teachers' performance will be misclassified based on measurement error.

  • Other research has shown that averaging estimates of teacher value-added over multiple years will increase precision and provide more reliable information on teacher performance.

Research Design:
Comparison of teacher value-added based on districts that administered the criterion-referenced Indiana Statewide Testing for Educational Progress Plus (ISTEP+) and the norm-referenced Measures of Academic Progress (MAP) in the same school year. The analysis examines reading and math achievement data.

Students and their reading and math teachers in grades 4 and 5 in 46 Indiana schools in 10 Indiana districts

Year data is from:
2005-06 to 2010-11


Data Collection and Analysis:
The study used three analytic strategies to quantify similarities and differences in estimates of teacher value-added from ISTEP+ and MAP: correlations of value-added estimates based on the two assessments, comparisons of the quintile rankings of value-added estimates of the two assessments, and comparisons of the classifications of value-added estimates on the two assessments according to whether their 95 percent confidence intervals were above, below or overlapping the sample average.


Reference in this Web site to any specific commercial products, processes or services, or the use of any trade, firm or corporation name is for the information and convenience of the public, and does not constitute endorsement or recommendation by the Education Commission of the States. Please contact Kathy Christie at 303-299-3613 or kchristie@ecs.org for further information regarding the information posting standards and user policies of the Education Commission of the States.