How Teacher Evaluation Methods Matter for Accountability: A Comparative Analysis of Teacher Effectiveness Ratings by Principals and Teacher Value-Added Measures

Issue/Topic: Teaching Quality--Evaluation and Effectiveness
Author(s): Harris, Douglas; Ingle, William; Rutledge, Stacey
Publication: American Educational Research Association
Published On: 2/14/2014

Policymakers are revolutionizing teacher evaluation by attaching greater stakes to student test scores and observation-based teacher effectiveness measures, but relatively little is known about why they often differ so much.

To find out why teacher value-added measures differ from principals' impressions of effectiveness

  • Quantitative analysis suggests that teacher value-added measures and informal principal evaluations are positively, but weakly, correlated.

  • Qualitative analysis suggests that some principals give high value-added teachers low ratings because the teachers exert too little effort and are "lone wolves" who work in isolation and contribute little to the school community.

  • While the correlation between the simple numeric principal ratings and teacher value-added are modest, principals do seem to know who their high flyers are, even if they do not always identify them in the ratings.

Policy Implications/Recommendations:
  • Results suggest that the method of evaluation may not only affect which specific teachers are rewarded in the short term, but shape the qualities of teacher and teaching students experience in the long term.

  • Teacher evaluation based on value-added is likely to reduce emphasis on teachers' personal traits like sociability and ability to work well with multiple school actors - traits that school principals currently value highly.

  • State and federal policymakers should recognize that the various measures differ not just in their validity, but in what they measure.

  • Results suggest that "incorrect" employment decisions from the standpoint of student achievement are more likely to emerge among less effective teachers who may have philosophical or personality-driven conflicts with principals.

Research Design:
Quantitative, qualitative

30 principals from 18 elementary schools, eight middle schools and four high schools

Year data is from:
2005, 2006


Data Collection and Analysis:
Interviews with principals were conducted over a two-year period. District personnel provided interview materials that allowed researchers to link principals' discussions of individual teachers to the district's administrative data that included test scores and teacher linkages while maintaining teacher confidentiality.


Reference in this Web site to any specific commercial products, processes or services, or the use of any trade, firm or corporation name is for the information and convenience of the public, and does not constitute endorsement or recommendation by the Education Commission of the States. Please contact Kathy Christie at 303-299-3613 or kchristie@ecs.org for further information regarding the information posting standards and user policies of the Education Commission of the States.