A researcher wants to conduct a study to determine whether a teacher professional development program increases student achievement. Which of the following studies is most likely to result in valid conclusions?

Study 1: The 4th-grade teachers in a school district receive professional development in reading during the school year. At the end of the school year, student achievement scores are examined.

Study 2: The 4th-grade teachers in a school district receive professional development in reading during the school year. The students of the teachers take an achievement test at the beginning and at the end of the school year.

Study 3: Prior to the beginning of the school year, half of the 4th-grade teachers in a school district are randomly assigned to receive professional development in reading during the year, and the other half are assigned to receive no professional development in reading. At the end of the school year, student achievement scores for the two groups are examined.

Study 4: Prior to the beginning of the school year, half of the 4th-grade teachers in a school district are randomly assigned to receive professional development in reading during the year, and the other half are assigned to receive no professional development in reading. The students of the teachers take an achievement test at the beginning and at the end of the school year.

Answer:

Study 1: Without a pretest, it is impossible to know whether the students made gains. Without a control group of teachers who were randomly assigned to receive no professional development, it is impossible to know whether the students’ scores were influenced by the professional development that their teachers experienced, so many rival explanations are possible here.

Study 2: The pretest makes it possible to measure student gains in reading. There could be many reasons, however, for gains other than the teacher professional development program. Without a control group of teachers who were randomly assigned to receive no professional development, it is impossible to know whether the students’ scores were influenced by the professional development that their teachers experienced or as a result of the normal instruction that students received in the school.

Study 3: This study compares a treatment group (teachers who receive professional development) with a control group (teachers who do not receive professional development). The teachers have been randomly assigned to the two groups. Without a pretest, however, it is impossible to know whether the students in the treatment made gains compared to the control group. Perhaps the treatment students were higher in achievement than the control students before the study began.

Study 4: This study uses a pretest-posttest data-collection strategy. It compares a treatment group with a control group, and the teachers have been randomly assigned to the two groups. The pretest makes it possible to measure student gains in reading. Study 4 is most likely to result in valid conclusions.