Scientifically-based Research
Sources of Education Research
Reading Education Research
Finding Education Research
References and Resources
What is Research?
The word “research” is used in many different ways. For example, people talk about “doing research” on which car to buy. They go to the library to “research” a particular topic such as a law or historical event.
In education, when people refer to research they may mean either empirical or non-empirical studies. Examples of non-empirical studies are studies that research the history of a practice, institution or individual, explore what a thinker or a number of thinkers have said about a specific topic, or use other written sources to compare educational practices in one country with those in another. Empirical research seeks information about something that can be observed in the real world or in the laboratory – what effect a certain kind of professional development has on a teacher’s ability to teach, what impact socioeconomic factors have on student performance, whether a particular curriculum improves students’ performance in mathematics, etc.
This Primer is concerned primarily with empirical research, which involves systematically gathering empirical information on questions related to education.
Education research differs along several dimensions. In general, there are two main types, descriptive and experimental. Descriptive research answers questions about what, how, or why something is happening. Experimental research answers questions about whether something causes an effect. Research data are quantitative, qualitative or a combination of the two. Depending upon the kinds of questions a research study seeks to answer and the kinds of data it intends to collect, it employs a particular plan for gathering data, called the research design.
For more information on dimensions of education research, see Appendix A, A RESEARCH TYPOLOGY.
The No Child Left Behind Act of 2001 (NCLB) makes more than 100 references to scientifically-based research in education.
For example:
- Districts with low-performing Title I schools should develop improvement plans that build on scientifically-based research.
- States seeking funds from the Reading First Initiative must contract with an entity that conducts scientifically-based reading research.
The U.S. Department of Education’s Institute of Education Sciences has released a publication that elaborates the concept of scientifically-based research. It is entitled Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide and can be viewed on the Web at http://www.ed.gov/rschstat/research/pubs/rigorousevid/index.html or downloaded at http://www.ed.gov/rschstat/research/pubs/rigorousevid/rigorousevid.pdf. For more information about the Institute of Education Sciences, go to their Web site at www.ed.gov/about/offices/list/ies/index.html?src=oc.
While NCLB’s definition emphasizes the importance of research method, the National Research Council (NRC) (www.nationalacademies.org/nrc/) has explained the importance of other aspects of scientific research. According to NRC’s 2002 publication, Scientific Research in Education, the scientific quality of a research study is determined by the degree to which the study follows the principles that underlie science. NRC identified six guiding principles for scientific research:
- Pose significant questions that can be investigated empirically
- Link research to relevant theory
- Use methods that permit direct investigation of the question
- Provide a coherent and explicit chain of reasoning
- Replicate and generalize across studies
- Disclose research to encourage professional scrutiny and critique.
For more details and a list of related guiding questions, see Appendix B, NRC’S PRINCIPLES OF SCIENTIFIC RESEARCH IN EDUCATION.
A primary source is a report of an original research study. A primary source usually provides enough details to replicate the research study. Primary sources are written by the researcher(s) or evaluator(s) who conducted the study. The main formats of primary sources are journal articles, technical reports from research institutions or education organizations, and reports on presentations at conferences.
A secondary source is a description and summary of one or more prior research studies. Secondary sources usually do not include enough details to replicate the original studies being described. Examples of secondary sources are literature reviews and books. Although newspaper articles also can be secondary sources, they often do not have enough information to help readers form a solid judgment about the research. Essays by education experts can be secondary sources of education research, but essays can be overly biased toward the views of the writer.
Caution:
Secondary sources have the potential to distort original research findings and can lead to conclusions that are based more on interpretation and opinion than on fact. Many debates about education topics arise because secondary sources draw conclusions that the original research does not warrant. When in doubt, always consult the original research study.
Use primary sources when it is important to know the details of a study and its results. Use secondary sources to obtain an overview of the research on a particular topic and reference information for original research studies (see also McMillan, 2000).
To research the topic of professional development schools for teacher preparation, start with a secondary source such as the Handbook of Research on Teacher Education (Sikula, Buttery and Guyton, 1996). This book has chapters written by education researchers on various topics related to teacher education. Then consult the primary sources cited in the chapter on professional development schools.
Reports on education research tend to follow similar formats. There are some noteworthy differences, however, depending on whether the report concerns a research study, an evaluation study or a literature review.
A research study, as the term is used in this Primer, systematically gathers empirical information to answer one or more questions related to education.
A researcher wants to know whether math teachers with master’s degrees in their field are more effective than math teachers with only an undergraduate mathematics major. The researcher observes the teaching of a number of math teachers, some of whom have master’s degrees and some of whom only have undergraduate majors. The researcher also examines students’ mathematics test scores of students to determine if the scores of those whose teachers have a master’s degree are higher than those whose teachers have only an undergraduate major.
For more information, see the guide to READING REPORTS ON RESEARCH STUDIES.
An evaluation study is designed to judge the effectiveness of an education program. Evaluation studies use some of the same research designs that research studies employ.
A school district hires an evaluator to conduct a study on the effectiveness of an after-school tutoring program. The evaluator collects data about the student participants, their achievement before and after tutoring, the type and amount of tutoring that occurred, and the characteristics of the tutors. The evaluator also collects achievement data from a comparison group of students who applied too late to receive tutoring. The evaluation results include data about changes in student achievement, as well as data about whether the program was implemented as planned.
For more information, see the guide to READING EVALUATION STUDIES.
A literature review is a comprehensive and systematic summary of past empirical research and/or evaluation studies on a specific topic. Another term commonly used for a literature review is research synthesis.
For more information, see the guide to READING LITERATURE REVIEWS.
THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC)
ERIC is a federally funded national system that provides access to education-related literature. Though currently in the process of significant revision, ERIC continues to provide a wealth of information for researchers, practitioners and policymakers. To appreciate fully what ERIC has to offer, spend some time exploring the ERIC Web site at http://www.eric.ed.gov.
For information about searching the ERIC database to find articles and other literature, see the brief SEARCHING ERIC TUTORIAL in this Primer.
OTHER ONLINE DATABASES
Although ERIC is probably the largest online database of education research, there are other online databases that are resources for finding education-related research. Libraries of institutions of higher education usually subscribe to these databases, and members of the institution have access to them. Often members of the general public with proper identification can use the libraries of their state-supported institutions of higher education.
Other databases that have citations for education research include the following:
Dissertation Abstracts – Abstracts of dissertations completed in the United States and in some foreign countries
Education Index – Citations of education-related articles from over 600 sources, with access to full-text articles at some libraries.
Many articles on education research exist as online documents on the World Wide Web. Success in searching for such documents depends on Internet searching skills.
Example:
- To conduct a search for articles on teacher preparation research, go to the Yahoo search engine at http://search.yahoo.com/search/options?p=
- Enter “teacher preparation research” into the “exact phrase” window
- Click on SEARCH
- The result will be a very large list of Web sites with information related to teacher preparation research.
Help with Internet searching techniques is available at the following Web sites:
http://library.albany.edu/internet/ http://www.sc.edu/beaufort/library/bones.html.
Caution: The requirements for posting articles on Web sites of organizations vary greatly. Some articles undergo a peer review that is similar to the review required for articles submitted to journals for publication. Other articles, however, are posted because the research supports the organization’s views. Always evaluate the quality of the research that is reported in online articles.
ELECTRONIC JOURNALS
Some education research journals exist online, such as Education Policy Analysis Archives, available at http://epaa.asu.edu/epaa/. With an electronic journal, it is possible to download and print full-text articles on education research.
U.S. DEPARTMENT OF EDUCATION WEB SITE
An important online source for education research is the Web site of the U.S. Department of Education (ED). Search for education research at http://www.ed.gov/index.jsp, which provides access to more than 200 ED-sponsored Web sites and more than 150 other federal agencies. An ED search can result in thousands of citations; for help with searching techniques, go to http://www.ed.gov/search/searchhelp.jsp.
Hint:
For access to a wide range of education statistics, see the Web site of the National Center for Education Statistics (NCES) at http://nces.ed.gov/index.html. NCES produces hundreds of reports based on its many data-collection efforts, including reports on the National Assessment of Educational Progress (NAEP) and on the Schools and Staffing Survey (SASS).
WHAT WORKS CLEARINGHOUSE
The U.S. Department of Education’s Institute of Education Sciences established the What Works Clearinghouse (WWC) in 2002 to provide an independent source of evidence on what works in education. The WWC intends to provide policymakers and educators the information needed to make decisions about education programs and interventions based on high-quality scientific research. Consult the WWC Web site for more details: http://www.w-w-c.org/about.html.
MANUAL SEARCHES
It is possible to conduct manual searches for education research by using the print versions of indexes for journals and abstracts. These indexes are available at most higher education libraries. Two examples of relevant indexes are the Current Index to Journals in Education, published by ERIC, and Psychological Abstracts, published by the American Psychological Association.
Some of the principles used for computer searches apply to manual searches. For example, it is important to determine the terms or descriptors used to identify articles related to a particular topic. Often a thesaurus that helps identify keywords to use in searches on different topics accompanies the index. With manual searches, it is generally a good idea to start with the most recent index because recent studies provide citations on prior research, which shortens the search process (see also McMillan, 2000).
References and Resources
Cooper, H. (1998). Synthesizing research: A guide for literature reviews (3rd ed.). Thousand Oaks, CA: Sage Publications.
Cooper, H., Charlton, K., Valentine, J.C., and Muhlenbruck, L. (2000). "Making the most of summer school: A meta-analytic and narrative review." Monographs of the Society for Research in Child Development, Serial No. 260, 65(1).
Institute of Education Sciences. Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. Washington, DC: U.S. Department of Education.
McMillan, J.H. (2000). Educational research: Fundamentals for the consumer (3rd ed.). New York: Addison Wesley Longman.
National Research Council (2002). Scientific research in education. Committee on Scientific Principles for Education Research. Shavelson, R.J., and Towne, L. (Eds.). Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
Sikula, J., Buttery, T.J. and Guyton, E. (Eds.) (1996). Handbook of research on teacher education. New York: Simon & Schuster Macmillan.
Weiss, C.H. (1998). Evaluation: Methods for studying programs and policies (2nd ed.). Upper SaddleRiver, NJ: Prentice Hall.
