Guiding Questions
References and Resources
The NRC Principles
Is education research scientific? Can education research be scientific? According to the 2002 National Research Council (NRC) report Scientific Research in Education, science is the same in all fields of study, whether it is chemistry, economics or education. What determines the scientific quality of a research study is the degree to which the study follows the principles that underlie science. The NRC identified six guiding principles for scientific research. The actual principles are quoted here verbatim from the NRC; the further elaboration of each principle paraphrases NRC’s discussion of the principles and includes explanatory text that is original to this Primer.
SCIENTIFIC PRINCIPLE 1: POSE SIGNIFICANT QUESTIONS THAT CAN BE INVESTIGATED EMPIRICALLY.
Empirical research involves investigation that uses observations to guide conclusions. Research questions that are significant do one or more of the following:
- Fill in the gaps in what we know about a topic.
- Seek to identify why something occurs.
- Solve a practical problem.
- Test a new idea or hypothesis.
- Expand on scientific knowledge from prior theories and research.
SCIENTIFIC PRINCIPLE 2: LINK RESEARCH TO RELEVANT THEORY.
Theories vary in scope; the more well-known scientific theories tend to be broad such as Einstein’s theory of relativity. Theories that are smaller in scope, sometimes referred to as conceptual frameworks, guide most research studies, particularly in the social sciences and education. Nonetheless, such theories provide the reason for the research design and interpretation of the findings. For example, the theory behind teacher professional development is that teacher learning influences instruction, which in turn influence student achievement. This theory is relatively small in scope because it applies only to teacher learning, in contrast to a theory such as Piaget’s, which applies to child and adolescent development. Theories that are small in scope however, can provide the rationale for scientific research.
SCIENTIFIC PRINCIPLE 3: USE METHODS THAT PERMIT DIRECT INVESTIGATION OF THE QUESTION.
This principle means that the research method should be appropriate to the research question. The appropriateness of one method over another is the subject of debate. This is particularly true in the social sciences where research studies usually involve human subjects. Principle 3 however, does not focus on a particular research method. Rather, it emphasizes that a report on a research study should indicate the following:
- The link between the research question and the method used and why the method is the most appropriate.
- A detailed description of the method and procedure so that other researchers can repeat the study.
- Possible problems or limitations with the research method.
As Principle 1 indicates, science involves the measurement of observations. In social science research, this means that human behavior will be observed, measured and recorded. The method used to measure observations is critical because errors in measurement can influence the results.
For this reason, research studies should report on the validity and reliability of the measuring instruments that are used.
SCIENTIFIC PRINCIPLE 4: PROVIDE A COHERENT AND EXPLICIT CHAIN OF REASONING.
Conclusions about the results of research are based on inferential reasoning. This means that researchers make logical judgments based on the results of their research and on conclusions from prior research. The logic of their judgments depends on their research questions and the methods they used. An important part of this logical reasoning is to rule out alternate or rival explanations, also referred to as threats to validity. To counter such threats, researchers need to indicate in their studies how they avoided or controlled for such errors.
SCIENTIFIC PRINCIPLE 5: REPLICATE AND GENERALIZE ACROSS STUDIES.
Replication means that a researcher who uses the same study method in the same situations or contexts as another researcher can make the same observations and obtain the same results. (Alternatively, the same researcher can obtain the same results on two different occasions.) Generalization refers to how much the results can be replicated in different contexts and with different populations. When the results of a study can be replicated and generalized, the results can be trusted more than results from studies without these characteristics. Usually, many research studies are needed to produce a body of knowledge that provides this information.
SCIENTIFIC PRINCIPLE 6: DISCLOSE RESEARCH TO ENCOURAGE PROFESSIONAL SCRUTINY AND CRITIQUE.
Through this principle, the National Research Council emphasizes that the accumulation of scientific knowledge depends on its dissemination to members of the scientific community for professional critique. Researchers should submit their reports to journals and publications that require peer review. Presentations on research at professional conferences also provide the opportunity for critique. To facilitate scrutiny, researchers should keep accurate and accessible records of their investigations so they can provide information for review purposes. For education research to advance, the community of education researchers must enforce the norms of scientific research when judging education research studies.
To determine whether an education research study is following scientific principles, ask the following questions about the study:
|
Scientific Principle 1. Pose significant questions that can be investigated empirically. Guiding Questions What is the research question? Will answering the research question provide new knowledge or solve a problem? Is it possible to answer the research question through observations of some type? Scientific Principle 2. Link research to relevant theory. Guiding Questions What theory or framework is being used to answer the research question? What is the relationship between the theory or framework and the way that the study is being conducted? Scientific Principle 3. Use methods that permit direct investigation of the question. Guiding Questions What methods were used to conduct the study? Does the study indicate how the method is appropriate for the research question? Is there detailed information on how the method was carried out so other researchers can repeat the study? Does the study report on the validity and reliability of the measuring instruments? Does the study describe potential problems with the method used? Scientific Principle 4. Provide a coherent and explicit chain of reasoning. Guiding Questions Does the study rule out explanations for the results other than the explanation given by the researcher? Does the study demonstrate how errors or threats to the validity of the results were avoided? Scientific Principle 5. Replicate and generalize across studies. Guiding Questions Is there sufficient information to repeat the study? Are there other studies that have found similar results but in different settings or with different participants? What additional research is needed to extend and generalize the results of the study? Scientific Principle 6. Disclose research to encourage professional scrutiny and critique. Guiding Questions Where has the study been published? Has the study been reviewed by other education researchers? |
National Research Council. (2002). Scientific research in education. Committee on Scientific Principles for Education Research. Shavelson, R. J., and Towne, L., Editors. Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
