By: Rachel Brown, Ph.D., NCSP
The term “evidence-based” has become very popular in schools and other professional service settings. For example, physicians and dentists now often refer to the importance of evidence-based practices. Since the passage of the 2001 revision of the U.S. Elementary and Secondary Education Act (ESEA), teachers have been expected to use only evidence-based teaching practices, or something very similar such as empirically-based or scientifically-based instruction. Although this requirement has become almost universal, the exact meaning of evidence-based instruction is not always clear. The Every Student Succeeds Act (ESSA), a major rewrite of the much-maligned No Child Left Behind Act (NCLB), is the first federal education law to define the term “evidence-based” and to distinguish between activities with “strong,” “moderate,” and “promising” support based on the strength of existing research. So, what, exactly, is evidence-based instruction?
Origins
Researchers have used terms such as evidence-based, empirical, and scientifically-valid for many years. Nonetheless, the application of such terms in public education was limited until 2001 when the ESEA was reauthorized in a version known as the NCLB. The ESEA is a law that makes federal funds for public education available to states and school districts if they comply with the provisions of the law. The NCLB version of the law expanded the scope of federal influence in public education by requiring states to collect more student data and show that students were meeting specific learning standards in order to have access to specific funding sources (NCLB, 2002). It is worth noting that the NCLB did not use the term evidence-based, but instead used the term scientifically-based. Here is the definition of scientifically-based instruction from the NCLB Act (2002):
SCIENTIFICALLY BASED RESEARCH.— The term ‘scientifically based research’—
- means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs; and
- includes research that—
(i) employs systematic, empirical methods that draw on observation or experiment;
(ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn;
(iii) relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations, and across studies by the same or different investigators;
(iv) is evaluated using experimental or quasi-experimental designs in which individuals, entities, programs, or activities are assigned to different conditions and with appropriate controls to evaluate the effects of the condition of interest, with a preference for random-assignment experiments, or other designs to the extent that those designs contain within-condition or across-condition controls;
(v) ensures that experimental studies are presented in sufficient detail and clarity to allow for replication or, at a minimum, offer the opportunity to build systematically on their findings; and
(vi) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.
The above definition is quite detailed and perhaps overwhelming. This is part of the reason why researchers and teachers have developed their own operational definitions. To be helpful, classroom teachers and school leaders need to understand and use evidence-based, scientifically-based, or whatever term will be used, with integrity and consistency.
Variations
Among the variations on evidence-based instruction are the terms scientifically-based, research-based, and empirically-based. There are others as well, but for this blog, I offer the two following condensed definitions (see also Brown-Chidsey & Bickford, 2016):
- Scientifically-based instruction includes materials and methods that have been tested and found to be effective in relation to the specific research questions addressed by an individual study that uses experimental methods.
- Evidence-based instruction includes materials and methods that have been tested and found to be effective for large groups of diverse students and across two or more experimental research studies.
For the purpose of this blog, the terms empirically-based and research-based are understood to mean the same thing as scientifically-based. The difference between scientifically-based and evidence-based instruction is that the former has supporting research from one study with a specific sample, and the latter has support research from two or more studies with diverse samples. This distinction is important because how well an instructional program will work for students relates to the extent it has been tested and verified as effective with multiple large groups of students. For this reason, the process that a school uses to review and adopt instructional materials is very important.
Adoption
The selection of instructional materials and methods for use with large and small groups of students is likely to have an effect on those students’ learning outcomes. Although different terms have been used to describe practices with some form of research base, these terms might not all mean the same thing. When schools consider what instructional practices to use, the amount of prior data, as well as the specific findings, can be important. Curriculum review teams are encouraged to carefully consider both the scientific findings and number of studies available when making decisions about instructional practices. In many U.S. schools, the materials and methods used for general classroom instruction (i.e., Tier 1 core instruction) must be approved by a district-wide committee and formally adopted by the district school committee or board. The rationale for such approval is that core instruction is used with all students, thus it has large effects. For this reason, very careful scrutiny of the practices adopted for core instruction is essential. As defined here, evidence-based instruction is likely to have a larger and stronger research base than scientifically-based instruction. The same principles apply to instruction for small groups and individual students, but the effects are much more limited for small groups and individuals. For this reason, when evidence-based instruction is not available for additional intensive instruction (i.e., intervention), short-term use and progress monitoring with a scientifically-based program might be justified. In such cases, it is important that regular data about the students’ progress are collected and reviewed so that the instruction can be changed if the students do not make effective progress.
Summary
Although the terms evidence-based, research-based, and scientifically-based instruction might seem synonymous, the educational research community has recently provided more detailed definitions. As explained here, the terms empirically-based, research-based, and scientifically-based are all usually interpreted to mean that the practice has been shown to be effective with one group of specific students in one setting. In contrast, the term evidence-based is understood to mean that a particular practice has been shown to be effective in two or more studies with different groups and settings of students. For this reason, evidence-based instruction is more likely to work with students across more varied schools and settings. School leaders, including teachers, principals, superintendents, and board members are encouraged to think carefully about the programs and practices they endorse for use with large and small groups of students. Their decisions are likely to have lasting effects on student learning outcomes.
References
Brown-Chidsey, R. & Bickford, R. (2016). Practical handbook of multi-tiered systems of support: Building academic and behavioral success in schools. New York: Guilford Press.
No Child Left Behind Act of 2001, P.L. 107-110, 20 U.S.C. § 6319 (2002).