If you’ve ever wondered if a new teaching strategy, school improvement program, or educational policy actually works from an evidence-based perspective, then consider subscribing to the What Works Clearinghouse (WWC) Study Review series. Every month (roughly) the WWC releases short (two-page) reviews of important educational studies by describing the intervention and the findings.
These high level snapshots of research are a quick read and point you to deeper information about the study’s details. In addition – and this is the big value-add! – the WWC provides a rating of each study based on the scientific quality of the research design and analysis.
Three Possible Ratings
1. Meets Evidence Standards Without Reservation
- This is the “gold star” rating for research design and implementation
2. Meets Evidence Standards With Reservations
- This middle rating indicates some questions that an observed effect (e.g., increased reading comprehension) was actually caused by the intervention (e.g., reading strategy)
3. Does Not Meet Evidence Standards
- This rating indicates a number of problems with the research design and/or implementation, calling into question any conclusions or outcomes
Three Reviews, Three Different Ratings
I’ve pulled three examples of current WWC reviews that represent the three types of ratings.
Example #1 – Meets Without Reservation
Example #2 – Meets With Reservations
WWC Review of the Report “Are Tenure Track Professors Better Teachers?”
Example #3 – Does Not Meet Evidence Standards
The first study used a randomized controlled trial design with good execution and sampling. The second study suffered from the use of a convenience sample and non-random assignment to the treatment and control groups. And in the third example, participants were not randomly assigned to either the treatment or control groups and the authors did not provide information about how the two groups compared prior to the intervention.
What’s the Take-Away?
The beauty of the WWC’s rating system is that it saves time. If you’re interested in finding evidence-based strategies and programs in education, then you can focus on the studies that meet the gold standard in research design and implementation, as found in Example #1. In the other two examples, however, the evidence is questionable, so any conclusions or findings about either study should be treated with caution.
Much More on the WWC Web Site
In addition to the single study reviews, the WWC also provides intervention reports, practice guides, and quick reviews in an easy to use searchable database.
Articles Cited
Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2013). Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management, 33(1), 94–111.
Figlio, D. N., Schapiro, M. O., & Soter, K. B. (2013). Are tenure track professors better teachers? (NBER Working Paper 19406). Cambridge, MA: National Bureau of Economic Research.
Pennebaker, J. W., Gosling, S. D., & Ferrell, J. D. (2013). Daily online testing in large classes: Boosting college performance while reducing achievement gaps. PLoS ONE, 8 (11), e79774. doi:10.1371/journal.pone.0079774