ST Math meets WWC quasi-experiment and ESSA Tier 2 requirements:
Third-Party Validation of ST Math Quasi-Experimental Study, read more below
ST Math Randomized Controlled Trial, read more on the IES What Works Clearinghouse
These join more than 100 studies on the efficacy of ST Math, including other third-party studies, MIND's own quasi-experimental studies across states and years, and MIND's subgroup studies.
WestEd recently published the largest study of its kind to evaluate an education technology math program nationally, including over 150,000 students between 2013 and 2016. This national-level study expands upon previously published WestEd independent validations of the effectiveness of ST Math at the district level (Los Angeles Unified School District 2013) and statewide (California 2014).
The study looked at grades 3, 4 and 5 in 474 schools that started using ST Math between 2013 and 2015, and included 16 states where complete state standardized test and demographic data was publicly available to the researchers.
The results were especially significant at the 239 schools that used ST Math consistently (where more than 85% of students used the program and on average completed at least 40% of their grade-level content by April 15). These high fidelity schools showed significant growth.
Schools that consistently used ST Math outgrew similar schools in statewide rank by 14 percentile points.†
When looking at the percent of students in one grade who achieved math proficiency on their state test at a given school, ST Math had an average effect size of 0.35 on statewide ranking (z-score). As a reference point, the federal What Works Clearinghouse defines 0.25 effect size and above as “substantively important.”
A first-of-its-kind independent review validating nonprofit research center WestEd’s study of ST Math® to federal evaluation research standards was published by nonprofit, research center SRI International. The review determined that the 2019 WestEd study on ST Math meets Every Student Succeeds Act (ESSA) Tier 2 and What Works Clearinghouse (WWC) standards.
Why Validate the Findings?
At MIND, we believe that ESSA Tier claims about program effectiveness should be based on rigorous and transparent evidence and on independent review. That’s why we developed a scalable, repeatable, high volume evaluation methodology; asked WestEd to perform and validate the method and report results; and asked SRI to formally review WestEd’s study to specific federal requirements.
The 2019 nationwide WestEd study was a breakthrough in effectiveness evidence for ST Math, as the size was at last sufficient to earn all the significance asterisks for our novel studies. What’s novel is we’re reporting out results on the highest stakes metric in the education market: school-wide performance improvement from year to year.
SRI’s third-party evaluation of the WestEd study to federal specifications provides an extra layer of scrutiny and accountability. There is currently no requirement for edtech ESSA evidence claims to be third-party validated, but MIND has put that requirement on ourselves because we believe full transparency to be so vital to improving the health of the education market.
What Did SRI Find?
SRI completed a technical review of the 2019 WestEd study against the WWC v4.0 Group Design Standards. SRI concluded that the WestEd ST Math study "fulfilled the design, analytic, and technical requirements" for 'Meets Evidence Standards with Reservations' according to the WWC standards.
SRI also reviewed the WestEd study design against ESSA levels of evidence provided by the U.S. Department of Education. SRI determined that the study provides moderate evidence (ESSA Tier 2) for ST Math efficacy in grades 3-5.