Schools that consistently used ST Math outgrew similar schools in statewide rank by 14 percentile points.†
When looking at the percent of students in one grade who achieved math proficiency on their state test at a given school, ST Math had an average effect size of 0.36 on statewide ranking (z-score). As a reference point, the federal What Works Clearinghouse defines 0.25 effect size and above as “substantively important.”
A first-of-its-kind independent review validating nonprofit research center WestEd’s study of ST Math® to federal evaluation research standards was published by nonprofit, research center SRI International. The review determined that the 2018 WestEd study on ST Math meets Every Student Succeeds Act (ESSA) Tier 2 and What Works Clearinghouse (WWC) standards.
Why Validate the Findings?
At MIND, we believe that ESSA Tier claims about program effectiveness should be based on rigorous and transparent evidence and on independent review. That’s why we developed a scalable, repeatable, high volume evaluation methodology; asked WestEd to perform and validate the method and report results; and asked SRI to formally review WestEd’s study to specific requirements.
The 2018 nationwide WestEd study was a breakthrough in effectiveness evidence for ST Math, as the size was at last sufficient to earn all the significance asterisks for our novel studies. What’s novel is we’re reporting out results on the highest stakes metric in the education market: school-wide performance improvement from year to year.
SRI’s third-party evaluation of the WestEd study to federal specifications provides an extra layer of scrutiny and accountability. There is currently no requirement for edtech ESSA evidence claims to be third-party validated, but MIND has put that requirement on ourselves because we believe full transparency to be so vital to improving the health of the education market.
What Did SRI Find?
SRI completed a technical review of the 2018 WestEd study against the WWC v4.0 Group Design Standards & Procedures. SRI concluded that the WestEd ST Math study design “meets the design, analytic, and technical requirements” for the ‘Meets Evidence Standards with Reservations’ level of the WWC standards.
SRI also reviewed the WestEd study design against ESSA levels of evidence provided by the U.S. Department of Education. SRI concluded the study design meets ESSA Tier 2, as it “fulfills the requirements of moderate evidence of significant favorable effects of ST Math on student mathematics achievement.”
