In the current edtech landscape, we rarely look beyond the one “gold standard” study for proof that a program is effective -- even if it’s many years old and only covers a single assessment. Here at MIND, we encourage everyone to raise their sights and look for a large number of studies, using recent program versions, garnering repeatable results on multiple state assessments, and over many varied districts. We believe a high volume of effectiveness studies is the future of a healthy market for product information in education.
Compared to lengthy, expensive RCTs (randomized controlled trials), organic quasi-experiments can study the adoption of a program as is, without the length and complexity of experiment planning. Methods of matching and comparing similar schools with and without the program can be made statistically rigorous and powerful. And if we study at the grade-level, we have the average state test performance data universally available. This is how we perform multiple annual, transparent evaluations of results of all ST Math school cohorts.
since 2011
120 studies across 27 states since 2011.
*Some states had multiple studies within a calendar year
In cases where gradewide performances are reported out by student subgroup, our quasi-experimental approach allows us to compare specific student subgroups with and without ST Math. In the data below, you can see similar effects of ST Math for the different subgroups in Texas.
Metric | All Students | ELL | EconDis | SPED | Gifted |
---|---|---|---|---|---|
TRT Baseline %Satisfactory+ | 34.56 | 34.16 | 29.52 | 9.92 | 91.08 |
TRT Current %Satisfactory+ | 52.54 | 51.72 | 46.42 | 23.04 | 94.75 |
CTL Baseline %Satisfactory+ | 33.71 | 33.52 | 29.83 | 10.52 | 90.67 |
CTL Current %Satisfactory+ | 44.94 | 43.28 | 37.96 | 15.84 | 92.08 |
TRT Baseline Mean Scale Score | 1534 | 1526.6 | 1515 | 1442.2 | 1721.4 |
TRT Current Mean Scale Score | 1574 | 1554.4 | 1553 | 1462.5 | 1777.4 |
CTL Baseline Mean Scale Score | 1531 | 1520.6 | 1515.9 | 1444 | 1720.8 |
CTL Current Mean Scale Score | 1551.9 | 1526.8 | 1527 | 1438.2 | 1743.1 |
TRT Delta Percentile | 10.52 | 5.88 | 12.65 | 19.04 | 9.83 |
CTL Delta Percentile | 1.38 | -3.28 | -2.65 | 4.04 | -1 |
Treatment n | 48 | 25 | 48 | 25 | 12 |
Number of TRT Years | 3 | 3 | 3 | 3 | 3 |
Metric | All Students | ELL | EconDis | SPED | Gifted |
---|---|---|---|---|---|
TRT Baseline %Satisfactory+ | 34.56 | 34.16 | 29.52 | 9.92 | 91.08 |
TRT Current %Satisfactory+ | 52.54 | 51.72 | 46.42 | 23.04 | 94.75 |
CTL Baseline %Satisfactory+ | 33.71 | 33.52 | 29.83 | 10.52 | 90.67 |
CTL Current %Satisfactory+ | 44.94 | 43.28 | 37.96 | 15.84 | 92.08 |
TRT Baseline Mean Scale Score | 1534 | 1526.6 | 1515 | 1442.2 | 1721.4 |
TRT Current Mean Scale Score | 1574 | 1554.4 | 1553 | 1462.5 | 1777.4 |
CTL Baseline Mean Scale Score | 1531 | 1520.6 | 1515.9 | 1444 | 1720.8 |
CTL Current Mean Scale Score | 1551.9 | 1526.8 | 1527 | 1438.2 | 1743.1 |
TRT Delta Percentile | 10.52 | 5.88 | 12.65 | 19.04 | 9.83 |
CTL Delta Percentile | 1.38 | -3.28 | -2.65 | 4.04 | -1 |
Treatment n | 48 | 25 | 48 | 25 | 12 |
Number of TRT Years | 3 | 3 | 3 | 3 | 3 |
Percent of students satisfactory or advanced in grades 3-5, 2014-2017. See full reports below:
All Students | Gifted | ELL | EconDis | SPED
Independent education research firm WestEd recently conducted a study of 3rd to 5th grades in more than 450 schools and 16 states. In the 14 states with schools that consistently implemented ST Math, the average effect size for math proficiency was .36, well above what is considered “substantively important" by the federal What Works Clearinghouse.