Internal program evaluation indicates positive effects on students’ long-term achievement

June 4, 2020

Heart Math Tutoring’s mission is to ensure all elementary students develop the strong foundation in math and enthusiasm for academics needed for long-term success, by helping schools use volunteers as tutors. During the spring, Heart Tutoring conducted an internal evaluation specifically determining the program’s effect on students’ long-term success.

Thanks to a data-sharing agreement with Charlotte-Mecklenburg Schools (CMS), Heart Tutoring was able to conduct internal analysis of 30,965 observations of deidentified students using fixed effects regression models by school, year, and grade, or by school-grade-year combinations.

As determined by the Fall NWEA MAP scores of 1^st graders, on average, the achievement scores of students who participate in Heart’s program are below those of their peers who do not participate in Heart. This trend is also observed in the EOG scores of 3^rd – 5^th graders in the year prior to being served by Heart, with fifth graders starting an average of 2 years of schooling behind the non-Heart students at their schools, on average.

One measure of success for the program’s efficacy would be if Heart Tutoring helps close the gap between achievement of Heart and non-Heart students.

The internal analysis found encouraging effects:

• Students who participated in Heart during 1^st and 2^nd grade achieved 3^rd grade EOG scores reflecting no significant difference from peers, one year after participating in Heart, despite starting significantly behind. In essence, the initial gap between Heart students and their non-Heart peers was eliminated by third grade.
• Students who mastered 60% or more of the skills associated with their grade level targets on the Heart Tutoring post-assessment had no significant difference in EOG scores compared to non-Heart peers a year after program participation, closing what was initially a significant difference.
• Growth on Heart Tutoring pre/post assessments is correlated with growth on NC End of Grade tests.

Please explore the linked slides for more details on the internal analysis.