Minnesota Reading Corps K-3 Impact Evaluation
Problem
The limited number of previous evaluations on volunteer-only tutoring programs has shown variation in their effectiveness to improve children’s literacy proficiency.
The Minnesota Reading Corps program is a data driven statewide initiative with a mission to help every Minnesota child become a proficient reader by the end of third grade. Minnesota Reading Corps’ Kindergarten through third grade (K-3) program engages a diverse group of volunteer AmeriCorps members with no prerequisite literacy or education backgrounds to provide tutoring services to at-risk K-3 elementary school students.
Solution
NORC designed and implemented an Impact and Process Evaluation of the Minnesota Reading Corps Program.
Between 2011 and 2014 on behalf of the Corporation for National and Community Service (CNCS), NORC utilized a Randomized Controlled Trial (RCT) design, whereby students within the same grade were matched at the beginning of the school year on fall benchmark (i.e., pretest) literacy scores and randomly assigned to receive or not receive tutoring.
The evaluation consisted of three components: (1) a feasibility study (Phase I), (2) a process assessment (Phase I & II), and (3) an impact evaluation (Phase II).
(1) The Feasibility Study (Phase I): The feasibility study included the following:
- A thorough review of the literature
- A field assessment with six similar literacy enrichment or reading tutoring programs
- The development of two impact evaluation designs (for pre-K and K-3 programs), assessment tools, and random assignment procedures
- Fielding an AmeriCorps member survey.
(2) The Process Assessment (Phase I & II): The process assessment consisted of identifying, recruiting, and conducting detailed site visits with 25 Minnesota Reading Corps programs, including Head Start programs, community-based preschools, and public schools to assess program context, fidelity to the Reading Corps model, and facilitators and barriers to program implementation. In addition, a baseline and follow-up survey were completed with AmeriCorps members to determine the program's effect on their educational goals and civic engagement.
(3) The Impact Evaluation (Phase II): NORC conducted Phase II of the project to implement an experimental impact evaluation of the K-3 program and a quasi-experimental evaluation of the PreK program to measure the impact of Minnesota Reading Corps on students' literacy levels.
The results from both studies showed that younger students (i.e., PreK, Kindergarten, and first grade students) participating in the Minnesota Reading Corps program achieved significantly higher literacy gains than did similar students not participating in the program.
Result
Volunteer AmeriCorps members of the Minnesota Reading Corps can produce significantly greater increases in student literacy outcomes among Kindergarten, First and Third grade students over one semester of tutoring.
Key Study Findings include:
- Kindergarten, first, and third grade students who received Minnesota Reading Corps tutoring achieved significantly higher literacy assessment scores than students who did not.
Kindergarten students who participated in the program produced more than twice as many correct letters sounds by the end of the first semester than did students in the control condition. In addition, by the end of the first semester, students who received tutoring were able to produce approximately 11% more letter sound segments in nonsense words than students in the control group. In contrast to the findings for Kindergarten and first grade students, the effect of the program was significant, but small for third grade students and not statistically significant for second grade students.
- In Kindergarten and first grade, Minnesota Reading Corps tutoring resulted in statistically significant impacts across multiple racial groups and despite important risk factors.
A statistically significant impact of tutoring was detected among Kindergarten and first grade students despite gender, minority group status, Dual Language Learner (DLL) status, and Free or Reduced Price Lunch (FRPL) eligibility. For each of these characteristics, students who received tutoring significantly outperformed control students who did not receive tutoring on grade-specific literacy assessments.
- The Minnesota Reading Corps program is replicable in multiple school settings using AmeriCorps members with varied backgrounds.
Student assessment scores did not vary by AmeriCorps member characteristics nor by the specific school at which the tutoring occurred. These results support the conclusion that the Minnesota Reading Corps program is replicable in multiple school settings using AmeriCorps members with diverse backgrounds. Findings from an earlier process assessment concluded that the program’s approach to training, coaching and supervision, as well as their intentional recruitment of members, many of whom have no formal training or experience in education, is effective.
The overall results suggest that the Minnesota Reading Corps program produces the largest effects most quickly with the youngest students, particularly Kindergarten students. In later grades (i.e., second and third), when students begin the more complex task of reading connected text, the program did not produce changes in student oral reading fluency outcomes in the short-term. However, it is possible that the Minnesota Reading Corps program may have a larger effect on third grade students’ oral reading scores, but these improvements take more time to emerge in older students.
Related Tags
Project Leads
-
Carrie E. Markovitz
Program Area Director for Youth ResearchProject Director & Co-Principal Investigator -
Marc Hernandez
Principal Research ScientistPrincipal Investigator -
Carol Hafford
Senior FellowSenior Staff -
Heidi Whitmore
Senior FellowSenior Staff