Data Research and Development Center and IERI Research Cmmunity header

The IERI Research Community :: Projects

Scaling Up the Implementation of a Pre-Kindergarten Mathematics Curricula: Teaching for Understanding with Trajectories and Technologies

PRINCIPAL INVESTIGATOR:
Douglas H. Clements

CO-INVESTIGATORS:
Julie Sarama, Prentice Starkey, Alice Klein, Ann Wakeley

CATEGORIES:
Math

PROJECT OVERVIEW:
Background: Although the successes of research-based, visionary educational practices have been documented, equally well known is the paucity of successful efforts to bring these practices to scale. The same research corpus provides guidelines to scale up successful interventions. There may be no more challenging issue than that of effectively scaling up an educational intervention with the diverse population who teaches Pre-K and the diversity of program structures in the early childhood system in the U.S.

Purpose: This study used a randomized field trial design to evaluate the efficacy of a research-based model for scaling up a preschool mathematics intervention, which is based on 10 research-based guidelines, including research-based mathematics curricula with an emphasis on teaching for understanding following developmental guidelines, or learning trajectories, and using technology at multiple levels. We then implemented that model and evaluated the implementation with a limited scale up study. Within an experimental design involving 25 classrooms serving children at-risk for later school failure, we examined the impact of the model, using measures of fidelity of implementation, classroom observations of mathematics environment and teaching, and child outcomes.

Intervention: The TRIAD intervention includes collaboration of key groups to establish and maintain (a) a Pre-K mathematics curriculum, with all components of the curriculum a teacher's manual, demonstration videotapes, manipulatives, software, teaching strategies, assessments, and professional development based on a common core of understanding the learning trajectories through which children develop mathematically, (b) professional development for teachers, (c) on-site support for teacher by facilitators during the school year, and (d) supportive roles and materials for parents. TRIAD's collaborative of key groups is dedicated to developing organizational structures that intensify and focus, rather than dissipate and scatter, teachers motivation to engage in and maintain this challenging practice.

Setting: Pre-K contexts included public preschool and Head Start classrooms in New York and California. These programs serve an ethnically diverse population of low-income families: African-American and Latino families comprise the largest group (60% in New York programs; 80% in California), with Asian-American, Caucasian, and Middle-Eastern families comprising the remainder. The programs had 99% (Head Start) and 74% (state-funded) children receiving reduced or free lunch (NYS).

Research Design: In 16 classrooms in NY and 10 in CA, 8 kindergarten-intending children were randomly selected; one teacher moved early in the school year, resulting in totals of 25 teachers and 209 children who participated in all aspects of data collection. Average class size was 16.8 in the TRIAD classes and 17.5 in the control classes. The control teachers continued using their school s mathematics activities, which, typical for preschools, showed a mixture of influences. The largest group used a city-wide set of activities that included number sense, operations, modeling/representing, measurement, data and reasoning, and uncertainty. Other control classrooms used the Creative Curriculum (Teaching Strategies Inc., 2001) or homegrown materials.

All instruments were created for this project. Children's mathematical knowledge was assessed with our created "Building Blocks Assessment" (BBA), which uses an individual interview format, with explicit protocol, coding, and scoring procedures and assesses children's thinking and learning along research-based developmental progressions within areas of mathematics considered significant for preschoolers, as determined by a consensus of participants in a national conference on early childhood mathematics standards, rather than mirroring the experimental curriculum's objectives or activities. Classroom teaching practices and environment were assessed by two newly-created observational instruments, "Fidelity of Implementation" and "Classroom Observation of Early Mathematics Environment and Teaching" (COEMET) created based on research on the characteristics and teaching strategies of effective teachers of early childhood mathematics. Teacher and parent questions completed the data collection.

Findings: We expected our research-based implementation to lead to substantive gains across multiple contexts, leading to our inclusion of the following contextual variables: location (NY and CA); types of Pre-K programs (e.g., Head Start and State Preschools) and child/family characteristics (e.g., child SES). Implementation variables are features that we encourage (but can not control absolutely); the main one for this study is fidelity of implementation. Analyses of children achievement data were computed on Rasch T scores using the average classroom average scores because the classroom was the unit of random assignment and thus the unit of analysis. There was a significant main effect for time (pre-post), F(1, 23) = 462.63, p < .0001, MSE=2.63 and a significant treatment by time interaction, F(1, 23) = 31.18, p < .0001, MSE=2.63, partial eta squared=.58 {the proportion of total variation attributable to the factor, partialling out other factors from the total nonerror variation, Pierce, 2004 #2633}. There were no other main effects and no significant interactions involving state or type of program. (Using the child as the unit of analysis only for exploratory child-level variables, an ANOVA similarly revealed no significant interactions of treatment by child SES which, by design, had limited variance by ethnic group, or by gender.) The TRIAD group significantly outperformed the control group, with an effect size of .46 (Cohen's d) comparing posttest scores and .71 comparing gain scores.

All but 10 of the 61 Fidelity items were generated by 4-point Likert scales from Strongly Disagree (1) to Strongly Agree (4). The mean Likert score was 2.8 (SD = .60). A factorial repeated measures analyses was computed on the total Fidelity score (this Rasch T score also included the 6 additional variables, such as number and duration of activities) to ascertain whether fidelity increased or decreased in the experimental classrooms. However, the ANOVA revealed no significant effect for time. Finally, the mean of the Fidelity scores correlated positively, but only with marginal significance, with average child gain scores (r = .48, p =.11, recall the low n). In addition, factorial repeated measures analyses were conducted on the Rasch T scores to test for differences in the quality and quantity of the mathematics environment and teaching in the treatment, compared to the control, classrooms. The repeated measures ANOVA computed on the COEMET's T scores showed a significant treatment effect, F(1, 23)= 7.14, p = .014, MSE=78.23, but no significant main effect for time and no significant treatment by time interaction (analyses were computed on only the two scores, Winter and Fall, for which both sites had data). The TRIAD group had the higher scores.

PROJECT PUBLICATIONS:
Presentations
The Annual Meeting of the American Educational Research Association, San Francisco, CA, April, 2006. Scaling Up the Implementation of a Pre-Kindergarten Mathematics Curricula: A Program Evaluation.

The 83rd Annual Meeting of the National Council of Teachers of Mathematics, Anaheim, CA, April, 2005. PreK Mathematics Across Diverse Settings--Issues of Scaling Up.

ON THE WEB:
For more information about this project, visit the Building Blocks website at http://www.gse.buffalo.edu/org/buildingblocks/, the TRIAD website at http://www.UBBuildingBlocks.org/, and Julie Sarama's website at http://www.gse.buffalo.edu/faculty/viewfaculty.asp?id=50.