Data Research and Development Center and IERI Research Cmmunity header

The IERI Research Community :: Projects

Working with Teachers and Leveraging Technology to Scale Opportunities to Learn More Complex and Conceptually Difficult Middle School Mathematics

Jeremy Roschelle


Background: Technologies that represent complex concepts in new, more accessible ways (e.g. through visualization, simulation, and modelling) have a potentially profound role in democratizing students' access to important advanced mathematics and science. Yet ambitious combinations of technology and curriculum can be difficult for teachers to implement. Therefore, gains from interventions may not scale. In deciding whether to invest in interventions that incorporate dynamic representations, educational leaders need to know whether such inventions work, which characteristics of teachers and their settings predict successful implementation, and what happens as the initial support for an intervention is withdrawn.

Purpose: Our project evaluates a mature technology-based innovation for 7th and 8th grade mathematics in a random assignment experiment. The innovation, called SimCalc, integrates curriculum, software, and teacher professional development in an effort to deepen students' understanding of proportionality, linear function, and rates of change. We seek to understand whether the innovation works in the classrooms of a geographically and demographically diverse sample of Texas 7th and 8th grade teachers and to model which the factors relating to teachers and their setting that are associated with the creation of enhanced opportunities to learn.

Setting: The project takes place in about a third of the geographical regions of Texas. The Texas Education Authority divides Texas into 20 regions; within each an "Educational Service Center" (ESC) provides professional development to the regions' teachers. The project began in September 2004 and will continue through August 2009. In the 2005-6 school year, we contrast 7th grade teachers implementing SimCalc with a control. In the 2006-7 school year, we contrast 7th grade teachers implementing our intervention for a second time with those who implement it for the first time, thus gaining insight into the consequences of additional professional development and experience. In the 2007-8 school year, we observe what happens as research support is scaled back. In addition, we perform an experiment with 8th grade teachers in the 2006-7 school year, this time increasing the realism by having regional leaders providing all teacher professional development.

Research Design: The project employs the following methodologies: correlational, descriptive, experimental, interview, longitudinal (by teacher, not student), observation, statistical modeling, videography, case studies.

We recruited a pool of volunteer teachers through the ESCs, asking the ESCs to recruit a wide variety of teachers in their region. For the 7th grade study, we randomly selected a sample of 140 teachers from the pool. In the first year of this study, teachers in the control condition attended an acclaimed workshop on proportionality--the same workshop was used to introduce the intervention to the experimental group and the experimental group received additional training in SimCalc materials. The control teachers taught their normal proportionality unit; the experimental teachers taught a SimCalc replacement unit. The study has a delayed treatment design--the control teachers go to a workshop and teach the SimCalc unit in the 2nd year and we compare them to the teachers who are now receiving a second workshop and teaching SimCalc a second time. For the 8th grade study, we accepted all 80 teachers whose applications qualified. In the control condition, teachers go to a high quality technology-based workshop and teacher their regular 8th grade linear functions unit.

The main outcome measure in each grade level is a test of student understanding of the relevant mathematics. We constructed these measures through an extensive assessment development process, which included specification of an assessment blueprint, development of items, expert panel review of items, protocol analysis as students solved items, collection of pilot data and application of item response theory, and assembly of a final test. We also constructed measures of teachers' "Mathematical Knowledge for Teaching" using a similar process. In addition, we collect background data on teachers and their schools, teachers' responses to an attitude survey, classroom logs, and
observations in a subsample of classrooms. The primary analysis technique will be hierarchical linear modeling, with students nested within teachers. We plan to complement statistical analysis with three sets of case studies.

Findings: First findings will be available in Fall 2006. We plan to make our quantative data sets available to others after our results are peer-reviewed and published. In addition, we have offered a workshop in our assessment development process at AERA.

For more information about this project, visit the SimCalc website at