Data Research and Development Center and IERI Research Cmmunity header

The IERI Research Community :: Projects

Studying Robust-Design Strategies for Developing Innovations Effective and Scalable in Challenging Classroom Settings

Christopher Dede


Background: Studies of robust-design and extreme scalability via multi-user virtual environments (MUVEs) address objectives such as research on the strengths and limits of emerging technologies, the development of new methodologies for studying learning and teaching, the enhancement of students' engagement and educational outcomes in STEM, and more effective teacher professional development. MUVEs may be a promising complement to more conventional kinds of instruction, particularly for low-performing students unmotivated by conventional pedagogy and skeptical about their ability to learn science, thereby substantially increasing the pool of scientific expertise available to the nation and the sophistication of its citizens in making science-related decisions.

Purpose This project is studying the extent to which a technology-based curricular innovation developed through "robust-design" strategies is effective in increasing students' educational outcomes across a range of challenging classroom settings. Evolving an intervention for extreme scalability even into contexts in which some of its conditions for success are attenuated or lacking requires "ruggedizing" its efficacy when parts of its intended enactment are absent. This project is assessing such a strategy for extreme scalability through design-based research on large-scale implementations of a multi-user virtual environment (MUVE) curriculum across a spectrum of contexts.

Intervention: The River City virtual "world" consists of a city with a river running through it; different forms of terrain that influence water runoff; and various neighborhoods and institutions, such as a hospital and a university. Students work in teams to develop hypotheses regarding one of three strands of illness in River City (water-borne, air-borne, and insect-borne). In year one, in order to ruggedize our design for motivation, we have added the ability for a student's avatar to gain "levels" that reward achievement of various curricular objectives with enhanced "powers" in the MUVE, each linked to academic content.

Setting: In the first year of our work, we are implementing intensively in schools in southeastern Wisconsin and in Miami, Florida, with scattered implementations in districts across several other states.

Research Design: Across hundreds of classroom settings in the first of our project's three years, we are implementing an extended version of our MUVE, approximately four weeks in length, which should result in substantial effect-size gains in student outcomes beyond what we have achieved in two-week interventions. Each year, our robust-design methods will draw on the prior implementation s results to develop and study a further ruggedized version with increased effectiveness in extreme conditions. Through this research, we will develop and validate sophisticated methodologies for determining the sensitivity of the MUVE's impact to four selected, varying contextual conditions (extent of teacher professional development involvement, class size, student previous achievement, and student engagement) and for assessing the MUVE's overall effectiveness.

In total, across three years, our sample will contain approximately 300 teachers and 30,000 students in 7-9th grade classrooms with high proportions of ESL and free-and-reduced lunch students. This is a convenience sample of teachers and students selected to be representative of a diverse national population with various socioeconomic, linguistic, ethnic, and cultural characteristics.

Our affective instrument is an adaptation of three independently validated surveys, Self-Efficacy in Technology and Science, Patterns for Adaptive Learning Survey, and the Test of Science Related Attitudes. Our Disease Content and Problem Solving Skills Instrument builds on previously published and validated instruments for assessing science process skills. We also have a variety of instruments for measuring student and teacher engagement. We will use multilevel modeling (Singer & Willett, 2003) to investigate all research questions, while allowing for the clustering of children with class, where appropriate.

Findings: Findings from year one are not yet available.

Extensive reports on our work, as well as research videos and images, are available on our project website:

You can learn more by visiting the project website at