Data Research and Development Center and IERI Research Cmmunity header

The IERI Research Community :: Projects

Scaling Up Standards-Based Accountability

PRINCIPAL INVESTIGATOR:
Brian M. Stecher

CO-INVESTIGATOR:
Laura Hamilton

CATEGORIES:
Math, Science

PROJECT OVERVIEW:
Background: Each of the 50 states must implement a program of standards, assessments, and consequences to comply with the No Child Left Behind Act (NCLB) of 2001 (20 U.S.C. 6311 et seq.). The Implementing Standards-Based Accountability (ISBA) project is designed to identify factors that enhance the implementation of these standards-based accountability (SBA) systems, foster changes in school and classroom practice, and promote improved student achievement.

Purpose: Our research addresses four critical questions:

  • What strategies are used in implementing SBA at the state, district, and school levels?
  • Which state, district, and school implementation strategies are associated with changes in classroom practice?
  • Which features of SBA implementation are associated with student attainment of academic standards?
  • How valid are the standards-based test scores as indicators of changes in student achievement?

    Intervention: The project is studying the implementation of standards-based accountability (SBA) systems in the wake of NCLB. We consider these to be three interventions, the California, Georgia and Pennsylvania version of SBA. They share the NCLB framework of standards, assessments, support and consequences, but each has its own unique features.

    Setting: Schools, districts, and state agencies in California, Georgia and Pennsylvania comprised the setting for the study. Respondents were surveyed and/or interviewed in 2004, 2005, and 2006.

    Research Design: Three states were selected (California, Georgia, and Pennsylvania) to represent a range of approaches to implementing SBA, and to provide both geographic and demographic diversity. Within each state districts were sampled stratified by size. In 2003-04 we recruited 68 districts, and in 2004-05 additional districts were added to both increase the overall number of districts and the number of districts with high percentages of schools not meeting NCLB achievement goals. The total sample consisted of 92 districts in 2004-05 the latest year for which the data has been compiled. We randomly sampled elementary schools and middle schools from the sampled districts, selecting between one and five schools of each type from each district according to a prearranged pattern based on district size. Overall, we recruited 301 schools to participate in the study in 2004-05. We surveyed superintendents, principals, and science and mathematics teachers with respondents in 2004-05 numbering 67 superintendents, 260 principals, and 3668 teachers. In addition, each year we conducted case study visits to three schools in each of two districts in each state. At each school we interviewed the principal, science and mathematics teachers, parents, and other staff relevant to the implementation of SBA. In 2003 we interviewed state education administrators and stake holders, and in 2004 we conducted telephone interviews with 45 superintendents in participating districts.

    Study outcomes consist of teacher and school implementation practices and student achievement in 2003-04, 2004-05, and 2005-06. Each of the three sample states is providing student achievement results for its state test for the years 2002-03 through 2005-06. Implementation data has been obtained through superintendent, principal, and teacher surveys in 2004, 2005, and 2006. Survey data is being analyzed using a variety of statistical techniques including cross-tabulations, factor analysis, and hierarchical modeling. Further depth was added to our analysis of survey data through qualitative analysis of the case study interviews, and interviews with state education administrators and stake holders. Results were synthesized into state and district level summaries. School district superintendent interviews conducted in 2003-04 were analyzed using the QRS N6 qualitative analysis software package.

    Findings: A preliminary review of the first two years of data collection indicates that during that time, SBA has had a substantial impact on school operations and instructional practices. The following findings are descriptive, however, when the data analysis is completed, we will have longitudinal findings and results correlated to student achievement data. Some examples of key findings include:
  • Principals identified three strategies as the most important for improving their schools: increasing the use of student achievement data to inform instruction, matching curriculum and instruction with standards and/or assessments, and providing additional instruction to low-achieving students.
  • More than three-quarters of superintendents in our study reported that because of their state's accountability system, the use of data has changed for the better throughout their district.
  • More than two-thirds of principals reported using state test data develop a school improvement plan, focus teacher professional development, make changes to curriculum and instructional materials, and identify students who need additional instructional support.
  • Among our three study states, most teachers reported increased focus on standards, on content emphasized on state tests, and on problem formats used in state tests.
  • Most teachers reported aligning their instruction with state standards and assessments.
  • Approximately 25% of teachers said they focus more on students who are close to proficient on state accountability tests than they would in the absence of a state testing program.
  • More than half of superintendents and principals reported that the academic rigor of the curriculum had changed for the better in their district/school as a result of their state accountability system.
  • Over three-quarters of teachers reported that because of pressure to meet Adequate Yearly Progress targets, they were focusing more on improving student achievement.
  • The most consistent obstacle to making adequate yearly progress was the requirement to meet accountability targets within subgroups.
  • More than half of superintendents and principals reported that the morale of the district/school staff had changed for the worse as a result of their state's accountability system.

PROJECT PUBLICATIONS:
The papers listed below can be found at: http://www.rand.org/education/projects/isba/.

Stecher, B. and S. Naftel. (2006) Implementing Standards-Based Accountability (ISBA). WR380-EDU, RAND, Santa Monica, CA.

Russell, J. and Jennifer Sloan McCombs. (2006) Superintendent, Principal, and Teacher Understanding of and Attitudes about Accountability. WR-376-EDU, RAND, Santa Monica, CA.

Marsh, J. and A. Robyn. (2006) School and District Improvement Efforts in Response to the No Child Left Behind Act, WR-382-EDU, RAND, Santa Monica, CA.

Hamilton, L. and M. Berends. (2006) Instructional Practices Related to Standards and Assessments, WR-374-EDU, RAND, Santa Monica, CA.

Stecher, B. and L. Hamilton. (2006) Using Test-Core Data in the Classroom, WR-375-EDU, RAND, Santa Monica, CA.

Hamilton, L. and B. Stecher. (2006) Measuring Instructional Responses to Standards-Based Accountability, WR-373-EDU, RAND, Santa Monica, CA.

ON THE WEB:
You can learn more about this project by visiting the RAND ISBA website at http://www.rand.org/education/projects/isba/.