West Virginia Safe and Supportive (S3) Schools Project: Year 2 Implementation Evaluation Report

West Virginia Department of Education. 1900 Kanawha Boulevard East, Charleston, WV 25305. Tel: 304-558-3660; Fax: 304-558-0198; Web site: http://wvde.state.wv.us

N

70

Intervention; Educational Improvement; High Schools; Educational Environment; School Culture; Program Implementation; Fidelity; Scoring Rubrics; Program Effectiveness; Program Evaluation; Scores; Comparative Analysis; Student Attitudes; Teacher Attitudes; Surveys; Effect Size; Secondary School Students; Student Behavior; Standards; Expectation; Teacher Behavior; Secondary School Teachers; Statistical Analysis; Nonparametric Statistics

The 4-year federal Safe and Supportive Schools (S3) program supports targeted interventions to improve and measure conditions for learning at the high school level. For 2011-2012 (Year 2), two evaluation questions were investigated: (EQ1) To what extent do participating schools implement the program with fidelity relative to the WV Model for Positive School Climate (WVMPSC), and (EQ2) to what extent do program initiatives improve school climate and culture? Method of study: To assess implementation fidelity, we developed 4-point rubrics for each WVMPSC core activity, which were used in school level assessments by school climate specialists (SCSs) and S3 school teams. To assess school climate improvement, we compared overall school climate index scores for S3 schools between the 2010-11 and the 2011-12 school years to assess change over time, and in the WV School Climate Survey asked students and staff to indicate whether 22 items corresponding to the school climate index have changed compared to the previous year. Findings: Regarding fidelity of implementation, across most core activities--aligned with all stages of implementation--improvements were made, moving from being altogether missing or implemented with weak fidelity in 2011, to being implemented at weak to moderate fidelity in 2012. Both school-based S3 teams and SCSs indicated marked improvements relative to the strategic steps of the WVMPSC over the 2 years. SCSs tended to be more guarded in their assessments, however. For some core activities they indicated schools' implementation fidelity to be at lower levels than the school S3 teams rated themselves, including (a) informing parents and community partners about the S3 initiative and securing their commitment; (b) building understanding of S3 behavioral norms among school staff; and (c) using assessment results to identify factors contributing to school climate problems, set priorities or plan activities, and select appropriate interventions. Regarding impacts of the S3 program on school climate and culture, S3 intervention schools showed significant improvement, with medium to large effect sizes in school climate as measured by the WV School Climate Index. Based on survey data, however, there appears to be a fairly wide gulf between students and staff, with students much more likely to report that conditions stayed about the same, whereas staff were much more likely to report conditions had gotten better. Limitations of study: It will not be possible to determine if the improvements in the Index observed in this year's study are genuine until data are collected for the full 4 years (including a comparison group of nonintervention schools). Recommendations: Schools should (a) establish or refine behavior norms and expectations to be brief, positively stated, and inclusive of students and staff; (b) expand approaches for communicating and teaching behavior norms and expectations; (c) select and implement school climate interventions based on thorough assessments of factors leading to school climate problems; and (d) investigate gaps between students' and staffs' perceptions of school climate improvements, to identify factors driving the perceptions of both groups. Appended are: (1) Strategic Steps of the WV Model for a Positive School Climate; (2) Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate; (3) School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012; (4) S3 Team and School Climate Specialist Fidelity Assessment Ratings, 2011 to 2012; (5) Comparison of S3 Team and School Climate Specialist Fidelity Assessments Ratings, 2011 and 2012; and (6) Chi Square Analysis of Student and Staff Perceptions of School Climate Change.

As Provided

11

2016

ED565480