Date on Master's Thesis/Doctoral Dissertation
Counseling and Human Development
Counseling and Personnel Services, PhD
Valentine, Jeffrey C.
Committee Co-Chair (if applicable)
Immekus, Jason C.
Natesan Batley, Prathiba
School climate; practitioner-developed instruments; alignment method; measurement invariance; alidity testing
School climate is increasingly recognized by scholars and policymakers as a crucial factor associated with students’ educational experiences. Hence, practitioners endeavor to equitably measure and improve school climate to promote favorable student academic and behavior outcomes. Unfortunately, school climate research is fragmented, and a research-practice gap exists in best scale development and validity testing practices. The result is a proliferation of practitioner-developed school climate measures lacking solid theory-grounding and evidence to support intended score interpretations and uses. In response, Whitehouse et al. (2021) proposed a validity testing framework for practitioner-developed instruments aimed at supporting culturally responsive school climate measurement. Their framework, however, suffers from key limitations regarding the transparency of content validity assessment, breadth of validity evidence reported, and methods used to examine measurement invariance. Therefore, this study sought to replicate and extend their validity testing framework by using a standardized rubric to assess content validity, examining measurement invariance via the alignment method, and analyzing the predictive validity of group mean scores. By applying the extended validity testing framework to a practitioner-developed school climate student survey, the study also aimed to provide useful evidence to a large urban district with respect to the validity of comparing survey scores across Black and White middle school student groups and using scores to inform continuous improvement of student learning. Results suggest the extended framework is superior to the original for obtaining general content, factorial, and predictive validity evidence, and assessing measurement invariance across racial subgroups, provided the number and size of groups are adequate. Findings suggest the district’s middle school student survey is culturally responsive, although it may not sufficiently address all critical school climate dimensions. To improve the survey, the district must settle on a clear definition and taxonomy of school climate to facilitate a program of validity testing, and publicly document all available validity evidence. Future studies should clarify alignment sample size and simulation study requirements and extend the framework to assess additional validity concerns and for use with person-centered approaches.
Leach, Stephen, "A psychometric analysis of the comprehensive school survey (CSS) middle school student version." (2022). Electronic Theses and Dissertations. Paper 4031.