Estimating the Reliability of Skill Transitions in Longitudinal Diagnostic Classification Models

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
Read More Add to Saved list
  • Author(s): Madeline A. Schellman (ORCID Madeline A. Schellman (ORCID 0000-0002-5008-0040); Matthew J. Madison (ORCID Matthew J. Madison (ORCID 0000-0002-2944-7442)
  • Language:
    English
  • Source:
    Grantee Submission. 2024.
  • Publication Date:
    2024
  • Document Type:
    Reports - Research
  • Additional Information
    • Peer Reviewed:
      Y
    • Source:
      29
    • Sponsoring Agency:
      Institute of Education Sciences (ED)
      National Science Foundation (NSF)
    • Contract Number:
      R305D220020
      1921373
    • Subject Terms:
      Diagnostic Tests; Classification; Models; Psychometrics; Cognitive Measurement; Longitudinal Studies; Test Reliability; Item Response Theory; Measurement; Evaluation Methods
    • Accession Number:
      10.3102/10769986241256032
    • Abstract:
      Diagnostic classification models (DCMs) have grown in popularity as stakeholders increasingly desire actionable information related to students' skill competencies. Longitudinal DCMs offer a psychometric framework for providing estimates of students' proficiency status transitions over time. For both cross-sectional and longitudinal DCMs, it is important that researchers estimate and report reliability so stakeholders and end-users can evaluate the trustworthiness of results. Over the past decade, researchers have developed and applied various metrics for reliability in the DCM framework. This study extends these metrics onto the longitudinal DCM context and consists of three parts: (a) the theory and development of the new longitudinal DCM reliability metrics, (b) a simulation study to examine the performance of the developed metrics and establish thresholds, and (c) an empirical data analysis to illustrate an application of the developed metrics. This paper concludes with a discussion of our recommendations for applying the developed metrics. [This is the online first version of an article published in "Journal of Educational and Behavioral Statistics."]
    • Abstract:
      As Provided
    • IES Funded:
      Yes
    • Publication Date:
      2024
    • Accession Number:
      ED661523