Evaluating the Validity of National Multiassessment System in Postgraduate Surgical Training: A Retrospective Cohort Study.

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
Read More Add to Saved list
  • Additional Information
    • Source:
      Publisher: Elsevier Country of Publication: United States NLM ID: 101303204 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1878-7452 (Electronic) Linking ISSN: 18787452 NLM ISO Abbreviation: J Surg Educ Subsets: MEDLINE
    • Publication Information:
      Original Publication: New York, N.Y. : Elsevier
    • Subject Terms:
      Education, Medical, Graduate*/methods ; General Surgery*/education ; Internship and Residency* ; Educational Measurement*/methods ; Clinical Competence*; Retrospective Studies ; Humans ; Reproducibility of Results ; Female ; Cohort Studies ; Male ; United States
    • Abstract:
      Objective: The objective of this study was to evaluate the validity evidence supporting the use and interpretation of a multifaceted assessment system in the early years of surgical training.
      Design: This was a national retrospective cohort study analyzing the validity and reliability of an assessment process for surgical residents over a 2-year period. Data from all elements of the assessment process was evaluated using Messick's unified validity framework. Assessments were categorized as Workplace-based, Structured assessment performed in the academic center and Multiple Mini Interview.
      Setting: Our Institution is a health sciences university and the body responsible for the training and certification of all surgeons in our national program. Residents on the Core Surgical Training program undergo multiple assessments over the first 2 years of postgraduate training, both in the workplace and the academic training center, which inform their progression into higher surgical training in their chosen specialty.
      Participants: Data was collected from 2 cohorts of the entire population of postgraduate trainees nationally (N = 114).
      Results: Best practice standards for educational testing aligned with the results supporting the use of this assessment process. Findings indicate a robust assessment system, demonstrating validity evidence in content, response process, internal structure, relations to other variables, and consequences. Composite score reliability of the assessment was 0.89 which demonstrates a highly reliable process. Correlation between workplace-based assessments and standardized tests performed in the simulation setting was also very high (0.93).
      Conclusions: The Core Training Assessment System (CTAS) provides a psychometrically rigorous system to measure trainee competence during the initial years of training. Residency programs of all sizes can replicate the methods described here to demonstrate the validity of their assessment processes, thereby being able to stand over decisions on surgical competency.
      (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)
    • Contributed Indexing:
      Keywords: assessment; simulation; surgical training; workplace-based assessment
    • Publication Date:
      Date Created: 20240919 Date Completed: 20241009 Latest Revision: 20241009
    • Publication Date:
      20241010
    • Accession Number:
      10.1016/j.jsurg.2024.08.008
    • Accession Number:
      39299054