Administration of the American Board of Anesthesiology's virtual APPLIED Examination: successes, challenges, and lessons learned.

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
Read More Add to Saved list
  • Additional Information
    • Source:
      Publisher: BioMed Central Country of Publication: England NLM ID: 101088679 Publication Model: Electronic Cited Medium: Internet ISSN: 1472-6920 (Electronic) Linking ISSN: 14726920 NLM ISO Abbreviation: BMC Med Educ Subsets: MEDLINE
    • Publication Information:
      Original Publication: London : BioMed Central, [2001-
    • Subject Terms:
      Anesthesiology*/education ; Specialty Boards* ; Educational Measurement*/methods ; COVID-19*; Humans ; United States ; Clinical Competence/standards ; Certification/standards ; SARS-CoV-2 ; Pandemics
    • Abstract:
      In response to the COVID-19 pandemic, the American Board of Anesthesiology transitioned from in-person to virtual administration of its APPLIED Examination, assessing more than 3000 candidates for certification purposes remotely in 2021. Four hundred examiners were involved in delivering and scoring Standardized Oral Examinations (SOEs) and Objective Structured Clinical Examinations (OSCEs). More than 80% of candidates started their exams on time and stayed connected throughout the exam without any problems. Only 74 (2.5%) SOE and 45 (1.5%) OSCE candidates required rescheduling due to technical difficulties. Of those who experienced "significant issues", concerns with OSCE technical stations (interpretation of monitors and interpretation of echocardiograms) were reported most frequently (6% of candidates). In contrast, 23% of examiners "sometimes" lost connectivity during their multiple exam sessions, on a continuum from minor inconvenience to inability to continue. 84% of SOE candidates and 89% of OSCE candidates described "smooth" interactions with examiners and standardized patients/standardized clinicians, respectively. However, only 71% of SOE candidates and 75% of OSCE candidates considered themselves to be able to demonstrate their knowledge and skills without obstacles. When compared with their in-person experiences, approximately 40% of SOE examiners considered virtual evaluation to be more difficult than in-person evaluation and believed the remote format negatively affected their development as an examiner. The virtual format was considered to be less secure by 56% and 40% of SOE and OSCE examiners, respectively. The retirement of exam materials used virtually due to concern for compromise had implications for subsequent exam development. The return to in-person exams in 2022 was prompted by multiple factors, especially concerns regarding standardization and security. The technology is not yet perfect, especially for testing in-person communication skills and displaying dynamic exam materials. Nevertheless, the American Board of Anesthesiology's experience demonstrated the feasibility of conducting large-scale, high-stakes oral and performance exams in a virtual format and highlighted the adaptability and dedication of candidates, examiners, and administering board staff.
      (© 2024. The Author(s).)
    • References:
      Ophthalmology. 2020 Aug;127(8):e53-e54. (PMID: 32416123)
      Ann Surg. 2021 Sep 1;274(3):467-472. (PMID: 34183516)
      Anesth Analg. 2020 Jan;130(1):258-264. (PMID: 31688077)
      J Clin Anesth. 2023 Dec;91:111258. (PMID: 37734196)
      Ophthalmology. 2021 Dec;128(12):1669-1671. (PMID: 34446303)
      Plast Reconstr Surg Glob Open. 2023 Feb 17;11(2):e4822. (PMID: 36817278)
      Anesth Analg. 2019 Nov;129(5):e159-e162. (PMID: 31613812)
      Anesth Analg. 2019 Nov;129(5):1394-1400. (PMID: 31219924)
      Anesthesiology. 2018 Oct;129(4):812-820. (PMID: 29965814)
      Am J Surg. 2021 Apr;221(4):764-767. (PMID: 33563463)
      J Clin Anesth. 2013 Feb;25(1):12-9. (PMID: 23391341)
      AJOG Glob Rep. 2022 Nov 18;2(4):100136. (PMID: 36478663)
      Anesthesiology. 2018 Apr;128(4):813-820. (PMID: 29251641)
      Anesth Analg. 2020 Nov;131(5):1412-1418. (PMID: 33079864)
      Acad Med. 2024 Jun 7;:. (PMID: 38857338)
      Anesth Analg. 2021 Jul 1;133(1):226-232. (PMID: 33481404)
      J Vasc Surg. 2022 Nov;76(5):1398-1404.e4. (PMID: 35760241)
      Anesth Analg. 2021 Nov 1;133(5):1331-1341. (PMID: 34517394)
      AEM Educ Train. 2023 Mar 26;7(2):e10850. (PMID: 36994316)
      J Contin Educ Health Prof. 2013 Fall;33 Suppl 1:S20-35. (PMID: 24347150)
      Anesthesiology. 2020 Apr;132(4):908-916. (PMID: 31977521)
      J Surg Educ. 2021 May-Jun;78(3):740-745. (PMID: 32994158)
    • Contributed Indexing:
      Keywords: Virtual APPLIED Examination; Virtual Objective Structured Clinical Examination; Virtual Standardized Oral Examination; Virtual candidate experiences; Virtual examiner experiences
    • Publication Date:
      Date Created: 20240711 Date Completed: 20240712 Latest Revision: 20240714
    • Publication Date:
      20240714
    • Accession Number:
      PMC11241991
    • Accession Number:
      10.1186/s12909-024-05694-7
    • Accession Number:
      38992662