Clinical Assessment System Change

Clinical assessment at UWSOM is changing!
WWAMI-wide changes to the clinical assessment system are planned for Spring 2025.

 

Why change?

Research into assessment in higher education has provided ample evidence that our assessment practices may show considerable weaknesses. Often, the reliability and validity of assessments leave much to be desired, and assessment often has undesirable and unintended effects on student learning and/or teacher behavior.

Nationally:
  • Expanding notion of what it means to be a doctor in the 21st century​
  • Inflexible training models​
  • To improve educational and clinical outcomes​
  • To strive for equity​
At UW:
  • Our grading system is not equitable​
  • How clinical grades are determined is not transparent​
  • Lack of student trust in clinical grades
  • Fixed mindset on grades​

Acknowledging that there is no single assessment modality that meets all quality criteria, assessment practices must shift from a focus on isolated assessment methods toward a whole systems approach. The goal of a clinical assessment program, intertwined with the educational program and students’ learning trajectories, is that assessment should serve the purposes of, for, and as learning.

How did we get here?

On March 7, 2022, the UWSOM Curriculum Committee, acting upon the recommendations of the Program Evaluation and Assessment Committee (PEAC) and feedback from the Patient Care Committee, charged a workgroup to explore making broad changes to clerkship grading, centering equity and transparency.

The charge included the need to:

  • Create an equitable assessment strategy
  • Center transparency
  • Implement competency-based assessments
  • Assess a comprehensive and holistic skill set
  • Make the MSPE reflect assessment
  • Encourage a growth mindset/lifelong learning

It was additionally determined that the initial implementation of any changes to the clinical assessment system would be limited to required third- and fourth-year clerkships.

Members of the Clinical Assessment Workgroup were recruited from a wide range of stakeholders. A list of current workgroup members is below:

NAMEROLE
Erikka AllhusenFaculty, Pediatrics
Ayana AmaechiMS3, Idaho
Amy BaernsteinChair, Student Progress Committee; Faculty, Medicine
Frank BatchaCo-Director, WRITE Program; Assistant Clinical Dean, Idaho
Justin BullockUW Fellow, Nephrology
Markus BoosProgram Director, Dermatology
Paul BorghesaniClerkship Director, Psychiatry
Bekah BurnsClerkship Director, Pediatrics
Sarah ByquistMS3, Spokane
Andrea CabarelloCollege Head, Alaska
Julie CalcavecchiaDirector, FCM & The Colleges
Kris CalhounClerkship Director, Surgery
Michael CampionDirector, ALT
Andy CastrodaleSite Lead, Family Medicine
Adam CrandallMS3, Idaho
Erica CrittendonMS3, Seattle
Matt CunninghamDirector, Educational Evaluation
Sophia DeklotzMS4, Idaho
Aleksei DingelMS4, Idaho
Kellie EngleDirector, Curriculum
Jay EricksonAssistant Clinical Dean, Montana
Sara FearClerkship Administrator, Pediatrics
Gina FrancoAssistant Director, Curriculum
Douglas FranzenProgram Director, Emergency Medicine
Fiona GallahueProgram Director, Emergency Medicine
Jacob GrossClerkship Director - APCs, Anesthesiology
Emina GulbisMS3, Idaho
Josiah HansonClerkship Director, Dermatology
Molly JacksonAssistant Dean, The Colleges
Joshua JaureguiAssistant Dean, Clinical Curriculum
Geoff JonesAssistant Clinical Dean, Spokane
Toby KeysClerkship Director, Family Medicine
Sarubh KhandelwalFaculty, Surgery
Meghan KieferDirector, Educational Experience
Jordan KinderManager, Curriculum Management System
Elana King-NakaokaMS3, Idaho
Amanda KostCo-Director, WRITE Program; Faculty, Family Medicine
Eric KrausClerkship Director, Neurology
Christopher KweonProgram Director, Orthopedic Surgery
Andrea LarsonMS3, Spokane
Aidan LoeserMS3, Seattle
Justin MageeResearch Scientist, BIME
Carmelita Mason-RichardsonClerkship Administrator, Medicine
Karen McDonoughDirector, FCM
Heather McPhillipsAssociate Dean, Curriculum
Vicky MedirattaClerkship Director, Obstetrics & Gynecology
Susan MerelClerkship Director - E&F Clerkships, Medicine
Som MookherjeeDirector, CLIME; Faculty, Cascade College
Carl MorrisLiaison, Kaiser Permanente
Sheva MozafariMS3, Idaho
Linh NgoCareer Advisor
Kim O'ConnorClinical Learning Specialist; Faculty, Cascade College
Doug PaauwClinical Learning Specialist; Faculty, Medicine
Pam PentinClerkship Director - APCs & Sub-Is, Family Medicine
Darryl PotykAssociate Dean, Eastern Washington
Maya SardesaiAssistant Dean, Student Development
Caitlin SchrepelAssistant Program Director, Emergency Medicine
Robin ScottEducator, WRITE
Alicia ScribnerSite Director, Madigan
Katie SextonMS3, Seattle
Samartha ShresthaMS3, Spokane
Isha ShresthaMS3, Seattle
Sam SimpsonMS4, Idaho
Mathew SorensonProgram Director, Urology
Ken SteinbergProgram Director, Medicine
Rachel StraughnMS3, Seattle
Sarah ThomsonCareer Advisor
Sarah VillarealAssistant Clerkship Director, Obstetrics & Gynecology
Jenny WrightClerkship Director, Medicine

Project Timeline

Phase 1: Prepare COMPLETE

Workgroup members from Phase I defined the guiding principles for the new assessment strategy and explored models of assessment, including both programmatic assessment programs (e.g., CBME) and methods of assessment (e.g., workplace-based assessments). They ultimately recommended that all required clerkships be converted to a pass/fail grading system, with the additional recommendation that the clinical assessment system be reworked with a competency-based medical education (CBME) focus, utilizing entrustable professional activities (EPAs) as a workplace-based assessment.

Phase 2 of the Clinical Assessment Workgroup is currently in progress. More information about Phase 2, including goals and a current meeting schedule, can be found in the Project Phases & Reports section below.

Click the button below to register for an upcoming workgroup meeting:

Project Phases & Reports

The project encompasses the 4 phases detailed below and is currently slated for implementation in Spring 2025. Below you will find important project information and details as we aim for, progress toward, and achieve project milestones. Make sure to visit often as information will be updated regularly.

December 2022 – May 2023

Workgroup members from Phase I defined the guiding principles for the new assessment strategy and explored models of assessment, including both programmatic assessment programs (e.g., CBME) and methods of assessment (e.g., workplace-based assessments). They ultimately recommended that all required clerkships be converted to a pass/fail grading system, with the additional recommendation that the clinical assessment system be reworked with a competency-based medical education (CBME) focus, utilizing entrustable professional activities (EPAs) as a workplace-based assessment. Those recommendations were unanimously approved in Autumn 2023 by UWSOM Curriculum governance committees and the Faculty Council on Academic Affairs (FCAA).

Phase Milestones
  • Recruit a Clinical Assessment Workgroup​
  • Review reasons for change​
  • Define the overarching values, goals, and measurable outcomes for a new assessment system​​
  • Select a model for clinical assessment
Meetings

September 2023 – December 2024

Phase 2 of the Clinical Assessment Workgroup (the Design Phase) is scheduled to be completed by December 2024 and includes members of the first workgroup as well as additional stakeholders who will be impacted by changes to clinical assessment in required clerkships. This phase of the workgroup will define the components of the clinical assessment system for required clerkships, including:

  • A comprehensive set of milestones based on CLOOM’s revised program objectives
  • A set of longitudinally-assessed EPAs, including their use, timing, and any required revisions to clerkship curriculum
  • The structure and role of clinical competency committees
  • Requirements for and intended use of the technology support systems necessary for monitoring student progress across the curriculum
  • A model for coaching to competencies
  • A guide for determining clerkship grades, including suggested criteria and a standardized decision-making process
  • Requirements for faculty development around CBME, EPAs, and the new clinical assessment system
  • A strategy for addressing anticipated downstream effects on APCs and electives

It will also develop an implementation strategy for instituting pass/fail grading that aligns with planned changes to the clinical assessment system.

Eight subgroups will be selected to facilitate broad participation and collaboration across key stakeholder groups. Each subgroup, charged with one of the strategic priorities identified above, will receive terms of reference that define their purpose, mandate, timeline, and deliverables. The charge of these subgroups is centered on reviewing the medical education research literature and/or completing an environmental scan and then developing recommendations for review by the larger workgroup. Recommendations will then be presented to the Curriculum Committee and FCAA for approval.

Phase Milestones
  • Launch subgroups to determine components of new assessment system
  • Identify core roles and assign responsibilities​​​
  • Inventory available resources and request funding for additional required infrastructure
  • Create new and/or alter existing supporting infrastructure​
  • Create a staged implementation plan
  • Create a strategic plan for downstream effects of new assessment system
  • Adapt required clerkship curriculum to fit with new assessment system​
  • Create robust resource guides and training materials
  • Familiarize faculty, staff, and students with upcoming changes​​
  • Submit all recommended changes to Curriculum governance, FCAA, and/or UW Provost by December 2024
Meetings

Spring 2025

Phase Milestones
  • Implement approved changes to curriculum
  • Implement new assessment methods
  • Implement technology support systems​
  • Continue to build faculty education and program evaluation strategies

2025 –

Phase Milestones
  • Make adjustments to initial implementation plans and processes as needed​
  • Track and monitor data to ensure desired outcomes are achieved​
  • Solicit feedback from stakeholders and other end users​​
  • Identify areas for improvement

FAQs

CBME is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. This outcomes-based approach to the design, implementation, assessment, and evaluation of a medical education program is structured around a framework of competencies (e.g., AAMC Physician Competency Reference Set). Curriculum is organized around the outcomes expected of a medical student and a student’s advancement is dependent on having achieved those expected outcomes.

CBME Core Components:

  • An outcomes competency framework
  • Progressive sequencing of competencies
  • Learning experiences tailored to competencies
  • Teaching tailored to competencies
  • Programmatic assessment

An entrustable professional activity (EPA) is a key task of a discipline (i.e. specialty or subspecialty) that an individual can be trusted to perform in a given health care context once sufficient competence has been demonstrated. EPAs are a common approach to CBME around the world. EPAs are linked to a specific stage of the competence continuum and integrate multiple milestones from various roles.

Some people think of an EPA as a basket or a bundle that holds numerous milestones. As students progress through the stages of the continuum, the EPAs become progressively more complex, reflecting students’ achievement of more sophisticated milestones.

The WWAMI Rural Integrated Training Experience (WRITE) program began using EPAs in Spring 2023. Preceptors use a student-generated survey to indicate entrustment level, context of encounter, and give formative feedback.

A milestone is an observable marker of an individual’s ability at a stage of expertise. Milestones illustrate the expected progression of competence from novice to mastery associated with each enabling competency.

Milestones:

  • Illustrate the developmental nature, features, and progression of the competencies
  • Assist learners in monitoring their own developmental progress
  • Allow individuals to monitor their own progress
  • Support the identification of learners whose progress is not following the typical development sequence to assist in early intervention
  • Guide curriculum development

Pass/fail grading in clerkships
Programmatic assessment
Workplace-based assessments (WBAs)

Questions? Contact Mary Sargent, sargem@uw.edu.