Clinical Assessment System Change
Clinical assessment at UWSOM is changing!
WWAMI-wide changes to assessment in required core clerkships are planned for Spring 2025.
Why change?
Research into assessment in higher education has provided ample evidence that our assessment practices may show considerable weaknesses. Often, the reliability and validity of assessments leave much to be desired, and assessment often has undesirable and unintended effects on student learning and/or teacher behavior.
Nationally:
- Expanding notion of what it means to be a doctor in the 21st century
- Inflexible training models
- To improve educational and clinical outcomes
- To strive for equity
At UW:
- Our grading system is not equitable
- How clinical grades are determined is not transparent
- Lack of student trust in clinical grades
- Fixed mindset on grades
Acknowledging that there is no single assessment modality that meets all quality criteria, assessment practices must shift from a focus on isolated assessment methods toward a whole systems approach. The goal of a clinical assessment program, intertwined with the educational program and students’ learning trajectories, is that assessment should serve the purposes of, for, and as learning.
How did we get here?
On March 7, 2022, the UWSOM Curriculum Committee, acting upon the recommendations of the Program Evaluation and Assessment Committee (PEAC) and feedback from the Patient Care Committee, charged a workgroup to explore making broad changes to clerkship grading, centering equity and transparency.
The charge included the need to:
- Create an equitable assessment strategy
- Center transparency
- Implement competency-based assessments
- Assess a comprehensive and holistic skill set
- Make the MSPE reflect assessment
- Encourage a growth mindset/lifelong learning
It was additionally determined that the initial implementation of any changes to the clinical assessment system would be limited to required third- and fourth-year clerkships.
Members of the Clinical Assessment Workgroup were recruited from a wide range of stakeholders. A list of current workgroup members is below:
| NAME | ROLE |
|---|---|
| Phaedra Allen | Business Analyst, ALT |
| Erikka Allhusen | Faculty, Pediatrics |
| Ayana Amaechi | MS3, Idaho |
| Amy Baernstein | Chair, Student Progress Committee; Faculty, Medicine |
| Frank Batcha | Co-Director, WRITE Program; Assistant Clinical Dean, Idaho |
| Justin Bullock | UW Fellow, Nephrology |
| Markus Boos | Program Director, Dermatology |
| Paul Borghesani | Clerkship Director, Psychiatry |
| Bekah Burns | Clerkship Director, Pediatrics |
| Sarah Byquist | MS3, Spokane |
| Andrea Cabarello | College Head, Alaska |
| Julie Calcavecchia | Director, FCM & The Colleges |
| Kris Calhoun | Clerkship Director, Surgery |
| Michael Campion | Director, ALT |
| Andy Castrodale | Site Lead, Family Medicine |
| Adam Crandall | MS3, Idaho |
| Erica Crittendon | MS3, Seattle |
| Matt Cunningham | Director, Educational Evaluation |
| Sophia Deklotz | MS4, Idaho |
| Neha Deshpande | Clerkship Director, Internal Medicine |
| Amy Dettori | Associate Clerkship Director, Pediatrics |
| Aleksei Dingel | MS4, Idaho |
| Kellie Engle | Director, Curriculum |
| Jay Erickson | Assistant Clinical Dean, Montana |
| Sara Fear | Clerkship Administrator, Pediatrics |
| Gina Franco | Assistant Director, Curriculum |
| Douglas Franzen | Program Director, Emergency Medicine |
| Fiona Gallahue | Program Director, Emergency Medicine |
| Jacob Gross | Clerkship Director - APCs, Anesthesiology |
| Emina Gulbis | MS3, Idaho |
| Josiah Hanson | Clerkship Director, Dermatology |
| Molly Jackson | Assistant Dean, The Colleges |
| Joshua Jauregui | Assistant Dean, Clinical Curriculum |
| Geoff Jones | Assistant Clinical Dean, Spokane |
| Karla Kelly | Program Operations Specialist, Learning Environment |
| Mallory Kester | E&F Clerkship Administrator, Internal Medicine |
| Toby Keys | Clerkship Director, Family Medicine |
| Sarubh Khandelwal | Faculty, Surgery |
| Meghan Kiefer | Director, Educational Experience |
| Jordan Kinder | Manager, Curriculum Management System |
| Elana King-Nakaoka | MS3, Idaho |
| Chris Knight | UWMC Faculty, Internal Medicine |
| Amanda Kost | Co-Director, WRITE Program; Faculty, Family Medicine |
| Eric Kraus | Clerkship Director, Neurology |
| Christopher Kweon | Program Director, Orthopedic Surgery |
| Andrea Larson | MS3, Spokane |
| Aidan Loeser | MS3, Seattle |
| Justin Magee | Research Scientist, BIME |
| Erik Malmberg | Assistant Dean, Equity & Medical Student Engagement |
| Carmelita Mason-Richardson | Clerkship Administrator, Medicine |
| Karen McDonough | Director, FCM |
| Heather McPhillips | Associate Dean, Curriculum |
| Vicky Mediratta | Clerkship Director, Obstetrics & Gynecology |
| Susan Merel | Clerkship Director - E&F Clerkships, Medicine |
| Som Mookherjee | Director, CLIME; Faculty, Cascade College |
| Carl Morris | Liaison, Kaiser Permanente |
| Tom Moynihan | Manager, Departmental Computing, DOM IT |
| Sheva Mozafari | MS3, Idaho |
| Linh Ngo | Career Advisor |
| Lan Nguyen | Clerkship Administrator, Dermatology |
| Kim O'Connor | Clinical Learning Specialist; Faculty, Cascade College |
| Doug Paauw | Clinical Learning Specialist; Faculty, Medicine |
| Pam Pentin | Clerkship Director - APCs & Sub-Is, Family Medicine |
| Darryl Potyk | Associate Dean, Eastern Washington |
| Melissa Riesland | Specialist, MSPE |
| Maya Sardesai | Assistant Dean, Student Development |
| Caitlin Schrepel | Assistant Program Director, Emergency Medicine |
| Robin Scott | Educator, WRITE |
| Alicia Scribner | Site Director, Madigan |
| Kristen Seiler | Clerkship Administrator, Surgery |
| Katie Sexton | MS3, Seattle |
| Samartha Shrestha | MS3, Spokane |
| Isha Shrestha | MS3, Seattle |
| Sam Simpson | MS4, Idaho |
| Mathew Sorenson | Program Director, Urology |
| Ken Steinberg | Program Director, Medicine |
| Rachel Straughn | MS3, Seattle |
| Sarah Thomson | Career Advisor |
| Sarah Villareal | Assistant Clerkship Director, Obstetrics & Gynecology |
| Christen White | Assistant Director, ALT |
| Sarah Wood | Director, Student Affairs |
| Jenny Wright | Clerkship Director, Medicine |
| Melissa Zent | Manager, MSPE |
Project Timeline
Phase 1: Prepare COMPLETE
Workgroup members from Phase I defined the guiding principles for the new assessment strategy and explored models of assessment, including both programmatic assessment programs (e.g., CBME) and methods of assessment (e.g., workplace-based assessments). They ultimately recommended that all required clerkships be converted to a pass/fail grading system, with the additional recommendation that the clinical assessment system be reworked with a competency-based medical education (CBME) focus, utilizing entrustable professional activities (EPAs) as a workplace-based assessment.
Phase 2 of the Clinical Assessment Workgroup is currently in progress. More information about Phase 2, including goals and a current meeting schedule, can be found in the Project Phases & Reports section below.
Click the button below to register for an upcoming workgroup meeting:
Project Phases & Reports
The project encompasses the 4 phases detailed below and is currently slated for implementation in Spring 2025. Below you will find important project information and details as we aim for, progress toward, and achieve project milestones. Make sure to visit often as information will be updated regularly.
December 2022 – May 2023
Workgroup members from Phase I defined the guiding principles for the new assessment strategy and explored models of assessment, including both programmatic assessment programs (e.g., CBME) and methods of assessment (e.g., workplace-based assessments). They ultimately recommended that all required clerkships be converted to a pass/fail grading system, with the additional recommendation that the clinical assessment system be reworked with a competency-based medical education (CBME) focus, utilizing entrustable professional activities (EPAs) as a workplace-based assessment. Those recommendations were unanimously approved in Autumn 2023 by UWSOM Curriculum governance committees and the Faculty Council on Academic Affairs (FCAA).
Phase Milestones
- Recruit a Clinical Assessment Workgroup
- Review reasons for change
- Define the overarching values, goals, and measurable outcomes for a new assessment system
- Select a model for clinical assessment
Meetings
- December 15, 2022 – Phase 1 Kickoff Meeting
- January 11, 2023 – Workgroup Meeting 2
- February 13, 2023 – Expert talk from Dan Schumacher
- March 8, 2023 – Workgroup Meeting 3 with expert talk from Karen Hauer
- March 4, 2023 – CLIME Grand Rounds: Justin Bullock
- March 13, 2023 – Expert talk from Bob Englander
- March 28, 2023 – CLIME Grand Rounds: Jason Frank
- May 3, 2023 – Workgroup Meeting 4
- Phase 1 Summary Report
September 2023 – December 2024
Phase 2 of the Clinical Assessment Workgroup (the Design Phase) is scheduled to be completed by December 2024 and includes members of the first workgroup as well as additional stakeholders who will be impacted by changes to clinical assessment in required clerkships. This phase of the workgroup will define the components of the clinical assessment system for required clerkships, including:
- A comprehensive set of milestones based on CLOOM’s revised program objectives
- A set of longitudinally-assessed EPAs, including their use, timing, and any required revisions to clerkship curriculum
- The structure and role of clinical competency committees
- Requirements for and intended use of the technology support systems necessary for monitoring student progress across the curriculum
- A model for coaching to competencies
- A guide for determining clerkship grades, including suggested criteria and a standardized decision-making process
- Requirements for faculty development around CBME, EPAs, and the new clinical assessment system
- A strategy for addressing anticipated downstream effects on APCs and electives
It will also develop an implementation strategy for instituting pass/fail grading that aligns with planned changes to the clinical assessment system.
Five subgroups have been selected to facilitate broad participation and collaboration across key stakeholder groups. Each subgroup, charged with one or more of the strategic priorities identified above, have received terms of reference that define their purpose, mandate, timeline, and deliverables. The charge of these subgroups is centered on reviewing the medical education research literature and/or completing an environmental scan and then developing recommendations for review by the larger workgroup. Recommendations will then be presented to the Curriculum Committee and FCAA for approval.
Click here for more information about each subgroup’s charge.
Phase Milestones
- Launch subgroups to determine components of new assessment system
- Identify core roles and assign responsibilities
- Inventory available resources and request funding for additional required infrastructure
- Create new and/or alter existing supporting infrastructure
- Create a staged implementation plan
- Create a strategic plan for downstream effects of new assessment system
- Adapt required clerkship curriculum to fit with new assessment system
- Create robust resource guides and training materials
- Familiarize faculty, staff, and students with upcoming changes
- Submit all recommended changes to Curriculum governance, FCAA, and/or UW Provost by December 2024
Meetings
- September 2023 – January 2024 – Topic Engagement Meetings
- January 31, 2024 – Phase 2 Kickoff Meeting
- February 14, 2024 – Workgroup Meeting 2 with expert talk from Michael Ryan
- May 29, 2024 – Workgroup Meeting 3 with expert talk from Bill Cutrer
- September 19, 2024 – Workgroup Meeting 4
- December 2, 2024 – Workgroup Meeting 5
Spring 2025
Phase Milestones
- Implement approved changes to curriculum
- Implement new assessment methods
- Implement technology support systems
- Continue to build faculty education and program evaluation strategies
2025 –
Phase Milestones
- Make adjustments to initial implementation plans and processes as needed
- Track and monitor data to ensure desired outcomes are achieved
- Solicit feedback from stakeholders and other end users
- Identify areas for improvement
FAQs
CBME is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. This outcomes-based approach to the design, implementation, assessment, and evaluation of a medical education program is structured around a framework of competencies (e.g., AAMC Physician Competency Reference Set). Curriculum is organized around the outcomes expected of a medical student and a student’s advancement is dependent on having achieved those expected outcomes.
CBME Core Components:
- An outcomes competency framework
- Progressive sequencing of competencies
- Learning experiences tailored to competencies
- Teaching tailored to competencies
- Programmatic assessment
An entrustable professional activity (EPA) is a key task of a discipline (i.e. specialty or subspecialty) that an individual can be trusted to perform in a given health care context once sufficient competence has been demonstrated. EPAs are a common approach to CBME around the world. EPAs are linked to a specific stage of the competence continuum and integrate multiple milestones from various roles.
Some people think of an EPA as a basket or a bundle that holds numerous milestones. As students progress through the stages of the continuum, the EPAs become progressively more complex, reflecting students’ achievement of more sophisticated milestones.
The WWAMI Rural Integrated Training Experience (WRITE) program began using EPAs in Spring 2023. Preceptors use a student-generated survey to indicate entrustment level, context of encounter, and give formative feedback.
A milestone is an observable marker of an individual’s ability at a stage of expertise. Milestones illustrate the expected progression of competence from novice to mastery associated with each enabling competency.
Milestones:
- Illustrate the developmental nature, features, and progression of the competencies
- Assist learners in monitoring their own developmental progress
- Allow individuals to monitor their own progress
- Support the identification of learners whose progress is not following the typical development sequence to assist in early intervention
- Guide curriculum development
Pass/fail grading in clerkships
Programmatic assessment
Workplace-based assessments (WBAs)
Questions? Contact Mary Sargent, sargem@uw.edu.