Clinical assessment at UWSOM is changing!
WWAMI-wide changes to the clinical assessment system are planned for Spring 2025.
Research into assessment in higher education has provided ample evidence that our assessment practices may show considerable weaknesses. Often, the reliability and validity of assessments leave much to be desired, and assessment often has undesirable and unintended effects on student learning and/or teacher behavior.
- Expanding notion of what it means to be a doctor in the 21st century
- Inflexible training models
- To improve educational and clinical outcomes
- To strive for equity
- Our grading system is not equitable
- How clinical grades are determined is not transparent
- Lack of student trust in clinical grades
- Fixed mindset on grades
Acknowledging that there is no single assessment modality that meets all quality criteria, assessment practices must shift from a focus on isolated assessment methods toward a whole systems approach. The goal of a clinical assessment program, intertwined with the educational program and students’ learning trajectories, is that assessment should serve the purposes of, for, and as learning.
How did we get here?
On March 7, 2022, the UWSOM Curriculum Committee, acting upon the recommendations of the Program Evaluation and Assessment Committee (PEAC) and feedback from the Patient Care Committee, charged a workgroup to explore making broad changes to clerkship grading, centering equity and transparency.
The charge included the need to:
- Create an equitable assessment strategy
- Center transparency
- Implement competency-based assessments
- Assess a comprehensive and holistic skill set
- Make the MSPE reflect assessment
- Encourage a growth mindset/lifelong learning
It was additionally determined that the initial implementation of any changes to the clinical assessment system would be limited to required third- and fourth-year clerkships.
Members of the Clinical Assessment Workgroup were recruited from a wide range of stakeholders. A list of current workgroup members is below:
|Chair, Student Progress Committee; Faculty, Medicine
|Co-Director, WRITE Program; Assistant Clinical Dean, Idaho
|UW Fellow, Nephrology
|Program Director, Dermatology
|Clerkship Director, Psychiatry
|Clerkship Director, Pediatrics
|College Head, Alaska
|Director, FCM & The Colleges
|Clerkship Director, Surgery
|Site Lead, Family Medicine
|Director, Educational Evaluation
|Assistant Clinical Dean, Montana
|Clerkship Administrator, Pediatrics
|Assistant Director, Curriculum
|Program Director, Emergency Medicine
|Program Director, Emergency Medicine
|Clerkship Director - APCs, Anesthesiology
|Clerkship Director, Dermatology
|Assistant Dean, The Colleges
|Assistant Dean, Clinical Curriculum
|Assistant Clinical Dean, Spokane
|Clerkship Director, Family Medicine
|Director, Educational Experience
|Manager, Curriculum Management System
|Co-Director, WRITE Program; Faculty, Family Medicine
|Clerkship Director, Neurology
|Program Director, Orthopedic Surgery
|Research Scientist, BIME
|Clerkship Administrator, Medicine
|Associate Dean, Curriculum
|Clerkship Director, Obstetrics & Gynecology
|Clerkship Director - E&F Clerkships, Medicine
|Director, CLIME; Faculty, Cascade College
|Liaison, Kaiser Permanente
|Clinical Learning Specialist; Faculty, Cascade College
|Clinical Learning Specialist; Faculty, Medicine
|Clerkship Director - APCs & Sub-Is, Family Medicine
|Associate Dean, Eastern Washington
|Assistant Dean, Student Development
|Assistant Program Director, Emergency Medicine
|Site Director, Madigan
|Program Director, Urology
|Program Director, Medicine
|Assistant Clerkship Director, Obstetrics & Gynecology
|Clerkship Director, Medicine
Phase 1: Prepare COMPLETE
Workgroup members from Phase I defined the guiding principles for the new assessment strategy and explored models of assessment, including both programmatic assessment programs (e.g., CBME) and methods of assessment (e.g., workplace-based assessments). They ultimately recommended that all required clerkships be converted to a pass/fail grading system, with the additional recommendation that the clinical assessment system be reworked with a competency-based medical education (CBME) focus, utilizing entrustable professional activities (EPAs) as a workplace-based assessment.
Phase 2 of the Clinical Assessment Workgroup is currently in progress. More information about Phase 2, including goals and a current meeting schedule, can be found in the Project Phases & Reports section below.
Click the button below to register for an upcoming workgroup meeting:
Project Phases & Reports
The project encompasses the 4 phases detailed below and is currently slated for implementation in Spring 2025. Below you will find important project information and details as we aim for, progress toward, and achieve project milestones. Make sure to visit often as information will be updated regularly.
December 2022 – May 2023
Workgroup members from Phase I defined the guiding principles for the new assessment strategy and explored models of assessment, including both programmatic assessment programs (e.g., CBME) and methods of assessment (e.g., workplace-based assessments). They ultimately recommended that all required clerkships be converted to a pass/fail grading system, with the additional recommendation that the clinical assessment system be reworked with a competency-based medical education (CBME) focus, utilizing entrustable professional activities (EPAs) as a workplace-based assessment. Those recommendations were unanimously approved in Autumn 2023 by UWSOM Curriculum governance committees and the Faculty Council on Academic Affairs (FCAA).
- Recruit a Clinical Assessment Workgroup
- Review reasons for change
- Define the overarching values, goals, and measurable outcomes for a new assessment system
- Select a model for clinical assessment
- December 15, 2022 – Phase 1 Kickoff Meeting
- January 11, 2023 – Workgroup Meeting 2
- February 13, 2023 – Expert talk from Dan Schumacher
- March 8, 2023 – Workgroup Meeting 3 with expert talk from Karen Hauer
- March 4, 2023 – CLIME Grand Rounds: Justin Bullock
- March 13, 2023 – Expert talk from Bob Englander
- March 28, 2023 – CLIME Grand Rounds: Jason Frank
- May 3, 2023 – Workgroup Meeting 4
- Phase 1 Summary Report
September 2023 – December 2024
Phase 2 of the Clinical Assessment Workgroup (the Design Phase) is scheduled to be completed by December 2024 and includes members of the first workgroup as well as additional stakeholders who will be impacted by changes to clinical assessment in required clerkships. This phase of the workgroup will define the components of the clinical assessment system for required clerkships, including:
- A comprehensive set of milestones based on CLOOM’s revised program objectives
- A set of longitudinally-assessed EPAs, including their use, timing, and any required revisions to clerkship curriculum
- The structure and role of clinical competency committees
- Requirements for and intended use of the technology support systems necessary for monitoring student progress across the curriculum
- A model for coaching to competencies
- A guide for determining clerkship grades, including suggested criteria and a standardized decision-making process
- Requirements for faculty development around CBME, EPAs, and the new clinical assessment system
- A strategy for addressing anticipated downstream effects on APCs and electives
It will also develop an implementation strategy for instituting pass/fail grading that aligns with planned changes to the clinical assessment system.
Eight subgroups will be selected to facilitate broad participation and collaboration across key stakeholder groups. Each subgroup, charged with one of the strategic priorities identified above, will receive terms of reference that define their purpose, mandate, timeline, and deliverables. The charge of these subgroups is centered on reviewing the medical education research literature and/or completing an environmental scan and then developing recommendations for review by the larger workgroup. Recommendations will then be presented to the Curriculum Committee and FCAA for approval.
- Launch subgroups to determine components of new assessment system
- Identify core roles and assign responsibilities
- Inventory available resources and request funding for additional required infrastructure
- Create new and/or alter existing supporting infrastructure
- Create a staged implementation plan
- Create a strategic plan for downstream effects of new assessment system
- Adapt required clerkship curriculum to fit with new assessment system
- Create robust resource guides and training materials
- Familiarize faculty, staff, and students with upcoming changes
- Submit all recommended changes to Curriculum governance, FCAA, and/or UW Provost by December 2024
- September 2023 – January 2024 – Topic Engagement Meetings
- January 31, 2024 – Phase 2 Kickoff Meeting
- February 14, 2024 – Workgroup Meeting 2 with expert talk from Michael Ryan
- May 29, 2024 – Workgroup Meeting 3 with expert talk from Bill Cutrer
- Implement approved changes to curriculum
- Implement new assessment methods
- Implement technology support systems
- Continue to build faculty education and program evaluation strategies
- Make adjustments to initial implementation plans and processes as needed
- Track and monitor data to ensure desired outcomes are achieved
- Solicit feedback from stakeholders and other end users
- Identify areas for improvement
CBME is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. This outcomes-based approach to the design, implementation, assessment, and evaluation of a medical education program is structured around a framework of competencies (e.g., AAMC Physician Competency Reference Set). Curriculum is organized around the outcomes expected of a medical student and a student’s advancement is dependent on having achieved those expected outcomes.
CBME Core Components:
- An outcomes competency framework
- Progressive sequencing of competencies
- Learning experiences tailored to competencies
- Teaching tailored to competencies
- Programmatic assessment
An entrustable professional activity (EPA) is a key task of a discipline (i.e. specialty or subspecialty) that an individual can be trusted to perform in a given health care context once sufficient competence has been demonstrated. EPAs are a common approach to CBME around the world. EPAs are linked to a specific stage of the competence continuum and integrate multiple milestones from various roles.
Some people think of an EPA as a basket or a bundle that holds numerous milestones. As students progress through the stages of the continuum, the EPAs become progressively more complex, reflecting students’ achievement of more sophisticated milestones.
The WWAMI Rural Integrated Training Experience (WRITE) program began using EPAs in Spring 2023. Preceptors use a student-generated survey to indicate entrustment level, context of encounter, and give formative feedback.
A milestone is an observable marker of an individual’s ability at a stage of expertise. Milestones illustrate the expected progression of competence from novice to mastery associated with each enabling competency.
- Illustrate the developmental nature, features, and progression of the competencies
- Assist learners in monitoring their own developmental progress
- Allow individuals to monitor their own progress
- Support the identification of learners whose progress is not following the typical development sequence to assist in early intervention
- Guide curriculum development