CQI Spotlight – We heard you!

Element 3.2 Community of Scholars/Research Opportunities (April 2023)

Jung Lee (Director of EQI), Dr. Cynthia Sprenger (Director of Medical Student Scholarship), Toby Keys (Director, Rural Underserved Opportunities Program (RUOP)) and Dr. Collette Abbott (MD, Global Health Immersion Program (GHIP) Director)

Introduction: What does EQI do with your survey feedback?

The Office of Educational Quality Improvement (EQI) oversees the medical school education accreditation at UWSOM. Maintaining the medical school accreditation requires continuous monitoring on 12 standards and 93 elements of the medical education program required by the accrediting body, Liaison Committee on Medical Education (LCME). While monitoring the compliance level of 12 standards and 93 elements, EQI office relies heavily on student feedback.  EQI utilizes data from student surveys including the End-of-Phase surveys, Year-2-Questionnaire (Y2Q) and Graduation Questionnaire (GQ) surveys. End-of-Phase surveys are conducted internally and Y2Q and GQ are administered by the Association of American Medical Colleges (AAMC).

EQI understands that completing these surveys can feel onerous, at times, for busy medical students.  However, they are critical to improving our medical school.  Student feedback guides the EQI’s work in continuous quality improvement (CQI) in partnership with responsible teams. The main CQI tool we use is the Plan-Do-Study-Act (PDSA) tool, which includes the key components for addressing gaps in quality in the educational programs.  These components include the following: identifying root causes (Plan), collecting data (Do), implementing changes (Study), and measuring the impacts of the implementation strategies (Act).

Element 3.2 Community of Scholars/Research Opportunities

In this newsletter, we will share the status of the LCME Element 3.2 Community of Scholars/Research Opportunities.

The following diagram shows the LCME decisions on element 3.2 since the last accreditation visit in 2018.

• In 2019, LCME determined the element to be ‘Satisfactory with Need for Monitoring’ and noted that ‘the school has identified additional resources to improve student access to research opportunities at its regional campuses; data on student satisfaction with these improvements are not yet available.’
• In 2021 and 2022, the school submitted two status reports, demonstrating the CQI efforts with data from the End-of-Phase surveys.
• In November 2022, LCME downgraded the Element from ‘Satisfactory with Need for Monitoring’ to ‘Unsatisfactory’ and required another status report due in December 2023. See more info under ‘PLAN.’

Diagram 1. Citation on 3.2 and LCME Decisions since 2018 LCME Accreditation Visit

To resolve this citation from LCME, the school is required to:

1. Demonstrate how the school is determining the effectiveness of strategies to provide both clinical and non-clinical research/scholarship opportunities.
2. Provide the rationale for further changes.
3. Address the persistent student dissatisfaction with the ease of access to research opportunities and the sufficiency of information about the opportunities.

The Office of EQI partners with the Triple I Scholarship team (which includes the directors of RUOP, GHIP, and SoD/SoI) to develop strategies utilizing a Plan-Do-Study-Act (PDSA) tool.

PLAN

During this phase, the units gathered data and feedback to identify areas to improve.

The Triple I team surveys MS2 students experience after they complete their projects in fall. In 2021, the office compiled MS2 satisfaction rate with Ease of Access to Research Opportunities from 2018 to 2020 and submitted the report. See below Graph 1.

For the 2022 status report, LCME asked the school to survey all four cohorts’ satisfaction rates with ease of access to the research opportunities near Seattle as well as near Foundations site.  satisfaction with sufficiency of information about research opportunities.  See Graph 2, 3 and 4.

Based on survey data and meetings with stakeholders, the EQI and the Triple I team have identified the following as areas for improvement:

• Newly define the scholarly activities at UWSOM: Previously, the definition of research/scholarly activities only included Scholarship of Discovery (SoD) and Scholarship of Integration (SoI) and did not include community-based scholarly activities such as Rural and Underserved Opportunities Program (RUOP) and Global Health Immersion Program (GHIP).
• Diversify research opportunities: Students desired increased opportunities in remote research options, local clinical research, other options at regional campuses.
• Strengthen communication strategies: information about available opportunities, processes to get into the programs, the diverse range of scholarly opportunities and impacts of students’ research should be shared with the community to increase awareness of ongoing scholarly activities.
• Address barriers to participate in scholarly activities: Some students have shared their financial or summer curricula barriers to participate in scholarly projects.

DO

In this stage of PDSA, responsible units start implementing their plans to address issues and collecting and analyzing data.

Since November, the Triple I team, the Office of Curriculum, and the school leadership have actively worked on developing strategies as follows:

• Broaden the scope of research/scholarly activities surveys.
  • Scholarly activities reported in LCME surveys will now include the community-based research programs (RUOP and GHIP) as forms of scholarly activities.
  • 2023 End-of-Phase surveys are updated with this definition of scholarship and launched to collect student feedback in March.
• Expand diverse research opportunities.
  • The Office of Curriculum has started developing a Clinical and Translational Research Pathway to support students’ scholarly activities. A pilot of this Pathway is planned to begin with the E23 cohort.
  • The school started partnering with the Institute of Translational Health Sciences (ITHS) to plan a summer research course for students to engage in ITHS’s research practicum with a tentative plan to start with E24 cohort.
• Implement diverse communication strategies.
  • The Office of Curriculum launched a new article called, ‘Scholarship Spotlight’ to highlight students’ scholarly work and share available scholarly opportunities with students.
  • The Triple I Scholarship team plans to provide faculty orientations on how to navigate working with summer students on research projects.
  • The school plans to tighten communications between the leadership and students by frequently updating available research opportunities/resources and sharing important info/reminders for upcoming opportunities.
• Launch Triple I Workgroup composed with students, faculty, and staff to provide recommendations for the Triple I Scholarship program.
  • The Office of Curriculum charged the Triple I Workgroup to gather various stakeholders’ feedback and revisit the III program in multiple ways. The workgroup started examining the current requirement of III Scholarship and exploring the impacts and roles of the current requirement for students’ next steps of career such as residency applications. Also, the workgroup started identifying priorities and unintended consequences, utilizing a UWSOM Equity Tool. The workgroup plans to host multiple meetings with a goal to bring expertise from the finance team (Dean’s Office), the financial aid team, faculty researchers, the career advising team, and a panel of URiM students.

Study

Starting March 14, EQI launched End-of-Phase surveys. During ‘Study’ phase, the Triple I team and EQI will begin close monitoring of the survey results in April. In May and June, the Triple I team and EQI plan to analyze the survey results and study the impacts of implementation strategies.

Act

The last of PDSA cycle is an important phase for refining or developing a next improvement plan. The Triple I team and EQI plan to reflect on what we learn from the previous phases and the results of survey analysis in June and July. Then, we plan to adapt or adopt a plan for the next PDSA cycle.

Next Steps

EQI plans to share the data findings from the End-of-Phase student surveys and updates on the PDSA work in July. Please contact eqi@uw.edu for more information or if you have any questions.

Acknowledgement

While working together with the Triple I team, we learned that this element involves numerous stakeholders and takes a strategic approach with tenacity and focus on supporting students. EQI would like to acknowledge the hard work of the Triple I team, faculty research mentors, clinical preceptors, and staff on Element 3.2 Community of Scholars/Research Opportunities. Lastly, we sincerely thank our students who have provided feedback via surveys and meetings.

Continuous Quality Improvement: Career Advising Team’s Student-Centered and Data-Informed Approaches (December 2022)

Jung Lee, Director of Educational Quality Improvement, Sarah Thomson, Director of Career Advising, Maya Sardesai, Assistant Dean for Student Development

Continuous Quality Improvement (CQI) at UWSOM is essential for two purposes: (1) identifying programmatic strengths and gaps based on available benchmarks; and (2) ensuring compliance with the Liaison Committee on Medical Education (LCME) accreditation standards. Using the framework of CQI, responsible teams identify areas for improvement, establish quality benchmarks and goals, implement strategies tied to goals, and monitor outcomes regularly.

This month, we highlight the Career Advising team’s data-informed and student-centered approach to CQI. The 2018 accreditation visit by LCME resulted in a citation of “satisfactory with a need of monitoring” for Career Advising Services. The main reason for the citation was the lack of data endorsing student satisfaction with the effectiveness of career advising programs.

To resolve this citation, the Career Advising team implemented the Plan-Do-Study-Act (PDSA) tool to address the LCME’s concerns as described in detail below.

PLAN

As the team has strived to incorporate student feedback into their work, the team regularly has invited feedback from their student advisory board, reviews data from internal and external surveys which include all End-of-Phase surveys, internal survey data administered by the team, as well as Graduation Questionnaire (administered by Association of American Medical Colleges). Based on the feedback they received, the Career Advising team identified gaps in students’ access to career advisors and uniformity of services across the five state WWAMI region.

DO

The team implemented the following key strategic initiatives across all three curriculum phases.

• Strengthen the career advising office through investment in personnel and improved operations.
  • Increased staffing from two to three positions, including a Director of Career Advising
  • Maintain regular points of contact and facilitate work of career advising stakeholders throughout WWAMI.
• Implemented the ‘Campus Connections’ model for equitable access to career advising.
  • The ‘Campus Connections’ model is implemented through the Career Planning Process (CPP) which supports UWSOM students in advancing their career goals through longitudinal advising, programming, resources, and engagement opportunities. Each WWAMI site has a dedicated career advisor and each student works with this career advisor throughout their time in medical school.
• Inform students of career advising programs and opportunities through regular communication.
  • The team developed communication strategies to engage with students via various meetings and modes of communications such as follow-up emails, student weekly newsletters, a career services online advising management system, as well as a revamped career advising website.

STUDY

After implementing the initiatives above, the team reviewed student feedback from surveys to monitor the impact of their work. The following table demonstrates increased student satisfaction with the overall quality of Career Advising services over the three-year implementation period. MS3 and MS4 students who work closely with the Career Advising team in preparation for their residency applications especially, expressed strong satisfaction (78% for MS3s and 62% for MS4s in 2020, 83% for MS3s and 85% for MS4s in 2021 and 91% for MS3s and 86% for MS4s in 2022).

Below Graduation Questionnaire (GQ) table from students responses show that MS4 students’ overall satisfaction with career planning services improved significantly from 58% in 2020 to 75% in 2021. In 2022, satisfaction as reported on the GQ declined by 3% to 72% and the team is continuously adapting their strategies to improve the services.

ACT

The key findings from student surveys identified gaps in resources and services, and the need to further develop strategic partnerships. Consequently, the team implemented the following initiatives:

• Increased career advising support and career planning resources for Foundations Phase students.
  • Provided a required 1:1 appointment with dedicated career advisor
  • Provided an Autumn career planning workshop for MS2s
  • Improved web content including the addition of a career planning timeline and a video library of 80+ career exploration and specialty interest group panels
  • Provide seven career panels aligned with the Foundations Phase curriculum in partnership with faculty in 23 specialties
• Increased access & clarity of information available to third-year students for planning their fourth-year.
  • Developed a webinar with multiple SOM partners to provide a planning framework and answer career planning questions
  • Launched a fourth-year career planning webpage and led the vision and development of the general fourth year planning webpages with multiple SOM partners
  • Created 25 Specialty Guides in partnership with Specialty Career Advisors
• Implemented measures to assess student readiness for residency interviews.
  • Provided faculty mock interviews to prepare students for residency interviews
• Increased Career Planning Support for BIPOC, URiM & First-Generation students
  • Increased the number of residents listed in the Residents in Medicine (RIM) Directory who are Black, Indigenous or People of Color. The RIM is a list of UWSOM alumni who have offered to serve as a resource for students preparing to apply to residency
  • Developed the BIPOC Physician Roster

The Career Advising team’s work shows what it means to be student-centered in their services and resources by implementing changes based on student feedback while staying focused on the mission. We would like to acknowledge the tireless work of the rest of the Career Advising team including Tonja Brown and Linh Ngo for their dedicated services to our students across the regions.