Anesthesiology Milestones 2.0—An Update

Background

The Accreditation Council for Graduate Medical Education (ACGME) introduced the concept of Milestones to the graduate medical education community in 2013. Milestones represent key developmental concepts that are used in resident and fellow outcomes-based assessment. The introduction of the Milestones was intended to help programs more effectively track resident and fellow performance over the course of their training. The Milestones also provided a concrete mechanism for programs to use performance outcome data as part of the annual program evaluation and improvement process. The Milestones are organized under the six general competencies of medical knowledge (MK), patient care (PC), interpersonal and communication skills (ICS), practice-based learning and improvement (PBLI), professionalism (PROF), and systems-based practice (SBP) that were originally introduced in 1999. Anesthesiology’s specialty Milestones were implemented in 2014.

Since then, the graduate medical education community has been engaged in various types of research on the Milestones. A large number of validity studies have been carried out by several specialties. The ACGME is tracking the research and provides a bibliography of peer-reviewed articles, commentaries, editorials, and opinion pieces that specifically address the topic of the Milestones. The resources detailed in the bibliography are organized by National-Level Studies (Quantitative/Qualitative, Multi-Institutional Studies (Quantitative/Qualitative), Single Institution Studies (Quantitative/Qualitative), Medical School Milestones Studies (Quantitative/Qualitative), International Studies (Quantitative/Qualitative), and Commentaries/Editorials/Opinion Pieces. This bibliography is available on the Milestones webpage.

Milestones 2.0

At the onset of Milestone implementation in 2013, the ACGME committed to reviewing the process and content in three to five years. This review process, called Milestones 2.0, began in 2016. Anesthesiology has just started the process. The framework for the changes to Milestones 2.0 can be summarized into four areas. The four areas are:

1. selecting the Milestones Development Groups,
2. enhancing community engagement,
3. reducing complexity in the Milestones, and
4. providing additional tools and resources for program and sponsoring institution use.

It is impossible to tell at this point the degree to which the Anesthesiology Milestones will change; the community will not know until the work is complete. As with most specialties, some components of the Milestones (e.g. subcompetencies), may stay the same and some of the specific language may change.

Choosing the Development Group

During the fall of 2018, Anesthesiology Program Directors and Coordinators received a communication from the ACGME announcing the start of the Milestone 2.0 process. This included information regarding an upcoming program director survey designed to elicit feedback on the anesthesiology Milestones, as well as a Call for Volunteers to participate in the review process. The communication contained links to a form for those who wish to volunteer to be a part of the Milestones 2.0 Development Group for Anesthesiology. The Development Group will still include representatives from the American Board of Anesthesiology, the Anesthesiology Review Committee, and the specialty Program Director organizations. It will also include individuals from a variety of regions, program sizes, and experience. The Development Group will also include a public member and at least one resident.

Enhancing community engagement

The recent fall communication from the ACGME also included preparatory guidance for the Milestones feedback survey, which will be sent in January 2019. The feedback survey focuses on the current Patient Care and Medical Knowledge Milestones and will inform the Milestones Development Group during the review and revision process. The feedback survey is an important aspect of Milestones 2.0, as it is the primary mechanism for eliciting feedback directly from program directors. For each Patient Care and Medical Knowledge subcompetency there are six statements with responses for level of agreement (strongly agree, agree, disagree, strongly disagree). The statements include:

1. This set of Milestones represents a realistic progression of knowledge, skills, and behaviors;
2. The set discriminates between meaningful levels of competency;
3. The set should be edited;
4. The set should be deleted;
5. Additional content should be embedded in this set; and
6. I am able to evaluate my resident’s skills on this set.

Data gathered from the survey will be used by the Development Group to determine which areas in the current Patient Care and Medical Knowledge subcompetencies need to be reviewed/revised.

Reducing complexity

The Anesthesiology Development Group will also be tasked with reducing complexity within the Milestones. Currently, Anesthesiology has 25 subcompetencies, many of which contain multiple developmental trajectories. To increase usability, the group will attempt to keep the number of developmental trajectories in each subcompetency to no more than three.

In addition, the Anesthesiology Development Group will be asked to incorporate the ACGME’s newly-established “harmonized” Milestones for the remaining four competencies into the specialty Milestones. These harmonized Milestones were created by a multidisciplinary group and are available for use in Interpersonal and Communication Skills (ICS), Practice-based Learning and Improvement (PBLI), Professionalism (PROF), and Systems-based Practice (SBP). Through a process of public comment conducted in early 2017, they were determined to be appropriate for all specialties. There are eleven sets of harmonized Milestones. ICS has three: Patient- and Family-Centered Communication (ICS1), Interprofessional and Team Communication (ICS2), and Communication within Healthcare Systems (ICS3). PBLI has two: Evidence-Based and Informed Practice (PBLI1) and Reflective Practice and Commitment to Personal Growth (PBLI2). PROF has three: Professional Behavior and Ethical Principles (PROF1), Accountability/Conscientiousness (PROF2), and Self-awareness and Help-seeking (PROF3). Lastly, SBP has three: Patient Safety and Quality Improvement (SBP1), System Navigation for Patient-Centered Care (SBP2), and The Physician’s Role in Healthcare Systems (SBP3). Integrating the harmonized Milestones as a framework into the specialty Milestones will permit a meaningful review and understanding of some of the core principles of graduate medical education. Thus the second task of the Development Group will be to adapt the newly-integrated language of this framework into the Anesthesiology subcompetencies in order to meet the needs of the specialty.

Tools and resources

The final addition to the Milestones 2.0 process will be new tools and resources that will be made available with the revised Milestones. The first resource will be a Supplemental Guide that the Development Group will create to provide programs with an explanation of the intent of each subcompetency, examples of what to look for at each level, appropriate assessment methods, and any other additional resources they think would be useful to programs. This will be made available online as a Word document so that individual programs can input their own assessment methods or examples to tailor it to their needs. Through the process of contextualizing the Supplemental Guide for each individual program, programs will also be creating a shared mental model that can be used by faculty and Clinical Competency Committees. An Implementation Guidebook is also being written to help with the transition to the revised Milestones.

Anesthesiology has just begun this work. The Development Group tentatively plans to meet up to three times during the process, which will take roughly a year to complete. Using that timeline, Anesthesiology programs can anticipate being able to review the draft version of the revised Milestones in late 2019; they will be finalized and ready for implementation during the 2020-21 academic year. The ACGME Milestones Team looks forward to working with the entire Anesthesiology community during this process. Any questions about the review and revision process can be directed to [email protected].

Authors

Sydney Roberts, MHA

Milestones Project Manager at ACGME

Chicago, Illinois

Anne Gravel Sullivan, PhD

Executive Director, Anesthesiology Review Committee and Director, Distance Learning at ACGME

Chicago, Illinois

Laura Edgar, EdD

Executive Director, Milestones Development at ACGME

Chicago, Illinois