Summary of Panels at Annual Meeting

The AUA Annual Meeting occurred on April 26 and 27 in Chicago. The Education Advisory Board presented two panels: “Motivation, Meta-cognition, and Self-Regulation” and “The Science of Longitudinal Assessment.”

The first panel, Motivation, Meta-cognition, and Self-Regulation, had three speakers with the first speaker being Daniel Saddawi-Konefka, M.D., Assistant Professor of Anesthesia, Critical Care, and Pain Medicine at Massachusetts General Hospital. Dr. Saddawi-Konefka is the Program Director for the Core Anesthesiology Residency Program and he spoke on the science of motivation. Motivation is the aspect that moves people to certain behaviors. Motivation is either intrinsic or extrinsic with intrinsic motivation resulting in better outcomes. Extrinsic motivation refers to external rewards such as finances, stature, or material goods. Intrinsic motivation refers to satisfaction and fun. While both forms of motivation will drive behavior, intrinsic motivation produces better learning and creativity. While intrinsic and extrinsic motivation appear to be different, the two intersect and have an impact on motivation. An example would be an interesting job that offers financial bonuses. The fact that the job is interesting reflects intrinsic motivation while the financial bonuses provide the extrinsic motivation. The impact of one on the other was studied when individuals were asked to solve a puzzle. One group was not paid for solving the puzzle while the other group was. The paid group rated solving the puzzle as more interesting as compared to the group who was not paid. However, once the money was removed, interest in the puzzle decreased to a level lower than the group that was not paid. Tangible rewards have an effect on intrinsic motivation, decreasing it. The explanation for this phenomena is the over-justification effect which refers to an interesting activity for which an individual is paid. The individual begins to attribute the interest in the activity to the money and forget the intrinsic interest in the activity. Motivation is primarily based upon self-determination in which there are three basic psychological needs: autonomy (feeling congruous with behavior), competence (feeling effective), and relatedness (feeling connected). Understanding these three psychological needs allows one to predict the effect of an act on motivation. An unexpected reward has no effect on autonomy, competence, or relatedness so does not increase motivation. Positive verbal feedback does not affect autonomy, but it does improve competence and relatedness; as such, there is a positive effect on motivation. Means to improve motivation is to focus on the three basic needs. For autonomy, information provided must not be controlling; the perspective of the individual must be sought and the information used; and finally choices should be provided that focus on intrinsic drives. For competence, one should provide meaningful feedback, set the learner up for success, and provide support to achieve competence. For relatedness, one should acknowledge feelings, support individual connectedness, and improve community connectedness.

The second speaker was Avery Tung, M.D., Professor of Anesthesia and Critical Care at the University of Chicago. Dr. Tung is the Quality Chief in the Department of Anesthesia and President of the Society of Critical Care in Anesthesia and spoke on meta-cognition. Meta-cognition refers to an awareness or analysis of one’s own learning or thinking process, involving awareness, self-regulation, and memory monitoring. There are two components to meta-cognition: knowledge (knowing what you know) and regulation (planning, monitoring, and evaluating). Understanding these components, it becomes clear that metacognition improves as expertise improves. Failure to check context, hindsight bias, and overconfidence interfere with metacognition. When an individual must make a decision, the decision may be based upon intuitive impression (quick, based upon available data) or based upon one’s analytic impression (slow, considers all aspects and consequences). Some decisions must be quick while others must be well-thought. The problem is that many make all decisions too quick while others don’t make decisions at all because of over-analysis. Means to avoid making intuitive errors include considering alternatives, seeking feedback, and increasing knowledge. However, making all decisions analytic result in fatigue and consumes limited resources. Decisions must include both intuitive and analytic.

The final speaker for the first panel was William McGaghie, M.D., Professor of Medical Education and Preventive Medicine, Northwestern University. Dr. McGaghie presented self-regulation. Key aspects to self-regulation include recognizing that self-assessments of professional skills are unreliable. To truly evaluate professional skills, reliable assessments must be used. One may improve skills with simulation, which involves self-assessment and retraining. An example of this failure for self-assessment was videos of surgeons performing bariatric surgery. The videos were reviewed by several experts who rated the skills. Those individuals who were evaluated as having lower rated skills were also the surgeons most likely to have higher mortality and complications. This study highlights that clinical skills need to be based on objective measures. Understanding self-regulation requires an understanding of the Dunning Kruger Effect. This psychological principle is a cognitive bias where those of lower ability assume a higher self-ability due to a metacognitive effect of inflated self-assessment. This principle highlights the inability of self-assessment when it comes to clinical skills. Any time an individual is being evaluated, an apprehension occurs; objective data provides feedback for improvement and prevents the concept that it is being used for punishment.

Ann Harman, PhD, Chief Assessment Officer, The American Board of Anesthesiology, Raleigh-Durham, North Carolina

The second panel evaluated the science of longitudinal assessment. The first speaker was Ann Harman, Ph.D., Chief Assessment Officer of the American Board of Anesthesiology. Dr. Harman discussed knowledge over the continuum. There are several principles that help improve knowledge. The concept of priming is important in acquisition of knowledge. Priming involves asking questions before providing new information as the question prepares the mind of the learner for the new information. Another important point in mastery of knowledge is having the individual assess their confidence in knowledge. In this situation, the learner is asked to evaluate confidence which maintains a focus and accelerates time to mastery. Confidently held misinformation leads to error and injury. The knowledge must be presented over time. In spaced repetition, incorporating increasingly long delays before repeating a question makes the memory of information more durable by interrupting the forgetting curve. Finally, retrieval practice is important in gaining knowledge. In retrieval practice, recalling information from memory strengthens the memory pathway to that information and makes the memory more durable. Other factors that improve acquisition of knowledge is personalization (providing the learner with information that they need) and motivation (insuring that the information is challenging and relevant). The acquisition of knowledge requires frequent and regular assessment which generates reflection and reevaluation. The second speaker was Dr. Jonathan Wanderer, Associate Professor Vanderbilt University, Medical Director Vanderbilt Anesthesiology and Perioperative Informatics Research Division. Information may be used as a pedagogical tool. To provide an idea of the amount of information available, there are 10,067 tables in EPIC and the Perioperative Data Warehouse has 19,463,550,658 pieces of information. This information may be used by residents to understand their educational needs by summarizing their procedural history, clinical exposure and team management experience. Information may also be used for changing culture by providing data rather than opinion. It allows for the immediate access to information when needed rather than relying on the individual to recall the issue and search for the solution at a later time. Informatics may be used to develop practical management information by providing typical practices of medication and equipment use. The information obtained from informatics maximizes the resident experience by optimizing the situation and providing a tailored experience for the resident.

Dr. Keith Baker, Associate Professor of Anesthesia, Department of Anesthesia, Critical Care and Pain Medicine, Vice-Chair for Education, provided the final presentation for the panel, discussing the effect of time on technical skills. His presentation indicated that after an initial improvement phase, in most cases, technical skills do not automatically improve over time and instead they remain on a plateau.  As such, it is fair to conclude that time and experience do not always equate with expertise.  Another aspect that interferes with ongoing improvement of a technical skill is overconfidence.  Overconfidence can lead to insufficient attention being paid to potential errors, which can thus lead to mistakes.  An effective means to improve technical skills is to prioritize to actual or measured performance.  This stands in contrast to using time spent performing a skill as the metric of competence.  Outcome or mastery-based learning employs deliberate practice, which is designed to improve performance and overcome weaknesses.  Technical skill improvement can also be hastened by using spaced practice in which there is exposure to the skill at intervals as compared to a single long exposure.  Ironically, learners prefer a single long exposure as compared to spaced practice even though spaced practice is distinctly more effective.  Another means to improve technical skills is sleep.  Sleep improves the learning of technical skills more than an equivalent duration of wakefulness.  Although time and effort often do improve performance, they are insufficient to achieve an expert level of performance.  The achievement of expertise requires time, effort and the active reinvestment of resources into improvement by choosing to engage in deliberate practice.  Thus, becoming an expert is an individual choice and not an expected outcome of time and experience.

The EAB is grateful to the speakers for sharing their knowledge and providing the members of the AUA with superb presentations concerning the science behind education.

Author

Robert Gaiser, MD

Professor of Anesthesiology

University of Kentucky

Lexington, Kentucky