Make No Assumptions. Give Lots of Tests.

I greatly appreciate the honor and opportunity to offer this Presidential Address to my colleagues in the Western Surgical Association (WSA). I have so many wonderful people who need to be thanked for helping me get the most out of my life: initially my parents, for teaching me about unconditional love, the value of effort, and the importance of repetition; my brothers, Steve and Mike, for either pulling or pushing me to competitive success–they were great teammates and opponents in a constant effort to seek excellence in athleticism and academics; my wife, Cathy, the love of my life and the woman who loves me for who I am. She is an extraordinary teacher and educator herself, and I have leaned on her heavily in my efforts in surgical education. I am eternally grateful she has put up with this surgeon/workaholic and stuck with me for 32 wonderful and chaotic years. Thank you, Cathy. My wife and I were blessed with 3 amazing children 20+ years ago, and more recently, with a stellar daughter-in-law and an absolutely lovable granddaughter. The love and support of my family has been critical to my professional success. My academic career was propelled forward by hundreds of surgical colleagues. Three big thank yous go to my uncle Harrison, a retired general surgeon, to past WSA president, Dr Jon van Heerden, and to Dr Mike Sarr. All 3 mentors took an amazing and heartfelt interest in me as a person and assisted greatly in helping me find meaning in my craft and my place in academic surgery. Thank you, gentlemen. Other surgical colleagues, well recognized by this audience, were similarly helpful in making me a better surgeon and a better educator. Thank you. I am indebted to my extraordinary secretaries, clinic nurses, program coordinators, and a research team that made me look better than I really was. I was blessed with multiple operating room crews that were extraordinarily patient when my trainee and I struggled to repair the hernia, complete the anastomosis, or simply close the skin. All of these scrub nurses and surgical assistants provided a learning environment that was paramount to my success and that of my trainees. A special thank you to the WSA, the members and guests in attendance tonight, to Dr Sherry Wren for a wonderful introduction, to the officers, committees, and a truly passionate executive council that made this job a pleasurable one. A big shout out to our administrative team, LP etc. They have been efficient, effective, and caring members of our WSA family. And lastly, thank you to the 300+ interns who evaluated more than 25,000 patients and operated by my side for 25 years. Having them come back 4 years later to scrub in with me as chief residents was among the highlights of my career: interns really can improve! But learning surgery continues to get harder: more information to learn, more surgical techniques to master, and far more complex patients to care for. All this under time constraints makes being efficient and effective about learning critical to the future of surgery. Teaching surgery has not gotten easier, with issues of resident duty hours, trainee autonomy, HIPAA regulations, patient complexity, and carving out time to teach. Too many surgical staff, myself included, find it hard to be dedicated teachers each day and challenge our trainees to learn. But challenge them we must! I have titled this Presidential Address, “Make No Assumptions. Give Lots of Tests.” The objectives of this talk are to remember and incorporate 2 tenets that I believe will make for better and more effective surgical education:1.Make no assumptions of your learners.2.Testing trainees enhances memory creation and retrieval, making learning more effective. As a young staff surgeon and general surgery program director at the Mayo Clinic in the mid-1990s, I had an “Aha!” educational moment: I worked with a solid trainee performing a thyroidectomy as a first case on M-W-F of that first week. I assisted the intern in doing much of all 3 cases. On Monday he had no idea how to ligate in continuity and divide the middle thyroid vein. “It is an important technique,” I said. The intern nodded and responded, “I like it.” On Wednesday we again got to the stage of mobilizing the thyroid lobe out of the neck, and I asked my trainee, “How are we going to take care of the middle thyroid vein?” The intern responded, “I am not sure.” I then proceeded to help him ligate in continuity: verbalizing how to grasp the vein, dissect posterior to it, and pass two 3-0 silks. We each tied our own silk ligature and he divided the vein and trimmed the 3-0 silks. “Oh, now I see.” On Friday, same operation, same blank look when we came to the middle thyroid vein. He had no concept of ligation in continuity. After helping him do it for the third time that week, I proceeded to give him some politely worded, but negative, feedback, which included me saying he didn’t remember any of the 5 steps I taught him about ligation in continuity. I closed with, “Do you understand my feedback to you?” He nodded his head and said: Intern: “Roger!” Me: “Roger, what?” Intern: “Roger, I heard your feedback.” Me: “What did you hear as my feedback?” Intern: “Whatever it was that you just told me.” While I wanted to scream, I burst out laughing and realized I had a long way to go in surgical education. Humbling indeed. I was making assumptions that because his hands were working pretty well and he was nodding a lot, that he was learning the technique and incorporating it into his skill set. Actually, NONE of my teaching was turned into learning. I made at least 4 bad assumptions that week:1.Good teaching leads to good learning. (Or poor teaching leads to poor learning.)2.Experts or experienced surgeons make good teachers.3.There ARE dumb questions and pathetic answers.4.Modern learners are less motivated and driven than I was. No. Just because someone is teaching does not mean others are learning. There is absolutely no guarantee that good teaching leads to good learning; same with poor teaching–that may or may not lead to poor learning. Learning something, anything–that must be done by the trainee. Only the trainee can learn, meaning changing what lies within their mind. They must be engaged in an experience, code it into their brain, and then be capable of bringing out that information or skill when they desire. Teaching is useful, but learning is incumbent on the student. Learning is something students must do.1Mayer R. The promise of educational psychology, Vol 2: Teaching for meaningful learning. Upper Saddle River, NJ: Merrill Prentice Hall; 2002.Google Scholar More about this later, but it is clear I did not get my intern to do it. Sometimes true, but evidence exists that experts often teach over the heads of their learners. I did this to my intern. Experts have information and skills filed away in an orderly fashion in their brains; novice learners have no such platform from which to build knowledge and skill. Inexperienced learners in a stressful environment (operating room, ICU, emergency room, etc) may be overwhelmed and learn little to nothing. A great teacher is not afraid to question learners frequently, using learner feedback (either correct or incorrect responses) to tailor ongoing teaching to assist the learner. I failed my intern. Bad assumption. When learners ask an “off the wall” question or respond with an answer that seemingly makes no sense, they are giving the instructor clear feedback that learning is not occurring or registering. Funny questions or poor responses should be immediate triggers for teachers to re-evaluate the learning points and try again to offer something understandable and useful. I missed these obvious cues with my intern. The assumptions that baby boomers, generation X, generation Y, or generation Z are somehow inherently less driven, committed, intelligent, or talented than previous generations are bad assumptions. I was thinking of this when my intern failed at ligation in continuity. Big mistake on my part. In my opinion, in 2020 this assumption of modern learners is wrong, and I believe it is detrimental to our learners’ psyche and detrimental to us as surgeon educators. Millennial learners are bright, talented people who were brought up in a different culture, but when turned on to surgery, they are every bit as resilient, resourceful, and committed to their education as the baby boomer crowd. No less than future US President John Adams made a comment about American youth in the 1760s being less driven and motivated than their parents.2McCullough D. John Adams. Simon & Schuster, 2001: 1-751Google Scholar I believe John Adams was speaking of young men and women who eventually fought in the Revolutionary War. I think we can all agree that our forefathers and foremothers were plenty driven and motivated, and they stepped up when challenged. I think gen Y, gen Z, and ongoing generations will do just fine as well if we respect them, teach them, and really engage them. Let’s agree to avoid making assumptions when it comes to learners and surgical education. Humans are born with more than 2 billion neurons. While we may lose a few million of them over the course of a lifetime, the brain improves function over time by adding new connections with each experience, pruning unwanted connections, and ramping up speed with myelin coating of axons. Indeed, at age 16, we have enough neural connections–upwards of 15 trillion pathways–and enough storage capacity to record literally every second of our lives from birth to death. The storage capacity of our brains for memory is virtually unlimited.3Brown P. Roediger H. McDaniel M. Make It Stick. The Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 100Google Scholar The ability to retrieve information from the brain is not. This difficulty in memory retrieval is the crux of the problem for all learners, and especially surgical learners in 2020. There are thousands of facts, concepts, and skills to encode and store in the brain and then organize, consolidate, and process so that the information is readily available. This metamorphosis takes time and effort. A lot of effort. On the upside, the human brain is a learning machine. Although our brains’ architecture and gross structure are determined genetically, the fine neural networks are shaped by experience and can be substantially modified.4Bruer J.T. Neural connections: Some you use, some you lose.Phi Delta Kappan. 1999; 81: 4Google Scholar Increased experience and numerous repetitions build up more neural connections, stronger and faster pathways, and more easily retrieved information for that subject or skill. The thickness of the myelin-coated pathways correlates with increased practice; such myelin deposition improves the strength and speed of the electrical/chemical signals and, as a result, improves function and performance.5Fields R.D. White matter matters.Scientific American. 2008; 298: 42-49Crossref PubMed Google Scholar Over the past 40 years, significant advancements have occurred in understanding how humans learn and how to improve our abilities to imbed knowledge and skill within our brains so that retrieval is efficient and effective. In 1979, when I was a freshman at the University of Wisconsin, I was a good student. I went to class. I took notes. I read the assigned reading. I highlighted the key facts. I turned my homework in on time. And like many of you, I crammed for each final exam: re-reading my notes and highlights to prepare me for the exam. Cramming for the English literature, organic chemistry, and Russian history exams–what a mistake! While I got As in the classes, within weeks I remembered nothing of the material. Zilch! Educators, psychologists, and researchers have made great strides over the past 4 decades to better understand the actual science of learning. I wish they could have taught me in 1979 about the weakness of “cramming,” or so-called “blocked,” or massed learning. They would have told me it is effective for short-term use, but it is abysmal for long-term retention. In our line of work, teaching and training surgical learners for 30-year careers, there is no place for cramming. What else does the science of learning tell us in 2020? Learning is a process. It involves changing knowledge, skills, beliefs, or attitudes. But as surgical educators, we must understand, that learning is not something done to students. On the contrary, it must be something students do themselves.1Mayer R. The promise of educational psychology, Vol 2: Teaching for meaningful learning. Upper Saddle River, NJ: Merrill Prentice Hall; 2002.Google Scholar,6Ambrose S. Bridges M.W. Lovett M.C. et al.How Learning Works. John Wiley & Sons, Inc, Hoboken, NJ2010: 3Google Scholar,7Zull J. The art of changing the brain: Enriching the practice of teaching by exploring the biology of learning. Stylus, Sterling, VA2002: 112Google Scholar That fact bears repeating: Learning is something that trainees do themselves. Learning always builds on previous knowledge.3Brown P. Roediger H. McDaniel M. Make It Stick. The Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 100Google Scholar,6Ambrose S. Bridges M.W. Lovett M.C. et al.How Learning Works. John Wiley & Sons, Inc, Hoboken, NJ2010: 3Google Scholar Unless there is some change in neural connections, no learning can occur.7Zull J. The art of changing the brain: Enriching the practice of teaching by exploring the biology of learning. Stylus, Sterling, VA2002: 112Google Scholar Teachers can impart knowledge and information, but only learners can create or change their neural pathways. Neurons form new connections with one another with every new experience we have. The more times the pathway is then used or tested, the stronger the connection becomes, and therefore, better retention, retrieval, and performance.8Lang J. Cheating lessons: Learning from academic dishonesty. Harvard University Press, Cambridge, MA2014: 114Google Scholar Periodic retrieval of any learning helps strengthen connections to the memory and the cues to recalling it. In fact, the ease of retrieving a new memory, after a delay of a few days, is a good indicator of learning.9Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 126Google Scholar The amazing thing with our brains is that as new knowledge is added to and connects with previous knowledge, each additional bit of information or skill “sticks” better to the growing neural connections.10Ambrose S. Bridges M.W. Lovett M.C. et al.How Learning Works. John Wiley & Sons, Inc, Hoboken, NJ2010: 15Google Scholar The more we learn, the better we learn. There is virtually no limit to how much learning we can remember, as long as we relate it to what we already know.11Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 76Google Scholar This is true in surgery: offer a new, unusual, or trivial fact to a master surgeon and she will easily incorporate it into her mind and use it efficiently; a surgical intern has no such “safety net” of neural connections and the miniscule fact falls through and fails to “stick,” unless novice learners are offered further teaching, repetitions and retrieval practice. The more we learn, the more possible connections we create for further learning. Importantly, for better memory creation, Roediger and Karpicke12Roediger III, H. Karpicke J.D. The power of testing memory: basic science research and implications for educational practice.Perspect Psycholog Sci. 2006; 1: 181-210Crossref PubMed Scopus (1134) Google Scholar admonish that “to measure any type of learning, you have to administer a test.” Testing is a good thing. Some other study habits are not! Repetition has been called the Mother of Learning. Indeed, for centuries, teachers, coaches, and surgeons have encouraged their students to practice their multiplication tables, practice their free throws, and tie thousands of square knots, respectively. Seeing, hearing, tasting, smelling, and feeling the subject or skill multiple times is beneficial…to a point. What students are often advised to do is often plain wrong, and mere repetition by itself does not necessarily enhance learning. By learning, I mean something that is usable for the long haul; something the trainee can rely on for a career.13Roediger III, H. Karpicke J.D. Test-enhanced learning: Taking memory tests improves long-term retention.Psycholog Sci. 2006; 17: 249-255Crossref PubMed Scopus (1503) Google Scholar As I did 40 years ago, at least 80% of college students (and our surgical trainees) today re-read material to learn. There are 3 reasons to avoid this learning strategy: it is time consuming, it does NOT result in durable memory, and it unwittingly creates self-deception.14Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 12Google Scholar Re-reading text, scanning notes, or reading highlighted passages takes little effort. The brain can race through the material and the learner is deceived into believing he or she knows the subject matter. The brain is making no effort to learn the material. Despite hours of scanning or highlighting, facts are NOT placed in long-term memory. Karpicke and colleagues15Karpicke J.D. Butler A.C. Roediger H.L. Metacognitive strategies in student learn: Do students practice retrieval when they study on their own?.Memory. 2010; 17: 471-479Crossref Scopus (352) Google Scholar note that “the amount of time of study is no measure of mastery.” Repetition by itself does not lead to long-term memory, and not all practice is alike.16Tulving E. Subjective organization and the effects of repetition in multi-trial free recall learning.J Verbal Learn Verbal Behav. 1966; 5: 193-197Crossref Scopus (146) Google Scholar Similarly, any learning that is achieved during a solitary, massed practice session is transitory and invariably is lost over the next several days.17Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 10Google Scholar There are, however, an assortment of learning strategies that do lead to long-term retention of knowledge and skill acquisition. Psychologists Robert and Elizabeth Bjork18Bjork R.A. Bjork E.L. A new theory of disuse and an old theory of stimulus fluctuation.in: Healy A. Kossly S. Shiffrin R. From Learning Processes to Cognitive Processes: Essays in Honor of William K Estes, Vol w. Erlbaum, Hillsdale, NJ1992: 35-67Google Scholar coined the term, “Desirable Difficulties” for enhancing learning. They suggest that the harder one tries to remember something, the more likely it will be retained and retrievable for use. The more effort you make to learn something, the better you will retain it and retrieve it. Simply reading or reviewing notes or reading highlighted material is effortless for the brain, and therefore, does not lead to retention or become retrievable. Extensive research shows that when learning is harder or more effortful, it’s stronger and lasts longer.18Bjork R.A. Bjork E.L. A new theory of disuse and an old theory of stimulus fluctuation.in: Healy A. Kossly S. Shiffrin R. From Learning Processes to Cognitive Processes: Essays in Honor of William K Estes, Vol w. Erlbaum, Hillsdale, NJ1992: 35-67Google Scholar For instance, a study of students learning about the circulatory system found that the group that read a passage twice (minimal brain effort) on the subject fared worse on testing than a group that read the passage once and then explained to themselves what each sentence meant (self-explanation = more effort). The self-explain group scored 32% better.19Chi M.T.H. DeLeeuw N. Chiu M.H. LaVancher C. Eliciting self-explanation improves understanding.Cognit Sci. 1994; 18: 439-477Google Scholar Although both groups studied for the same amount of time, effortful retrieval made for stronger learning in the self-explain group. To be desirable, a difficulty must be something learners can overcome through increased effort.18Bjork R.A. Bjork E.L. A new theory of disuse and an old theory of stimulus fluctuation.in: Healy A. Kossly S. Shiffrin R. From Learning Processes to Cognitive Processes: Essays in Honor of William K Estes, Vol w. Erlbaum, Hillsdale, NJ1992: 35-67Google Scholar For instance, slightly blurring some words in a text to read would be a desirable difficulty, readers would have to slow down, squint, and truly concentrate to recognize and digest the words and their meaning; blurring them beyond recognition would make this an undesirable difficulty. There are numerous ways to allow or make learners bring forth more effort to learning. As surgeons, I bet you can think of many. Let me expand on a couple that will resonate with creating better young surgeons to follow in your footsteps. Benedict Carey20Carey B. How We Learn. Random House, New York2014: 66Google Scholar offers that nothing in learning science comes close in terms of immediate, significant, and reliable improvements to learning as distributive learning. Spreading out learning sessions over 2 or 3 or more separate opportunities takes extra time and effort, but this “spaced learning” facilitates retention and long-term retrieval. As an example, Moulton and colleagues21Moulton C.A.E. Duborwski A. McRae H. et al.Teaching surgical skills: What kind of practice makes perfect?.Ann Surg. 2006; 244: 400-409Crossref PubMed Scopus (28) Google Scholar split a group of 38 surgical residents into 2 groups: half were taught microsurgery (vascular anastomosis on rat aortas) over 4 sequential weekly sessions, while the other group learned in a 1 day-long session. Total training time was equal for both groups. Waiting 1 month after each group’s last teaching session, all 38 trainees were tested on their microsurgery vascular anastomosis skills. When it counted for the test, the distributive learning group was faster, had fewer wasted motions, and had greater success with the anastomosis than the massed learning group.21Moulton C.A.E. Duborwski A. McRae H. et al.Teaching surgical skills: What kind of practice makes perfect?.Ann Surg. 2006; 244: 400-409Crossref PubMed Scopus (28) Google Scholar Whenever possible, learning should be spaced out over time; multiple shorter learning sessions will lead to better retention than 1 longer learning experience; allowing students to step back after a first session and actually forget some material is useful. With second and third sessions, the effort it takes to retrieve the lessons of the first session creates stronger memories and eventually allows easier retrieval. Distributive learning feels harder and requires concentration over multiple sessions, but the final outcome is better, and more durable, long-term memory. Novice learners need objective feedback to speed their transition toward competency. Both positive and negative feedback are useful, but surgeons need to understand there are several ways to offer this information to trainees, and some work better than others. For starters, delaying the feedback briefly produces better long-term learning than immediate feedback. Brown and colleagues22Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 40Google Scholar note that in motor learning, trial and error with delayed feedback is a more awkward, but more effective, way of acquiring skill than trial and correction through immediate feedback. “Immediate feedback is like the training wheels on a bicycle: the learner quickly comes to depend on the continued presence of the correction.”22Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 40Google Scholar While hovering over a young learner (say, an intern working on ligating in continuity the middle thyroid vein) in the operating room leads to better patient safety, surgeon educators will serve their surgical trainees best in the cadaver or simulation labs by allowing them to make mistakes and delay feedback. Allowing learners to ponder what is working, what is failing, and how they might change their technique is called “reflection”; such thinking helps create lasting memories. Following that, learner effort with feedback from a savvy surgeon with to and fro discussion leads to memories that are stronger and easier to retrieve in the future. When learners commit errors and are given corrective feedback, the errors are not learned.23Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 91Google Scholar Allowing students the opportunity to practice and take risks, fail and get feedback, and repeat the effort, without their grades or evaluation or confidence suffering, is a powerful learning opportunity.24Lang J. Small Teaching. Jossey-Bass, San Francisco2016: 206Google Scholar Indeed, Thomas Edison joked, “I’ve not failed. I’ve just found 10,000 ways that don’t work.” Ericsson was similarly high on learners failing, because “failure becomes an essential experience on the path to mastery.”25Ericsson K.A. The influence of experience and deliberate practice on the development of superior expert performance.in: Charness N. Feltovich P.J. Hoffman R. The Cambridge Handbook of Expertise and Expert Performance. Ericcson KA. Cambridge University Press, Cambridge2006: 683-704Crossref Google Scholar In our operating rooms, ICUs, and emergency rooms, we need our trainees to be at their best to offer exceptional patient outcomes, but anywhere in controlled teaching/learning sessions, encouraging trial and error effort, accepting mistakes, and learning to correct them creates deep memories. If the student’s performance did not meet expectations, try again. What matters is that one learns from the effort.24Lang J. Small Teaching. Jossey-Bass, San Francisco2016: 206Google Scholar In our own wonderful, but imperfect surgical training program, we use bi-annual Objective Structured Clinical Examination (OSCE)-type assessments of our surgical residents. The fall exam is identical to the spring exam with about 12 knowledge (anatomy, reading CT scans, etc) and skill (cricothyrotomy, chest tube insertion, GI anastomosis, etc) stations, appropriate for each postgraduate year level. Our expectation is that learners will meet performance minimums in the spring. For those not meeting the standard, they are tutored and offered feedback within practice sessions, with the expectation of re-taking that station’s exam as many times as they like until they achieve superior marks (well above the minimum standard).26Gas B.L. Buckarma E.H. Mohan M. et al.Objective assessment of general surgery residents followed by remediation.J Surg Educ. 2016; 73: e71-e76Crossref PubMed Scopus (24) Google Scholar Every single general surgery resident in the last decade eventually met or exceeded all standards set before them. Springtime results were consistently 20% to 50% better than fall exam scores. When general surgery residents offer effort, they clearly do learn. Repetitions, reflection, with forgetting, and subsequent retrieval practice help the brain to better organize and consolidate the information. Experts package such material in “chunks” and eventually gain easy ability to offer knowledge and skills when a topic is offered. Say “Zollinger-Ellison Syndrome” to a master endocrine surgeon and she suddenly can spit out information as though she has a surgical atlas, a surgical history book, and dozens of personal experiences with gastrinomas playing on a teleprompter in front of her. She kind of does. She has a vast mental representation of what she knows on the subject. With her well-organized mind and extensive neural pathways, she really does see, feel, and hear virtually everything she knows about gastrinomas. Hours of deliberate practice combined with memory consolidation, organization, and reorganizing her neural pathways lead to efficiency and intelligence and proficiency. While most of us really do not enjoy being tested–it often is hard work–the learning advantages are impressive: tests strengthen memory, tests make subsequent retrieval easier, tests clarify what you know, tests tell you what you don’t know, and tests are objective. Even better, tests need not be high stakes exams, they need not be lengthy, and they can be administered by anyone and are often best if given by learners themselves. Let’s take a closer look at tests and testing and why surgeons should be testing others and themselves. The Science of Successful Learning27Brown P. Roediger H. McDaniel M. Make It StickThe Science of Successful Learning. The Belknap Press of Harvard University Press, Cambridge, MA2014: 20Google Scholar suggests “testing” can be as simple as reading material and then closing the book or covering the page and trying to explain or recite what was just read. Rather than simply moving on or even re-reading the material, making the extra effort to retrieve the knowledge is a “test,” or a retrieval challenge, and represents more effort on the part of the student. In effect, retrieval–or testing–interrupts forgetting; the sheer act of attempting to recall the facts, skill, thoughts, or knowledge strengthens and modifies the memory and creates