make stick

No matter what you may set your sights on doing or becoming, if you want to be a contender, it’s mastering the ability to learn that will get you in the game and keep you there. In the preceding chapters, we resisted the temptation to become overtly prescriptive, feeling that if we laid out the big ideas from the empirical research and illustrated them well through examples, you could reach your own conclusions about how best to apply them. But early readers of those chapters urged us to get specifi c with practical advice. So we do that here. We start with tips for students, thinking in par tic u lar of high school, college, and graduate school students. Then we speak to lifelong learners, to teachers, and fi nally to trainers. While the fundamental principles are consistent across these groups, the settings, life stages, and learning materials differ. 8 Make It Stick Make It Stick ê 201 To help you envision how to apply these tips, we tell the stories of several people who, one way or another, have already found their way to these strategies and are using them to great effect. Learning Tips for Students Remember that the most successful students are those who take charge of their own learning and follow a simple but disciplined strategy. You may not have been taught how to do this, but you can do it, and you will likely surprise yourself with the results. Embrace the fact that signifi cant learning is often, or even usually, somewhat diffi cult. You will experience setbacks. These are signs of effort, not of failure. Setbacks come with striving, and striving builds expertise. Effortful learning changes your brain, making new connections, building mental models, increasing your capability. The implication of this is powerful: Your intellectual abilities lie to a large degree within your own control. Knowing that this is so makes the diffi culties worth tackling. Following are three keystone study strategies. Make a habit of them and structure your time so as to pursue them with regularity. Practice Retrieving New Learning from Memory What does this mean? “Retrieval practice” means self- quizzing. Retrieving knowledge and skill from memory should become your primary study strategy in place of rereading. How to use retrieval practice as a study strategy: When you read a text or study lecture notes, pause periodically to ask yourself questions like these, without looking in the text: What are the key ideas? What terms or ideas are new to me? How Make It Stick ê 202 would I defi ne them? How do the ideas relate to what I already know? Many textbooks have study questions at the ends of the chapters, and these are good fodder for self- quizzing. Generating questions for yourself and writing down the answers is also a good way to study. Set aside a little time every week throughout the semester to quiz yourself on the material in a course, both the current week’s work and material covered in prior weeks. When you quiz yourself, check your answers to make sure that your judgments of what you know and don’t know are accurate. Use quizzing to identify areas of weak mastery, and focus your studying to make them strong. The harder it is for you to recall new learning from memory, the greater the benefi t of doing so. Making errors will not set you back, so long as you check your answers and correct your mistakes. What your intuition tells you to do: Most studiers focus on underlining and highlighting text and lecture notes and slides. They dedicate their time to rereading these, becoming fl uent in the text and terminology, because this feels like learning. Why retrieval practice is better: After one or two reviews of a text, self- quizzing is far more potent for learning than additional rereading. Why might this be so? This is explained more fully in Chapter 2, but here are some of the high points. The familiarity with a text that is gained from rereading creates illusions of knowing, but these are not reliable indicators of mastery of the material. Fluency with a text has two strikes against it: it is a misleading indicator of what you have learned, and it creates the false impression that you will remember the material. By contrast, quizzing yourself on the main ideas and the meanings behind the terms helps you to focus on the central Make It Stick ê 203 precepts rather than on peripheral material or on a professor’s turn of phrase. Quizzing provides a reliable mea sure of what you’ve learned and what you haven’t yet mastered. Moreover, quizzing arrests forgetting. Forgetting is human nature, but practice at recalling new learning secures it in memory and helps you recall it in the future. Periodically practicing new knowledge and skills through self- quizzing strengthens your learning of it and your ability to connect it to prior knowledge. A habit of regular retrieval practice throughout the duration of a course puts an end to cramming and all- nighters. You will need little studying at exam time. Reviewing the material the night before is much easier than learning it. How it feels: Compared to rereading, self- quizzing can feel awkward and frustrating, especially when the new learning is hard to recall. It does not feel as productive as rereading your class notes and highlighted passages of text feels. But what you don’t sense when you’re struggling to retrieve new learning is the fact that every time you work hard to recall a memory, you actually strengthen it. If you restudy something after failing to recall it, you actually learn it better than if you had not tried to recall it. The effort of retrieving knowledge or skills strengthens its staying power and your ability to recall it in the future. Space Out Your Retrieval Practice What does this mean? Spaced practice means studying information more than once but leaving considerable time between practice sessions. How to use spaced practice as a study strategy: Establish a schedule of self- quizzing that allows time to elapse between study sessions. How much time? It depends on the material. If you are learning a set of names and faces, you will need to Make It Stick ê 204 review them within a few minutes of your fi rst encounter, because these associations are forgotten quickly. New material in a text may need to be revisited within a day or so of your fi rst encounter with it. Then, perhaps not again for several days or a week. When you are feeling more sure of your mastery of certain material, quiz yourself on it once a month. Over the course of a semester, as you quiz yourself on new material, also reach back to retrieve prior material and ask yourself how that knowledge relates to what you have subsequently learned. If you use fl ashcards, don’t stop quizzing yourself on the cards that you answer correctly a couple of times. Continue to shuffl e them into the deck until they’re well mastered. Only then set them aside— but in a pile that you revisit periodically, perhaps monthly. Anything you want to remember must be periodically recalled from memory. Another way of spacing retrieval practice is to interleave the study of two or more topics, so that alternating between them requires that you continually refresh your mind on each topic as you return to it. What your intuition tells you to do: Intuition persuades us to dedicate stretches of time to single- minded, repetitive practice of something we want to master, the massed “practicepractice- practice” regime we have been led to believe is essential for building mastery of a skill or learning new knowledge. These intuitions are compelling and hard to distrust for two reasons. First, as we practice a thing over and over we often see our per for mance improving, which serves as a powerful reinforcement of this strategy. Second, we fail to see that the gains made during single- minded repetitive practice come from short- term memory and quickly fade. Our failure to perceive how quickly the gains fade leaves us with the impression that massed practice is productive. Make It Stick ê 205 Moreover, most students, given their misplaced faith in massed practice, put off review until exam time nears, and then they bury themselves in the material, going over and over it, trying to burn it into memory. Why spaced practice is better: It’s a common but mistaken belief that you can burn something into memory through sheer repetition. Lots of practice works, but only if it’s spaced. If you use self- quizzing as your primary study strategy and space out your study sessions so that a little forgetting has happened since your last practice, you will have to work harder to reconstruct what you already studied. In effect, you’re “reloading” it from long- term memory. This effort to reconstruct the learning makes the important ideas more salient and memorable and connects them more securely to other knowledge and to more recent learning. It’s a powerful learning strategy. (How and why it works are discussed more thoroughly in Chapter 4.) How it feels: Massed practice feels more productive than spaced practice, but it is not. Spaced practice feels more diffi - cult, because you have gotten a little rusty and the material is harder to recall. It feels like you’re not really getting on top of it, whereas in fact, quite the opposite is happening: As you reconstruct learning from long- term memory, as awkward as it feels, you are strengthening your mastery as well as the memory. Interleave the Study of Different Problem Types What does this mean? If you’re trying to learn mathematical formulas, study more than one type at a time, so that you are alternating between different problems that call for different solutions. If you are studying biology specimens, Dutch painters, or the principles of macroeconomics, mix up the examples. Make It Stick ê 206 How to use interleaved practice as a study strategy: Many textbooks are structured in study blocks: They present the solution to a par tic u lar kind of problem, say, computing the volume of a spheroid, and supply many examples to solve before moving to another kind of problem (computing the volume of a cone). Blocked practice is not as effective as interleaved practice, so here’s what to do. When you structure your study regimen, once you reach the point where you understand a new problem type and its solution but your grasp of it is still rudimentary, scatter this problem type throughout your practice sequence so that you are alternately quizzing yourself on various problem types and retrieving the appropriate solutions for each. If you fi nd yourself falling into single- minded, repetitive practice of a par tic u lar topic or skill, change it up: mix in the practice of other subjects, other skills, constantly challenging your ability to recognize the problem type and select the right solution. Harking back to an example from sports (Chapter 4), a baseball player who practices batting by swinging at fi fteen fastballs, then at fi fteen curveballs, and then at fi fteen changeups will perform better in practice than the player who mixes it up. But the player who asks for random pitches during practice builds his ability to decipher and respond to each pitch as it comes his way, and he becomes the better hitter. What your intuition tells you to do: Most learners focus on many examples of one problem or specimen type at a time, wanting to master the type and “get it down cold” before moving on to study another type. Why interleaved practice is better: Mixing up problem types and specimens improves your ability to discriminate between types, identify the unifying characteristics within a type, and improves your success in a later test or in real- world settings Make It Stick ê 207 where you must discern the kind of problem you’re trying to solve in order to apply the correct solution. (This is explained more fully in Chapter 3.) How it feels: Blocked practice— that is, mastering all of one type of problem before progressing to practice another type— feels (and looks) like you’re getting better mastery as you go, whereas interrupting the study of one type to practice a different type feels disruptive and counterproductive. Even when learners achieve superior mastery from interleaved practice, they persist in feeling that blocked practice serves them better. You may also experience this feeling, but you now have the advantage of knowing that studies show that this feeling is illusory. Other Effective Study Strategies ELABORATION improves your mastery of new material and multiplies the mental cues available to you for later recall and application of it (Chapter 4). What is it? Elaboration is the pro cess of fi nding additional layers of meaning in new material. For instance: Examples include relating the material to what you already know, explaining it to somebody else in your own words, or explaining how it relates to your life outside of class. A powerful form of elaboration is to discover a meta phor or visual image for the new material. For example, to better grasp the principles of angular momentum in physics, visualize how a fi gure skater’s rotation speeds up as her arms are drawn into her body. When you study the principles of heat transfer, you may understand conduction better if you imagine warming your hands around a hot cup of cocoa. For radiation, visualize how the sun pools in the den on a wintry Make It Stick ê 208 day. For convection, think of the life- saving blast of A/C as your uncle squires you slowly through his favorite back- alley haunts of Atlanta. When you learned about the structure of an atom, your physics teacher may have used the analogy of the solar system with the sun as the nucleus and electrons spinning around like planets. The more that you can elaborate on how new learning relates to what you already know, the stronger your grasp of the new learning will be, and the more connections you create to remember it later. Later in this chapter, we tell how the biology professor Mary Pat Wenderoth encourages elaboration among her students by assigning them the task of creating large “summary sheets.” Students are asked to illustrate on a single sheet the various biological systems studied during the week and to show graphically and through key words how the systems interrelate with each other. This is a form of elaboration that adds layers of meaning and promotes the learning of concepts, structures, and interrelationships. Students who lack the good fortune to be in Wenderoth’s class could adopt such a strategy for themselves. GENERATION has the effect of making the mind more receptive to new learning. What is it? Generation is an attempt to answer a question or solve a problem before being shown the answer or the solution. For instance: On a small level, the act of fi lling in a missing word in a text (that is, generating the word yourself rather than having it supplied by the writer) results in better learning and memory of the text than simply reading a complete text. Many people perceive their learning is most effective when it is experiential— that is, learning by doing rather than by reading a text or hearing a lecture. Experiential learning is a Make It Stick ê 209 form of generation: you set out to accomplish a task, you encounter a problem, and you consult your creativity and storehouse of knowledge to try to solve it. If necessary you seek answers from experts, texts, or the Web. By wading into the unknown fi rst and puzzling through it, you are far more likely to learn and remember the solution than if somebody fi rst sat you down to teach it to you. Bonnie Blodgett, an award- winning gardener and writer, provides a strong example of generative learning in Chapter 4. You can practice generation when reading new class material by trying to explain beforehand the key ideas you expect to fi nd in the material and how you expect they will relate to your prior knowledge. Then read the material to see if you were correct. As a result of having made the initial effort, you will be more astute at gleaning the substance and relevance of the reading material, even if it differs from your expectation. If you’re in a science or math course learning different types of solutions for different types of problems, try to solve the problems before you get to class. The Physics Department at Washington University in St. Louis now requires students to work problems before class. Some students take umbrage, arguing that it’s the professor’s job to teach the solution, but the professors understand that when students wrestle with content beforehand, classroom learning is stronger. REFLECTION is a combination of retrieval practice and elaboration that adds layers to learning and strengthens skills. What is it? Refl ection is the act of taking a few minutes to review what has been learned in a recent class or experience and asking yourself questions. What went well? What could have gone better? What other knowledge or experiences does it remind you of? What might you need to learn for better Make It Stick ê 210 mastery, or what strategies might you use the next time to get better results? For instance: The biology professor Mary Pat Wenderoth assigns weekly low- stakes “learning paragraphs” in which students are asked to refl ect on what they learned the previous week and to characterize how their class learning connects to life outside the class. This is a fi ne model for students to adopt for themselves and a more fruitful learning strategy than spending hours transcribing lecture slides or class notes verbatim into a notebook. CALIBRATION is the act of aligning your judgments of what you know and don’t know with objective feedback so as to avoid being carried off by the illusions of mastery that catch many learners by surprise at test time. What is it? Everyone is subject to a host of cognitive illusions, some of which are described in Chapter 5. Mistaking fl uency with a text for mastery of the underlying content is just one example. Calibration is simply the act of using an objective instrument to clear away illusions and adjust your judgment to better refl ect reality. The aim is to be sure that your sense of what you know and can do is accurate. For instance: Airline pi lots use fl ight instruments to know when their perceptual systems are misleading them about critical factors like whether the airplane is fl ying level. Students use quizzes and practice tests to see whether they know as much as they think they do. It’s worth being explicit here about the importance of answering the questions in the quizzes that you give yourself. Too often we will look at a question on a practice test and say to ourselves: Yup, I know that, and then move down the page without making the effort to write in the answer. If you don’t supply the answer, you may be giving in to the illusion of knowing, when in fact you would have diffi - culty rendering an accurate or complete response. Treat prac- Make It Stick ê 211 tice tests as tests, check your answers, and focus your studying effort on the areas where you are not up to snuff. MNEMONIC DEVICES help you to retrieve what you have learned and to hold arbitrary information in memory (Chapter 7). What are they? “Mnemonic” is from the Greek word for memory, and mnemonic devices are like mental fi le cabinets. They give you handy ways to store information and fi nd it again when you need it. For instance: Here is a very simple mnemonic device that some schoolchildren are taught for remembering the US Great Lakes in geographic order, from east to west: Old Elephants Have Musty Skin. Mark Twain used mnemonics to teach his children the succession of kings and queens of En gland, staking the sequence and length of their reigns along the winding driveway of his estate, walking it with the children, and elaborating with images and storytelling. Psychology students at Bellerbys College in Oxford use mnemonic devices called memory palaces to or ga nize what they have learned and must be prepared to expound upon in their A-level essay exams. Mnemonics are not tools for learning per se but for creating mental structures that make it easier to retrieve what you have learned. Brief stories follow of two students who have used these strategies to rise to the top of their classes. Michael Young, Medical Student Michael Young is a high- achieving fourth- year medical student at Georgia Regents University who pulled himself up from rock bottom by changing the way he studies. Make It Stick ê 212 Young entered medical school without the usual foundation of premed coursework. His classmates all had backgrounds in biochemistry, pharmacology, and the like. Medical school is plenty tough under any circumstances, but in Young’s case even more so for lack of a footing. The scope of the challenge that lay before him became abruptly evident. Despite his spending every available minute studying his coursework, he barely eked out a 65 on his fi rst exam. “Quite honestly, I got my butt kicked,” he says. “I was blown away by that. I couldn’t believe how hard it was. It was nothing like any kind of schooling I had done before. I mean, you come to class, and in a typical day you get about four hundred PowerPoint slides, and this is dense information.”1 Since spending more time studying wasn’t an option, Young had to fi nd a way to make studying more effective. He started reading empirical studies on learning and became deeply interested in the testing effect. That’s how we fi rst learned of him: He emailed us with questions about the application of spaced retrieval practice in a medical school setting. Looking back on that stressful period, Young says, “I didn’t just want to fi nd somebody’s opinion about how to study. Everybody has an opinion. I wanted real data, real research on the issue.” You might wonder how he got himself into medical school without premed coursework. He had earned a master’s degree in psychology and worked in clinical settings, eventually as a drug addiction counselor. He teamed up with a lot of doctors, and he slowly began to wonder if he would be happier in medicine. Had he missed his calling? “I didn’t think of myself as being especially intelligent, but I wanted to do more with my life and the idea wouldn’t leave me.” One day he went to the biology department of his local university, Columbus State in Columbus, Georgia, and asked what courses he would need to become a doctor. They laughed. “They said, ‘Well, nobody Make It Stick ê 213 from this school becomes a doctor. People at the University of Georgia and Georgia Tech go to medical school, we haven’t had anybody go to medical school in a de cade.’ ” Not to be put off, Young cobbled together some courses. For example, for the biology requirement, the only thing he could take at Columbus State was a fi shing class. That was his biology course. Within a year he had gotten what ever medical background was available from the school, so he crammed for a month for the Medical College Admission Test and managed to score just well enough. He enrolled at Georgia Regents. At which point he found himself very far indeed from being over the hump. As his fi rst exam made all too clear, the road ahead went straight up. If he had any hope of climbing it, something about his study habits had to change. So what did change? He explains it this way: I was big into reading, but that’s all I knew how to do for studying. I would just read the material and I wouldn’t know what else to do with it. So if I read it and it didn’t stick in my memory, then I didn’t know what to do about that. What I learned from reading the research [on learning] is that you have to do something beyond just passively taking in the information. Of course the big thing is to fi gure out a way to retrieve the information from memory, because that’s what you’re going to be asked to do on the test. If you can’t do it while you’re studying, then you’re not going to be able to do it on the test. He became more mindful of that when he studied. “I would stop. ‘Okay, what did I just read? What is this about?’ I’d have to think about it. ‘Well, I believe it happens this way: The enzyme does this, and then it does that.’ And then I’d have to go back and check if I was way off base or on the right track.” The pro cess was not a natural fi t. “It makes you uncomfortable at fi rst. If you stop and rehearse what you’re reading Make It Stick ê 214 and quiz yourself on it, it just takes a lot longer. If you have a test coming up in a week and so much to cover, slowing down makes you pretty ner vous.” But the only way he knew of to cover more material, his established habit of dedicating long hours to rereading, wasn’t getting the results he needed. As hard as it was, he made himself stick to retrieval practice long enough at least to see if it worked. “You just have to trust the pro cess, and that was really the biggest hurdle for me, was to get myself to trust it. And it ended up working out really well for me.” Really well. By the time he started his second year, Young had pulled his grades up from the bottom of his class of two hundred students to join the high performers, and he has remained there ever since. Young spoke with us about how he adapted the principles of spaced retrieval practice and elaboration to medical school, where the challenges arise both from the sheer volume of material to be memorized and from the need to learn how complex systems work and how they interrelate with other systems. His comments are illuminating. On deciding what’s important: “If it’s lecture material and you have four hundred PowerPoint slides, you don’t have time to rehearse every little detail. So you have to say, ‘Well this is important, and this isn’t.’ Medical school is all about fi guring out how to spend your time.” On making yourself answer the question: “When you go back and review, instead of just rereading you need to see if you can recall the learning. Do I remember what this stuff was about? You always test yourself fi rst. And if you don’t remember, then that’s when you go back and look at it and try again.” Make It Stick ê 215 On fi nding the right spacing: “I was aware of the spacing effect, and I knew that the longer you wait to practice retrieval the better it is for memory, but there’s also a trade- off with how successful you are when you try to recall it. When you have these long enzyme names, for example, and this stepby- step pro cess of what the enzyme is doing, maybe if you learn ten steps of what the enzyme is doing, you need to stop and think, can I remember what those ten steps are? Once I found a good strategy for how much to space practice and I started seeing consistent results, it was easy to follow from there because then I could just trust the pro cess and be confi - dent that it was going to work.” On slowing down to fi nd the meaning: Young has also slowed down the speed at which he reads material, thinking about meaning and using elaboration to better understand it and lodge it in memory. “When I read that dopamine is released from the ventral tegmental area, it didn’t mean a lot to me.” The idea is not to let words just “slide through your brain.” To get meaning from the dopamine statement, he dug deeper, identifi ed the structure within the brain and examined images of it, capturing the idea in his mind’s eye. “Just having that kind of visualization of what it looks like and where it is [in the anatomy] really helps me to remember it.” He says there’s not enough time to learn everything about everything, but pausing to make it meaningful helps it stick. Young’s impressive per for mance has not been lost on his professors or his peers. He has been invited to tutor struggling students, an honor few are given. He has been teaching them these techniques, and they are pulling up their grades. “What gets me is how interested people are in this. Like, in medical school, I’ve talked to all of my friends about it, and now they’re really into it. People want to know how to learn.” Make It Stick ê 216 Timothy Fellows, Intro Psych Student Stephen Madigan, a professor at the University of Southern California, was astonished by the per for mance of a student in his Psych 100 course. “It’s a tough course,” Madigan says. “I use the most diffi cult, advanced textbook, and there’s just a nonstop barrage of material. Three- quarters of the way through the class, I noticed this student named Timothy Fellows was getting 90 to 95 percent of the points on all the class activities— exams, papers, short- answer questions, multiple- choice questions. Those were just extraordinary grades. Students this good— well he’s defi nitely an outlier. And so I just took him aside one day and said, ‘Could you tell me about your study habits?’ ”2 The year was 2005. Madigan did not know Fellows outside class but saw him around campus and at football games enough to observe that he had a life beyond his academics. “Psychology wasn’t his major, but it was a subject he cared about, and he just brought all his skills to bear.” Madigan still has the list of study habit Fellows outlined, and he shares it with incoming students to this day. Among the highlights were these: • Always does the reading prior to a lecture • Anticipates test questions and their answers as he reads • Answers rhetorical questions in his head during lectures to test his retention of the reading • Reviews study guides, fi nds terms he can’t recall or doesn’t know, and relearns those terms • Copies bolded terms and their defi nitions into a reading notebook, making sure that he understands them • Takes the practice test that is provided online by his professor; from this he discovers which concepts he doesn’t know and makes a point to learn them Make It Stick ê 217 • Reorganizes the course information into a study guide of his design • Writes out concepts that are detailed or important, posts them above his bed, and tests himself on them from time to time • Spaces out his review and practice over the duration of the course Fellows’s study habits are a good example of doing what works and keeping at it, so that practice is spaced and the learning is solidly embedded come exam time. Tips for Lifelong Learners The learning strategies we have just outlined for students are effective for anyone at any age. But they are centered around classroom instruction. Lifelong learners are using the same principles in a variety of less- structured settings. In a sense, of course, we’re all lifelong learners. From the moment we’re born we start learning about the world around us through experimentation, trial and error, and random encounters with challenges that require us to recall what we did the last time we found ourselves in a similar circumstance. In other words, the techniques of generation, spaced practice and the like that we present in this book are organic (even if counterintuitive), and it’s not surprising that many people have already discovered their power in the pursuit of interests and careers that require continuous learning. Retrieval Practice Nathaniel Fuller is a professional actor with the Guthrie Theater in Minneapolis. We took an interest in him after a dinner party where the Guthrie’s renowned artistic director, Joe Dowling, on hearing of our work, immediately suggested Make It Stick ê 218 we interview Fuller. It seems that Fuller has the capacity to so fully learn the lines and movements of a role for which he is understudy that he can go onstage at the last moment with great success, despite not having had the benefi t of learning and rehearsing it in the normal way. Fuller is a consummate professional of the stage, having refi ned his techniques for learning roles over many years. He is often cast in a leading role; at other times, he may play several lesser characters in a play while also understudying the lead. How does he do it? When he starts with a new script, Fuller puts it into a binder, goes through it, and highlights all of his lines. “I fi gure out how much I’ve got to learn. I try to estimate how much I can learn in a day, and then I try to start early enough to get that learned.”3 Highlighting his lines also makes them easy to fi nd and gives him a sense of the construction, so this use of highlighting is rather different from what students do in class when they highlight merely for purposes of rereading. “You get the shape of the line, and how the back- andforth works.” Fuller uses retrieval practice in various forms. First, he takes a blank sheet of paper and covers a page of the script. He draws it down, silently rendering the lines of the characters he’s playing opposite, because those lines cue his own, and the emotion in them is refl ected one way or another by his own character. He keeps his own line covered and attempts to speak it aloud from memory. He checks his accuracy. If he gets the line wrong, he covers it up and speaks it again. When he has spoken it correctly, he reveals the next passage and goes on. “Half of knowing your part is not just what to say, but knowing when to say it. I don’t have an exceptional brain for memorizing, but one of the keys I’ve found is, I need to try my Make It Stick ê 219 best to say the line without looking at it. I need to have that struggle in order to make myself remember it. “I’ll work like crazy. When I get to where it feels like diminishing returns, I’ll quit. Then I’ll come back the next day, and I won’t remember it. That’s where a lot of my friends will panic. I just have faith now that it’s in there, it’s going to come back a little bit better the next time. Then I’ll work on a new chunk, until I get to the end of the play.” As he progresses through the script, he’s constantly moving from familiar pages and scenes into newer material, the play taking shape like threads added to a growing tapestry, each scene given meaning by those that came before and extending the story in turn. When he reaches the end, he practices in reverse order, moving from the less familiar last scene to practice the more familiar one that precedes it and then continuing on through the last scene again. Then he goes to the part preceding both of those scenes and practices through to the end. His practice continues reaching back in this way until he has come to the beginning of the play. This working backward and forward helps him stitch less familiar material to more familiar, deepening his mastery of the role as a whole. Learning lines is visual ( just as they are laid out in the script), but, he says, it’s also “an act of the body, an act of the muscles, so I’m trying to say the lines in character, get how it feels.” Fuller examines the language of the script, the textures of the words, and the fi gures of speech for how they reveal meaning. He works to discover the way the character carries himself, the way he moves across the stage, his facial expressions— all facets that reveal the underlying emotions that drive each scene. These forms of elaboration help him develop an emotional approach to the role and a deeper connection to the character. Make It Stick ê 220 He also notches up his retrieval practice. In place of the written script, he now speaks every line of the other actors in the play into a palm- sized digital recorder, voiced “in character” as best he can discern it. He tucks the recorder in his hand. His thumb knows where to fi nd the controls. The thumb presses “play,” and Fuller hears the characters’ lines, then his cue; the thumb hits “pause,” and he speaks his line from memory. If in doubt about his accuracy, he checks the script, replays the passage if need be, speaks his lines, and then goes on with the scene. When he’s understudying a role, before the director and cast have worked out the blocking (how the players move in relation to one another and the set), Fuller practices at home, imagining his living room as the stage and the way the blocking might be laid out. There, as he goes through scenes with his recorder, hearing others’ lines and speaking his own, he is moving through the imagined scene, adding physicality to the part, reacting to imaginary props. When the actor he’s understudying is in rehearsal, Fuller observes from behind the theater seats at the back of the hall, walking through the blocking himself as the actors rehearse on stage. He continues to practice later at home, adapting the imaginary stage within his living room to the now- established blocking. Fuller’s learning pro cess is a seamless blend of desirable diffi culties: retrieval practice, spacing, interleaving, generation (of his character’s soul, carriage, motivations, and idiosyncrasies), and elaboration. Through these techniques, he learns the role and the many levels of meaning that make a per for mance come alive to himself and to his audience. Generation In 2013, John McPhee published a piece in the New Yorker about writer’s block. Age eighty- two at the time, McPhee of- Make It Stick ê 221 fered his remarks from the vantage of a high perch, atop an illustrious career that has earned him many awards and acknow ledg ment as a pioneer of the craft of creative nonfi ction. Writer’s block is the seemingly insurmountable barrier one must somehow clamber over if he is to have any hope of engaging his subject. Writing, like any art form, is an iterative pro cess of creation and discovery. Many would- be writers fail to fi nd their voices for the simple fact that, until they are clear about what they want to say, they cannot bring themselves to dive in. McPhee’s solution to this problem? He writes a letter to his mother. He tells her how miserable he feels, what hopes he’d had for the subject about which he wants to write (a bear), but that he has no idea how to go about it and, really, it seems that he’s not cut out to be a writer after all. He would like to put across the sheer size of the bear, and how utterly lazy it is, preferring to sleep fi fteen hours a day, and so on. “And then you go back and delete the ‘Dear Mother’ and all the whimpering and whining, and just keep the bear.” McPhee’s fi rst draft is an “awful blurting.” “Then you put the thing aside. You get in the car and drive home. On the way, your mind is still knitting at the words. You think of a better way to say something, a good phrase to correct a certain problem. Without the drafted version— if it did not exist— you obviously would not be thinking of ways to improve it. In short, you may actually be writing only two or three hours a day, but your mind, in one way or another, is working on it twenty- four hours a day— yes, while you sleep— but only if some sort of draft or earlier version exists. Until it exists, writing has not really begun.”4 This is the crux: Learning works the same way as McPhee’s “awful blurting.” Your grasp of unfamiliar material often starts out feeling clumsy and approximate. But once you engage the mind in trying to make sense of something new, the mind begins to “knit” at the problem on its own. You don’t engage the Make It Stick ê 222 mind by reading a text over and over again or by passively watching PowerPoint slides. You engage it by making the effort to explain the material yourself, in your own words— connecting the facts, making it vivid, relating it to what you already know. Learning, like writing, is an act of engagement. Struggling with the puzzle stirs your creative juices, sets the mind to looking for parallels and meta phors from elsewhere in your experience, knowledge that can be transferred and applied here. It makes you hungry for the solution. And the solution, when you arrive at it, becomes more deeply embedded with your prior knowledge and abilities than anything pasted onto the surface of your brain by PowerPoint. So take a page from McPhee: when you want to master something new, delete the whimpering and go wrestle the bear. Refl ection In Chapter 2 we tell how the Mayo Clinic neurosurgeon Mike Ebersold uses the habit of refl ection to improve his skills in the operating room. Refl ection involves retrieval (What did I do? How did it work?) and generation (How could I do it better next time?), invoking imagery and mental rehearsal as well (What if I take a smaller bite with the needle?). It was this habit of refl ection that brought him to devise a surgical solution for the repair of a delicate sinus structure in the back of the skull that cannot be tied off because the structure is somewhat fl at and tears when you snug the suture. Vince Dooley, Georgia Bulldogs football coach (Chapter 3), helped his players use refl ection and mental rehearsal to learn their playbooks and their adjustments for next Saturday’s game. The Minneapolis cop David Garman (Chapter 5) uses refl ection to improve his undercover strategies. The power of refl ection as a learning technique is apparent throughout the Make It Stick ê 223 personal memoir Highest Duty, by Captain Chesley Sullenberger. “Sully” is the pi lot who successfully and miraculously ditched US Airways Flight 1549 on the Hudson River in 2009. Time and again, in reading his autobiography, we see how he refi ned his understanding of fl ight and the control of his aircraft through training, personal experience, and the close observation of others. The pro cess started from his earliest days at the stick of a single- engine crop duster, continued to his jet fi ghter days, his time investigating commercial airline disasters, and his granular analysis of the few available examples of the ditching of commercial aircraft, where he paid par tic u lar attention to the lessons for pitch, speed, and level wings. The evolution of Captain Sullenberger shows us that the habit of refl ection is more than simply taking stock of a personal experience or the observed experiences of others. At its most powerful this habit involves engagement of the mind through generation, visualization, and mental rehearsal. Elaboration When we met the pianist Thelma Hunter, she was learning four new works for an upcoming concert per for mance: pieces by Mozart, Faure, Rachmaninoff, and William Bolcom. Hunter, who is eighty- eight, won her fi rst prize as a pianist at age fi ve in New York and has been performing ever since. She is not a prodigy, she insists, nor even particularly renowned, but she is accomplished. In addition to a busy life raising six kids with her husband, Sam, a heart surgeon, Hunter has enjoyed a long life of learning, teaching, and performing at the piano, and she is still in the game, sought after and bent to her life’s plea sure at the keyboard. Giving new learning multiple layers of meaning has been central to Hunter’s methods and illustrates the way elaboration Make It Stick ê 224 strengthens learning and memory. When she studies a new score, she learns it physically in the fi ngering, aurally in the sound, visually in the notes on the score, and intellectually in the way she coaches herself through transitions. Hunter has made some concessions to age. She never used to warm up before playing, but now she does. “My stamina is not as great as it used to be. My reach is not as big. Now, if I memorize something, I have to think about it. I never used to have to do that, I just worked through all the aspects of it and the memorizing came.”5 She visualizes the score and makes mental marginalia. “When I’m practicing, sometimes I say it out loud, ‘Up an octave, at this point,’ but in my mind’s eye I visualize the place on the sheet music, as well.” In comments that resonate with John McPhee’s observations about writing, Hunter says that at the point where a piece is almost memorized, “I’ll be driving, and I can think about the whole piece, which I do. The shape of it, as though I were a conductor, thinking, ‘Oh, that passage makes more sense if I speed it up. I have to practice that to get it faster.’ Those are the large things that I can think about away from the piano.” Hunter’s practice regimen is daily, working through new pieces, slowing down to parse the diffi cult passages, and then, because she now often performs with a cellist and violinist, the ensemble works through the pieces together to synchronize their individual interpretations. In Chapter 7 we describe Anders Ericsson’s research into how experts, through thousands of hours of solo, deliberate practice, build libraries of mental models that they can deploy to address a wide universe of situations they encounter in their area of expertise. Hunter describes experiences that would seem to manifest Ericsson’s theory. At times she must sit at the keyboard and devise a fi ngering plan for playing a diffi cult passage. Oddly, she says, after having been away from the piece Make It Stick ê 225 for a week, she will sit down and play it through, using a fi ngering pattern that she had not planned but feels entirely natural to her and familiar. It’s a paradox, though perhaps not entirely surprising. She credits her subconscious, drawing from her long years of playing, with fi nding a more fl uent solution than what she has devised by puzzling it out at the keyboard. But perhaps it has been the effort at the keys, like McPhee wrestling his bear, that has set her mind to sorting through the closets of her memory for something a little more elegant and natural to fi t the occasion. Tips for Teachers Here again we are leery of being too prescriptive. Every teacher must fi nd what’s right in his or her classroom. Yet specifi cs can be helpful. So here are some basic strategies that in our judgment will go a long way toward helping students become stronger learners in the classroom. Brief descriptions follow of what some teachers are already doing along these lines. Between the recommendations and the examples, we hope you will fi nd practical ideas you can adapt and put to work. Explain to Students How Learning Works Students labor under many myths and illusions about learning that cause them to make some unfortunate choices about intellectual risk taking and about when and how to study. It’s the proper role of the teacher to explain what empirical studies have discovered about how people learn, so the student can better manage his or her own education. In par tic u lar, students must be helped to understand such fundamental ideas as these: • Some kinds of diffi culties during learning help to make the learning stronger and better remembered. Make It Stick ê 226 • When learning is easy, it is often superfi cial and soon forgotten. • Not all of our intellectual abilities are hardwired. In fact, when learning is effortful, it changes the brain, making new connections and increasing intellectual ability. • You learn better when you wrestle with new problems before being shown the solution, rather than the other way around. • To achieve excellence in any sphere, you must strive to surpass your current level of ability. • Striving, by its nature, often results in setbacks, and setbacks are often what provide the essential information needed to adjust strategies to achieve mastery. These topics, woven throughout the book, are discussed in depth in Chapters 4 and 7. Teach Students How to Study Students generally are not taught how to study, and when they are, they often get the wrong advice. As a result, they gravitate to activities that are far from optimal, like rereading, massed practice, and cramming. At the beginning of this chapter we present effective study strategies. Students will benefi t from teachers who help them understand these strategies and stick with them long enough to experience their benefi ts, which may initially appear doubtful. Create Desirable Diffi culties in the Classroom Where practical, use frequent quizzing to help students consolidate learning and interrupt the pro cess of forgetting. Make Make It Stick ê 227 the ground rules acceptable to your students and yourself. Students fi nd quizzing more acceptable when it is predictable and the stakes for any individual quiz are low. Teachers fi nd quizzing more acceptable when it is simple, quick, and does not lead to negotiating makeup quizzes. (For one example, consider the way Kathleen McDermott, whose work we describe below, uses daily quizzing in her university class on human learning and memory.) Create study tools that incorporate retrieval practice, generation, and elaboration. These might be exercises that require students to wrestle with trying to solve a new kind of problem before coming to the class where the solution is taught; practice tests that students can download and use to review material and to calibrate their judgments of what they know and don’t know; writing exercises that require students to refl ect on past lesson material and relate it to other knowledge or other aspects of their lives; exercises that require students to generate short statements that summarize the key ideas of recent material covered in a text or lecture. Make quizzing and practice exercises count toward the course grade, even if for very low stakes. Students in classes where practice exercises carry consequences for the course grade learn better than those in classes where the exercises are the same but carry no consequences. Design quizzing and exercises to reach back to concepts and learning covered earlier in the term, so that retrieval practice continues and the learning is cumulative, helping students to construct more complex mental models, strengthen conceptual learning, and develop deeper understanding of the relationships between ideas or systems. (For an example, read in Chapter 2 how Andy Sobel uses cumulative lowstakes quizzing in his university- level course in po liti cal economics.) Make It Stick ê 228 Space, interleave, and vary topics and problems covered in class so that students are frequently shifting gears as they have to “reload” what they already know about each topic in order to fi gure out how the new material relates or differs. Be Transparent Help your students understand the ways you have incorporated desirable diffi culties into your lessons, and why. Be up front about some of the frustrations and diffi culties this kind of learning entails and explain why it’s worth persisting. Consider having them read the profi le earlier in this chapter of the medical student Michael Young, who vividly describes the diffi culties and ultimate benefi ts of using these strategies. Mary Pat Wenderoth, Biology Professor, University of Washington Mary Pat Wenderoth introduces desirable diffi culties in her classes to help students master their coursework. She also works at helping students learn how to be effective at managing their own learning— to be the capable student within the professional that they envision becoming. Along that path she tackles yet another challenge, helping students learn to judge where their grasp of course material stands on Bloom’s taxonomy of learning, and how to rise to the levels of synthesis and evaluation. Bloom’s taxonomy classifi es cognitive learning on six levels. It was developed in 1956 by a committee of educators chaired by psychologist Benjamin Bloom. The six levels range from gaining knowledge (the most fundamental level) to developing comprehension of the underlying facts and ideas, being able to apply learning to solve problems, being able to analyze ideas and relationships so as to make inferences, be- Make It Stick ê 229 ing able to synthesize knowledge and ideas in new ways, and, at the most sophisticated level, being able to use learning to evaluate opinions and ideas and make judgments based on evidence and objective criteria. Here are some of the main techniques Wenderoth uses. Transparency. At the outset, Wenderoth teaches her students about the testing effect, the principle of desirable diffi culties, and the perils of “illusions of knowing.” She promises to make her instructional philosophy transparent and to model these principles in class. As she explained to us recently, “The whole idea of the testing effect is that you learn more by testing yourself than by rereading. Well, it’s very hard to get students to do that because they’ve been trained for so long to keep reading and reading the book.”6 I can’t tell you how many times the students come to me and they show me their textbook and it’s highlighted in four different colors. I say to them, “I can tell you have done a lot of work and that you really want to succeed in this class because you have blue and yellow and orange and green highlighter on your book.” And then I have to try to tell them that any more time spent on this after the fi rst time was a waste. They’re, like, “How is that possible?” I say, “What you have to do is, you read a little bit and then you have to test yourself,” but they don’t quite know how to do that. So I model it in class for them. Every fi ve minutes or so I throw out a question on the material we just talked about, and I can see them start to look through their notes. I say, “Stop. Do not look at your notes. Just take a minute to think about it yourself.” I tell them our brains are like a forest, and your memory is in there somewhere. You’re here, and the memory is over there. The more times you make a path to that Make It Stick ê 230 memory, the better the path is, so that the next time you need the memory, it’s going to be easier to fi nd it. But as soon as you get your notes out, you have short- circuited the path. You are not exploring for the path anymore, someone has told you the way. At other times, Wenderoth will pose a question to the class and ask them to think about it. She has students write three possible answers on the whiteboard up front and then vote on which answer they think is correct by raising the number of fi ngers that corresponds with the answer on the board. She’ll instruct students to fi nd somebody with fi ngers “different from yours and talk to them and fi gure out who has the correct answer.” Wenderoth gives her students a new way to think about learning, and she gives them a new vocabulary for describing setbacks. When students trip over an exam question, they’ll commonly accuse the test of containing trick questions. When the student blames the test, she says, it’s not a good meeting ground for solving the problem. But now, students come to see her after a disappointing exam and say, “I have the illusion of knowing. How do I get better?” That’s a problem Wenderoth can help with. Testing groups. Wenderoth has transformed class “study groups” into “testing groups.” In a study group, the person who knows the most talks and the others listen. The emphasis is on memorizing things. However, in a testing group, they all wrestle with a question together, without opening the textbook. “Everybody has bits of information, and you talk with your colleagues and fi gure it out.” The emphasis is on exploration and understanding. Wenderoth will ask students in a testing group what ideas they don’t feel really clear on. Then she’ll send one student to Make It Stick ê 231 the whiteboard to try to explain the concept. As the student struggles, perhaps putting up the pieces of the answer she knows, the rest of the group are instructed to test her by asking questions whose answers will lead her to the larger concept. Throughout, all textbooks remain closed. Free recall. Wenderoth assigns her students to spend ten minutes at the end of each day sitting with a blank piece of paper on which to write everything they can remember from class. They must sit for ten minutes. She warns that it will be uncomfortable, they will run out of ideas after two minutes, but they must stick it out. At the end of ten minutes, they’re to go to their class notes and fi nd out what they remembered and what they forgot, and to focus on the material they forgot. What they glean from this exercise guides their notes and questions for the next class. Wenderoth fi nds that the free recall exercise helps students pull learning forward and develop a more complex understanding of how the material interrelates. Summary sheets. Every Monday, Wenderoth’s students are required to turn in a single sheet of certain dimensions on which they have illustrated the prior week’s material in drawings annotated with key ideas, arrows, and graphs. She’s teaching physiology, which is about how things work, so the summaries take on the form of large cartoons dense with callouts, blowups, directional arrows, and the like. The sheets help her students synthesize a week’s information, thinking through how systems are connected: “This is causing this, which causes this, which feeds back on those. We use a lot of arrows in physiology. The students can work with each other, I don’t care. The sheet they bring in just has to be their own.” Make It Stick ê 232 Learning paragraphs. From time to time, on a Friday, if she doesn’t feel she’s overburdening them, Wenderoth will assign students to write low- stakes “learning paragraphs” for which she poses a question and asks students to prepare a fi ve- or six- sentence response. A question might be “How is the GI tract like the respiratory system?” Or “You just got your tests back; what would you do differently next time?” The point is to stimulate retrieval and refl ection and to capture a week’s learning before it is lost to the countless other concerns and diversions of college life. “What I found over the years is, if I don’t do anything before the test, they don’t do anything until the day before the test.” The learning paragraphs also give her science majors practice in writing a passage of clear prose. She reads through the responses and makes a point to comment on them in class so that students know they’re being read. Bloom’s taxonomy of learning. To remove some of the abstraction from Bloom’s taxonomy, Wenderoth has translated her class material into the different levels of the taxonomy on an answer key to her tests. That is, for any given question, she provides a different answer for each level of the taxonomy: one that refl ects learning at the level of knowledge, a more thorough answer that refl ects understanding, a yet more complex answer that refl ects analysis, and so on. When students get their tests back, they also receive the answer key and are asked to identify where their answers fell on the taxonomy and to think about what they need to know in order to respond at a higher level of learning. Closing the achievement gap in the sciences. Wenderoth and her colleagues have experimented with class structure and the principles of active learning to help close the achievement gap in the sciences. Poorly prepared students seldom survive entry- Make It Stick ê 233 level science courses. As a result, even students whose interests and aptitudes might lead them to successful science careers never get through the door. For what ever reason, these students do not have a history from high school or family life of learning how to succeed in these highly challenging academic settings. “For most of us who have found our way in the sciences,” Wenderoth says, “any time we fell, there was somebody around to help us up, or to say, ‘This is how you get up.’ You were taught that when things don’t go well, you keep working anyway. You persevere.” In their experiments, Wenderoth and her colleagues have compared the results of “low- structure” classes (traditional lecturing and high- stakes midterm and fi nal exams) with “highstructure” classes (daily and weekly low- stakes exercises to provide constant practice in the analytical skills necessary to do well on exams). They also teach students the importance of having a “growth mindset” (see the work of Carol Dweck, discussed in Chapter 7)— that is, that learning is hard work and that struggle increases intellectual abilities. The results? High- structure classes in a gateway biology course signifi cantly reduced student failure rates compared to low- structure classes— narrowing the gap between poorly prepared students and their better prepared peers while at the same time showing exam results at higher levels on Bloom’s taxonomy. Moreover, it’s not just whether the student completes the practice exercises that matters. In the classes where exercises count toward the course grade, even at very low stakes, students achieve higher success over the course of the term compared to students in classes where the exercises are the same but carry no consequences for the grade. “We talk to the students about how these are the habits of mind,” Wenderoth says. “This is the discipline that you have to Make It Stick ê 234 have in order to succeed in the sciences. They’ve never thought about that, that every discipline has a culture. We teach them to think like the professionals they want to become. And when they fall, we show them how to get up again.”7 Michael D. Matthews, Psychology Professor, U.S. Military Academy at West Point The pedagogical philosophy at West Point is founded on an instructional system called the Thayer method, developed almost two hundred years ago by an early superintendent of the academy named Sylvanus Thayer. The method provides very specifi c learning objectives for every course, puts the responsibility for meeting those objectives on the student, and incorporates quizzing and recitation in every class meeting. Students’ grades at the academy rest on three pillars of training: academic, military, and physical. Mike Matthews, a professor of engineering psychology at the academy, says the load on students is enormous, greater than the hours available to them. In order to survive at the academy, West Point cadets must develop an ability to zero in on what’s essential and let the rest fall by the wayside. “This is about having very high expectations across multiple dimensions and keeping them real busy,” Matthews, says. In fact, as stunning as it sounds, Matthews will tell a student, “If you’ve read every word of this chapter, you’re not being very effi cient.” The point is not to “slide your eyes over the words.” You start with questions, and you read for answers.8 There’s little or no lecturing in Matthews’s courses. Class opens with a quiz on the learning objectives from the assigned reading. From there, on many days, students “take to the boards.” The classrooms have slate on all four walls, and a group of students are sent to each blackboard to collaborate Make It Stick ê 235 on answering a question given by the professor. These are higher- order questions than are given in the daily quiz, requiring the students to integrate ideas from the reading and apply them at a conceptual level. It’s a form of retrieval practice, generation, and peer instruction. One student is selected from each group to give a recitation to the class explaining how the group has answered the question, and then the group’s work is critiqued. All class meetings focus on constructs, not specifi c facts, and on the days the students do not take to the boards, they are engaged in other forms of exercise, demonstration, or group work aimed at understanding and articulating the larger concepts underlying the matter at hand. Clear learning objectives prior to each class, coupled with daily quizzing and active problem solving with feedback, keep students focused, awake, and working hard. One of the most important skills taught at West Point is something learned outside the classroom: how to shoot an azimuth. It’s a skill used for keeping your bearings in unfamiliar territory. You climb a tree or a height of land and sight a distant landmark in the direction you’re headed. Compass in hand, you note how many degrees your landmark lies off of due north. Then you descend into the bush and keep working your way in that direction. Periodically, you pause to shoot an azimuth and make sure you’re on course. Quizzing is a way of shooting an azimuth in the classroom: are you gaining the mastery you need to get where you’re trying to go? Matthews has had the privilege of seeing two of his students win Rhodes Scholarships. The most recent was Cadet Kiley Hunkler (now Second Lieutenant Hunkler). Hunkler will be spending the next two years at Oxford University, and then matriculating at Johns Hopkins Medical School. It was Hunkler who spoke to us of shooting an azimuth. “Everything at the academy is about self- responsibility, taking own ership for Make It Stick ê 236 fi nding your own way to the objective,” she said.9 The Medical College Admission Test, for example, encompasses four major course blocks: reading, chemistry, physiology, and writing. For each of these blocks, Hunkler created the learning objectives in her head that she deemed most important and then set out to answer them as she studied. “I took a practice test every three days, saw what I got wrong, and adjusted.” Shooting her azimuth. “A lot of students get hung up studying for months, trying to memorize everything, but for me it was more about understanding the concepts. So my azimuth check would be, Okay, what is this question asking, what’s the broader theme here, and does that match up with what I’ve outlined for this section.” One of this book’s authors (Roediger) attended Riverside Military Academy in Gainesville, Georgia, for high school. Riverside used a form of the Thayer method, with students having daily quizzes, problem sets, or assignments to be completed in class. The range of ability of these younger cadets was much more varied than at the elite US Military Academy at West Point, but the Thayer method worked well. In fact, such methods that include daily participation are especially likely to help students who are not prone to work hard on their own outside of class. The Thayer method is a strong encouragement for them to keep at it, and echoes what Mary Pat Wenderoth (above) has found in her empirical studies: that high- structure classes help students who lack a history of using effective learning techniques and habits to develop them and succeed in rigorous settings. Kathleen McDermott, Psychology Professor, Washington University at St. Louis Kathleen McDermott administers daily low- stakes quizzes in a university course on human learning and memory. It’s a class Make It Stick ê 237 of twenty- fi ve students that meets twice a week for fourteen weeks, minus midterms and a fi nal exam. She gives a fouritem quiz in the last three to fi ve minutes of every class. The questions hit the high points of the lecture, the readings, or both. If students have understood the material, they will get all four answers right, but they’ll have to think in order to do it. Anything covered in the course to date is fair game for a quiz, and she will sometimes draw from past material that she feels the students haven’t fully grasped and need to review. McDermott sets the ground rules very clearly at the start of the term. She lays out the research on learning and the testing effect and explains why the quizzes are helpful, even if they don’t feel helpful. Students are allowed to drop four quizzes across the semester. In exchange, absences need not be justifi ed, and no missed quizzes will be made up. Students initially are not happy about the quiz regime, and in the fi rst few weeks of the term McDermott will get email from students explaining why they had a legitimate excuse for an absence and should be allowed to make up a missed quiz. She reiterates the terms: four free absences, no makeups. McDermott says the quizzes provide an incentive for students to attend class and give students a way to contribute to their grade on a daily basis if they answer four out of four questions correctly. By the end of the semester, her students say that the quizzes have helped them keep up with the course and discover when they are getting off track and need to bone up. “The key with quizzes is to establish very clear ground rules for the student, and make them manageable for the professor,” McDermott says. “As a student, you’re either there and you take it, or you’re not. For the professor, no hassling over makeup tests.”10 The quizzes in totality count for 20 percent of a student’s grade in the course. In addition, McDermott gives two Make It Stick ê 238 midterm exams and a fi nal. The last two exams are cumulative. Having cumulative exams reinforces learning by requiring students to engage in spaced review. Columbia, Illinois, Public School District As recounted in Chapter 2, we have worked with teachers in a middle school in Columbia, Illinois, to test the effects of integrating low- stakes quizzing into the curriculum. Regular quizzing and other forms of retrieval practice have been adopted by teachers in the school who were a part of the research study and by others who were not but who observed the benefi cial results. The initial research project has since been extended into history and science classes in the district’s high school, where frequent retrieval practice is being used both to bolster learning and to help teachers focus instruction on areas where student understanding and per for mance need to be improved. The Illinois State Board of Education has adopted new math and En glish language arts standards for K– 12 education in line with the Common Core State Standards Initiative led by the National Governors Association and endorsed by the nation’s secretary of education. Common Core establishes standards for college and career readiness that students should be able to meet on graduation from high school. The Columbia School District, like others, is redesigning its curriculum and its tests to be more rigorous and to engage students in more writing and analysis work, with the aim of promoting the higher- level skills of conceptual understanding, reasoning, and problem solving that will enable students to meet the standards established by the state. As one example of this overhaul, the sciences curriculum is being vertically aligned so that students are reexposed to a subject at various stages of their school Make It Stick ê 239 careers. The result is more spaced and interleaved instruction. In physical sciences, for instance, middle school students may learn to identify the six basic machines (inclined plane, wedge, screw, lever, wheel and axle, and pulley) and how they work, and then may return to these concepts in subsequent grades, delving into the underlying physics and how these basic tools can be combined and applied to solve different problems. Tips for Trainers Here are some ways trainers are using the same principles as those who teach in schools, in a variety of less structured and nonclassroom settings. In- Service Training Licensed professionals in many fi elds must earn continuing education credits to keep their skills current and maintain their licenses. As the pediatric neurologist Doug Larsen describes in Chapter 3, this kind of training for doctors is typically compressed into a weekend symposium, out of respect for participants’ busy schedules, set at a hotel or resort, and structured around meals and PowerPoint lectures. In other words, the strategies of retrieval practice, spacing, and interleaving are nowhere to be seen. Participants are lucky to retain much of what they learn. If you see yourself in this scenario, there are a few things you might consider doing. One, get a copy of the pre sen ta tion materials and use them to quiz yourself on the key ideas, much as Nathaniel Fuller quizzes himself on the arc of a play, his lines, the many layers of character. Two, schedule follow- up emails to appear in your inbox every month or so with questions that require you to retrieve the critical learning you gained from the seminar. Three, contact your professional association Make It Stick ê 240 and ask them to consider revamping their approach to training along the lines outlined in this book. The testing effect forms the basis of a new commercial training platform called Qstream that helps trainers send learners periodic quizzes via their mobile devices to strengthen learning through spaced retrieval practice. Similarly, an emerging platform called Osmosis uses mobile and Web based software to provide learners access to thousands of crowdsourced practice questions and explanations. Osmosis combines the testing effect, spacing, and social networking to facilitate what its developers call “student- driven social learning.” Qstream (qstream.com) and Osmosis (osmose- it.com) suggest interesting possibilities for redesigning in- service training for professionals. Many other companies are developing similar programs. Kathy Maixner, Business Coach The Maixner Group is a consulting shop based in Portland, Oregon, that helps companies identify growth strategies and improve their sales tactics. Kathy Maixner fries big fi sh and little. One of the big fi sh added $21 million to its annual revenue as a result of hooking up with Maixner. One of the small ones, Inner Gate Acu punc ture (profi led at the close of this chapter), learned how to establish a solid business management footing under a clinical practice whose growth was outpacing its control systems. We’re interested in Maixner because the coaching techniques she has developed over her career line up so well with the learning principles described in this book. In short, Maixner sees her role as helping the client dig past the symptoms of a problem to discover its root causes, and then to generate possible solutions and play out the implications of different strategies before committing to them. Make It Stick ê 241 Maixner told us: “If you hand people the solution, they don’t need to explore how you got to that solution. If they generate the solution, then they’re the ones who are traveling down that road. Should they go left or right? We discuss the options.”11 Maixner’s years of experience working with clients in many different fi elds helps her see around corners, where the hazards lie. She often uses role- playing to simulate problems, getting her clients to generate solutions, try them out, get feedback, and practice what works. In other words, she introduces the diffi culties that make the learning stronger and more accurately refl ect what the client will encounter out in the marketplace. Farmers Insurance Corporate sales training can be complicated. Typically, it’s about corporate culture, beliefs and behavior, and learning to promote and protect the brand. It’s also technical, learning the features and advantages of the products. And it’s partly strategic, learning about the target market and how to generate prospects and make sales. At Farmers Insurance, whose principal sales force is a cadre of about fourteen thousand exclusive inde pen dent agents, training must also equip the company’s reps to become successful as entrepreneurs, building and managing their own agency. Farmers sells property and casualty policies and investment products like annuities and mutual funds to the tune of about $20 billion a year. Describing the full scope of their training could fi ll volumes, but we’ll focus on the way Farmers brings new agents on board, training them in the four areas of sales, marketing systems, business planning, and advocacy of the brand. The company’s new-agent training is an excellent example of interleaving the learning and practice of different Make It Stick ê 242 but related topics so that each adds meaning to the other, broadening and deepening competency. The company recruits upward of two thousand new agents annually. Many leave traditional jobs elsewhere, drawn to the rewards of running their own business and the opportunity to represent an established product line. Newly appointed agents arrive at one of two training campuses for an intensive weeklong program of learning exercises that spiral upward in sophistication. At the start, participants are given a pile of magazines, scissors, and marking pens with which to illustrate on posterboard what being a successful Farmers agent would look like to them personally, fi ve years down the road. For some, the poster shows fancy houses and cars. For others, kids are being sent to college and aging parents are being cared for. The point is simple: if your defi nition of success requires, say, $250,000 a year in revenues and twenty- fi ve hundred policies in force, we can help you work backward to set the metrics for where you need to be in four years, in three years, and even three months from now. The image on the poster shows where you’re headed, the metrics are your road map, and the skills that are learned over the coming days and months are the tools that will enable you to make the journey. From here, the week is not so much about teaching from the top down— there are no PowerPoint lectures as such— but about learning from the bottom up, as in: “What knowledge and skills do I need in order to succeed?” The learning unfolds through a series of exercises that cycle through the principal topics of sales, marketing systems, business planning, and advocacy of the company’s values and its brands— returning time and again to each, requiring that participants recall what they have learned earlier and apply it in a new, enlarged context. Make It Stick ê 243 For example, when participants fi rst arrive, they’re assigned to a red, blue, or green group. The red group is instructed to go meet people in the room. The blue group is instructed to go learn three things about somebody in the room. The green group is instructed to ask another member of the class about his or her family, prior occupation, favorite forms of recreation, and what he or she enjoys most. When the class reconvenes, they share what they have learned about others, and it is quickly evident that the green group, which had a structure for talking to others, learned much more than did their peers. When talking about sales later in the week the question comes up, what’s an effective way to learn about a prospective customer? Somebody will recall the initial get- acquainted exercise that proved so fruitful: asking about one’s family, occupation, recreation, and enjoyment. That icebreaker now morphs into a handy tool for getting to know a prospective client and it gets an acronym: FORE. Throughout the week the four principal training topics are repeatedly touched on, a point is made, and the exercises shift to related questions. In one session, participants brainstorm what kinds of marketing and development strategies might generate the fl ow of leads they need in order to meet sales targets. An effective sales and marketing system has a structure called 5- 4- 3- 2- 1. Five new business marketing initiatives every month, four cross- marketing and four retention programs in place, three appointments scheduled every day, two appointments kept (prospects often have to reschedule), one new customer sold on average two policies per sale. At twenty- two working days a month, that’s about fi ve hundred new policies in a year, making twenty- fi ve hundred over the fi ve- year horizon of the agent’s vision. Practice is a central learning strategy. For example, they practice how to respond to a sales lead. Trying to sell the Make It Stick ê 244 company’s products is how they learn about selling, but it’s also how they learn about the products they’re selling— not by sitting in front of PowerPoint slides gazing at long lists of product features. You be the agent, I’ll be the customer. Then we’ll switch. Interwoven with these exercises are others that help the new agents learn about the company’s history, what it stands for, and the value of its products in people’s lives, for instance through stories of how it has helped people recover from catastrophes like Hurricane Katrina. Given the emphasis on marketing and the limited resources new agents have to invest, how does an agent determine which strategies will pay? The question goes out: What’s a reasonable return to expect from a direct mail campaign? The agents mull it over and hazard guesses. Usually, one or more of the agents will have had direct- mail marketing experience and offer the sobering answer: returns are closer to 1 percent than the 50 percent many had guessed. Once you turn up a lead, how do you discover needs he or she has that the company’s products can meet? They return to the handy acronym FORE. Now, the habit of asking about one’s family, occupation, recreation, and enjoyment becomes something even more potent than a tool for getting acquainted. It provides an opening into four of the most important realms of a prospect’s life where insurance and fi nancial products can help that person protect his or her assets and achieve his or her fi nancial goals. At each pivot from one subject back to another, understanding deepens, and new skills take form. In this way, through generation, spaced practice, and interleaving of the essential core curriculum, with an eye always to the fi ve- year vision and road map, new agents learn what they need to do, and how, in order to thrive as a part of the Farmers Insurance family. Make It Stick ê 245 Jiffy Lube If you don’t expect innovations in training to spring from your local ser vice garage, Jiffy Lube may surprise you. An integrated suite of educational courses under the felicitous name Jiffy Lube University is helping the company’s franchisees win customers, reduce employee turnover, broaden their ser vice offerings, and boost sales. Jiffy Lube is a network of more than two thousand service centers in the United States and Canada that provide oil changes, tire rotation, and other automotive ser vices. Although the company is a subsidiary of Shell Oil Company, every outlet is owned and operated by an in de pen dent franchisee, who hires employees to serve customers. The rapid- oil- change business, like most others, has had to adjust to changes in the marketplace and advances in technology. Synthetic lubricants have made oil changes less frequent, and because cars have become more complicated, garage employees need higher levels of training to understand diagnostic codes and provide appropriate ser vices. No employee may work on a customer’s car until he or she has been certifi ed as profi cient. For this, they enter Jiffy Lube University, a Web- based learning platform. Certifi cation starts with interactive e-learning, with frequent quizzing and feedback to learn what a par tic u lar job entails and how it’s to be performed. When employees score 80 percent or better on an exam, they are eligible to begin training on the job, practicing new skills by following a written guide that breaks each service activity into its component steps. The steps may number as many as thirty and are performed as a part of a team, often involving call and response (for example, between a technician working from the top side of an engine and another underneath). A supervisor coaches the employee and rates his or Make It Stick ê 246 her per for mance on each step. When the technician demonstrates mastery, certifi cation is recorded in his or her permanent fi le, signed by the supervisor. Technicians must recertify every two years to keep their mastery up to snuff and adapt to operational and technical changes. Higher- level jobs for advanced ser vices like brake repair or running engine diagnostics are trained in the same manner. The e-learning and on- the- job training are active learning strategies that incorporate various forms of quizzing, feedback, and spaced and interleaved practice. All progress is displayed by computer on a virtual “dashboard” that provides an individualized learning plan, enabling an employee to track his or her per for mance, focus on skills that need to be raised, and monitor his or her progress against the company’s completion schedule. Jiffy Lube employees are typically eigh teen to twenty- fi ve years old and fi lling their fi rst job. As a technician is certifi ed in one job, he or she begins training in another, until he or she has trained in all store positions, including management. Ken Barber, Jiffy Lube International’s manager of learning and development, says training has to be engaging in order to hold employees’ attention. At the time we spoke, Barber was putting the fi nishing touches on a computer- based simulation game for company managers called “A Day in the Life of a Store Manager.” The ser vice center manager is confronted with various challenges and is required to select among a range of possible strategies for resolving them. The manager’s choices determine how the game unfolds, providing feedback and the opportunity to strive for better outcomes, sharpening decisionmaking skill. In the six years since Jiffy Lube University was launched, it has received many accolades from the training profession and earned accreditation by the American Council on Education. Make It Stick ê 247 Employees who progress through training in all job certifi cations can enroll at a postsecondary institution with seven hours of college credit under their belts. Since the program’s beginning, employee turnover has dropped and customer satisfaction has increased. “For most employees of a Jiffy Lube franchisee, this is a way into the workforce, and the training curriculum helps them to continue to grow and expand their knowledge,” Barber says. “It helps them fi nd a path to success.”12 Andersen Windows and Doors At Andersen Windows and Doors, a culture of continuous improvement turns learning on its head: the production workers teach the managers how to make the plant more effi cient. This story is a little different from the others in this chapter in two respects. It’s partly about creating a learning culture in the workplace, and partly about empowering employees to use what they learn to change the workplace. By encouraging employees to identify problems on the job and propose improvements, the company is supporting one of the most powerful learning techniques we have discussed, wrestling to solve a problem. A good place to focus is on the company’s division called Renewal by Andersen, which produces replacement windows of all types and sizes: double- hung, casement, gliding, picture windows, and specialty windows in nontraditional shapes. At Renewal by Andersen’s facility in Cottage Grove, Minnesota, their double- hung production line employs thirty- six people during an eight- hour shift that is divided into three work cells, one for sash fabrication, another for frame fabrication, and one for fi nal assembly. Each work cell has four work stations and is led by a crew leader who is responsible for Make It Stick ê 248 safety, quality, cost, and delivery within that cell. Workers change jobs every two hours to minimize repetitive stress injuries and broaden cross- training. Like interleaving the practice of two or more different but related topics, frequent switching between jobs builds an understanding of the integrated pro cess for which their unit is responsible and equips workers to respond more broadly to unexpected events that arise. It probably won’t surprise you to learn that every job is performed to a written standard that describes each step and the way it is to be taken. The written standard is essential for uniformity of product and quality. Without it, plant manager Rick Wynveen says, four different people will perform the job in four different ways, and produce four different versions of the product. When a new employee comes on board, he or she is trained following an instructional sequence of practice and feedback that Wynveen calls “tell— show—do—review.” The new worker is paired with an experienced worker, practice is on- the- job, and feedback brings learning and per for mance in line with the written standard. How do the workers train the managers? When a worker has an idea for improving productivity and management endorses it, for instance revamping the way parts arrive at a work station to make life easier for the worker and assembly faster, the worker who offered it takes leave from production to help implement the new standard. “Everyone’s idea is valuable,” Wynveen told us, “whether you’re an engineer, a maintenance technician, or a production worker.”13 Likewise, when one of the production line teams comes up short in meeting its targets, it’s the workers who are asked to identify the problem and redesign the production pro cess to solve it. The instructional role of employees is most dramatically illustrated in what Wynveen calls a Kaizen event. Kaizen is a Make It Stick ê 249 Japa nese term for improvement. It has been central to Toyota Motor Company’s success and has been adopted by many other companies to help create a culture of continuous improvement. When Wynveen wanted to effect a major increase in the productivity of the plant’s double- hung window line, he recruited a design team to engage in a Kaizen event. The team consisted of an engineer, a maintenance technician, a crew leader from the production line, and fi ve production workers. They were given the stretch goals of reducing the line’s space requirement by 40 percent and doubling production. (Stretch goals are ones that cannot be reached through incremental improvement but require signifi cant restructuring of methods.) The team met in a conference room eight hours a day for a week, in effect teaching each other the elements, capacities, and constraints of the production pro cess and asking themselves how to make it smaller and better. The following week they came back to Wynveen saying “Here’s what we think we can do.” Wynveen took their plan to each of the twelve work stations on the line with a simple question: What changes are needed to make this plan work? Production workers and their crew leaders put their heads together and redesigned the components to fi t the new plan. The line was disassembled and rebuilt in two halves, over two weekends, restarted, and fi ne- tuned over subsequent months, a pro cess that generated yet an additional two hundred improvements suggested by production workers: a learning pro cess of testing, feedback, and correction. The result? After fi ve months, the plant had met Wynveen’s stretch goals and cut costs in half. During the conversion and shakedown, the production teams never missed a shipment and never had a quality issue. The principle of engagement— actively seeking the ideas of employees from all levels of the Make It Stick ê 250 plant—is central to the company’s culture of continuous improvement. “Engagement is a management style of trust and a willingness to talk,” Wynveen says. The production employees learned how to refi ne the design as they worked, and the company provided a way for suggestions to be heard and for employees to participate in their implementation. A learning culture places the responsibility for learning with the employees and empowers them to change the system. Problems become information rather than failures. And learning by solving the problems (generation) and by teaching others (elaboration) becomes an engine for continuous improvement of per for mance by individuals and by the production line that they compose. Inner Gate Acu punc ture There are times when getting learning and teaching right can shape the trajectory of an entire life. Consider Erik Isaacman, a thirty- something husband, father of two, and passionate practitioner of traditional Chinese medicine: acu punc ture, massage, and herbal therapy. We close this chapter with the story of a turning point in Erik’s fl edgling practice, Inner Gate Acu punc ture in Portland, Oregon. It’s the story of a clinic that was succeeding in its therapeutic mission but struggling as a business. Erik and his business partner, Oliver Leonetti, opened Inner Gate in 2005, after earning graduate degrees in traditional Chinese medicine. Through networking and creative marketing, they began to build a stream of clients. Portland is fertile territory for alternative therapies. The business grew, and so did expenses: They leased larger space, hired an assistant to schedule appointments and manage the offi ce, brought in a third clinician, and hired a back- offi ce employee. “We were Make It Stick ê 251 growing 35 to 50 percent every year,” Erik recalled when we spoke. “The growth covered up a lot that was missing: We didn’t have the systems in place to manage costs. We didn’t have clear goals or a management hierarchy. It was fast becoming clear that we had no idea how to run a business.”14 One of Erik’s patients is the Oregon business coach Kathy Maixner. Maixner offered to help. “Unmanaged growth is scary,” she told us. “You jump ahead, then you fl ounder.” She asked a lot of questions that quickly focused Erik’s and Oliver’s thinking on critical gaps in their systems. The three then set out a schedule of frequent coaching sessions, between which Erik and Oliver generated elements of the missing infrastructure: operating manual, job descriptions, fi nancial goals, metrics for mea sur ing the per for mance of their clinicians. Every business serves two masters, its customer and its bottom line. “Our clinicians need to understand more than how to practice traditional Chinese medicine,” Erik said, as he refl ected on his and Oliver’s learning curve. “They need to understand how to turn a patient visit into a relationship, and how to help the patient understand his insurance coverage. Satisfying our customers is our highest priority. But we have to pay the bills, too.” Maixner used generation, refl ection, elaboration, and rehearsal in her coaching sessions, asking questions that exposed gaps in thinking or that invited the partners to strengthen their understanding of the behavior and tools they needed to adopt in order to be effective managers who delegate and empower their employees. They developed a system to track clinic metrics, like the number of patient visits, patient disappearance rates, and referral sources. They learned how to ensure they were paid appropriately by insurance companies, raising reimbursements from as little as 30 cents on the dollar. They drafted a uniform Make It Stick ê 252 protocol, or template, for clinicians to follow in seeing a new patient. They role- played conversations between themselves and their employees. Central to putting the clinic on sound footing has been Erik’s becoming an effective coach and teacher of his coworkers. “We’re not just letting it be intuitive,” he said. For example, the new protocol for clinicians to follow in a patient’s initial session helps to clarify what brought the patient in, the therapies that might be useful, how to describe these therapies in terms the patient would be likely to understand, how to discuss fees and insurance reimbursement options, and how to recommend a treatment plan. “If you’re the clinician, we’ll role- play: You are now the patient, and I’m the clinician. We raise questions, objections, and we practice how to respond and end up at the right place for the patient and for the clinic. Then we’ll switch roles. We record the role playing, and we listen to the differences: how you have responded to the patient, and how I have responded.” In other words, learning through simulation, generation, testing, feedback, and practice. As we write this, Inner Gate is in its eighth year, supporting four clinicians and two and a half administrative staff. A fi fth clinician is coming up to speed, and the partners are looking to open a second location. By dedicating themselves to being learners as well as teachers, Erik and Oliver have turned their passion into a solid enterprise, and a top- rated acu punc ture clinic in Portland. We have talked throughout this book about learning, not about education. The responsibility for learning rests with every individual, whereas the responsibility for education (and training, too) rests with the institutions of society. Education embraces Make It Stick ê 253 a world of diffi cult questions. Are we teaching the right things? Do we reach children young enough? How should we mea sure outcomes? Are our young people mortgaging their futures to pay for a college degree? These are urgent issues, and we need to wrestle through them. But while we’re doing that, the techniques for highly effective learning that are outlined in this book can be put to use right now everywhere learners, teachers, and trainers are at work. They come at no cost, they require no structural reform, and the benefi ts they promise are both real and long- lasting.