ch8

Being Mixed Up

Interleaving as an Aid to Comprehension

At a certain age—nine, ten, eleven, we were all there once—most of us are capable of the kind of blind devotion it takes to master some single, obscure skill that we’ve decided is central to our identity. Maybe it’s drawing a horse, or copying a guitar solo, or dribbling a basketball behind our back. Maybe it’s an ollie, that elementary skateboarding move, a kind of standing jump where the feet never leave the board. We don’t need a manual to tell us what to do, we just do it. Repeatedly. Head-down, nose-to-the-grindstone, just like we’ve been told. A belief in repetition is in the cultural water supply, in every how-to-succeed manual and handbook, every sports and business autobiography. There’s a reason that coaches, music instructors, and math teachers often run their students through drills, followed by more drills: Perform one hundred A-minor scales (or free throws, or wedge shots) in an afternoon and you will see progress. Do another two hundred and you’ll see more still.
Our faith in repetition never leaves us, not entirely. I sometimes think—if only I could channel my childlike devotion today when trying to learn something new. I’d channel it into the piano, or genetics, or mechanics. I’d practice like a machine, one skill at a time, until each one was automatic, driven deep into the marrow. Play Elgar, save some lives, fix the car when it broke down. At some level, I sort of believe it could still happen, given enough time. Some psychologists and writers have even tried to quantify that time. The path to exceptional performance, they argue, is through practice: ten thousand hours of it, to be exact. The gist of that rule is hard to resist, even if the number itself is arbitrary, because we read it in terms of repetition, as well as quantity. As the common exhortation goes: Don’t practice until you get it right. Practice until you can’t get it wrong.
Then I remember. I remember what happened in my own life when I did put in the time.
I was Mr. Repetition as a kid. As a student, as a music student, as an athlete. I was the one who did three hundred ollies in an afternoon, never quite getting it right. There I was, scraping up the driveway, only to look up and see some kid who didn’t have anywhere near my determination roll by, popping clean jumps without even thinking about it. Same for the behind-the-back dribble, the guitar solo, the inside-skate stop in hockey. I wanted it so bad, I’d throw myself into practicing but somehow never got good—while other kids who weren’t putting in nearly the same amount of dedicated time picked up the skills without seeming to sweat the details. Were they just … naturals? Did they have private teachers? Secret handshakes? I had no idea. I blamed my own lack of native gifts and kept looking for something that would come easily. What I never did was stop to ask whether my approach to practice was, in fact, the right one.
Nor did anyone else, at least not back in the early 1970s. At that time, scientists thought about practice in the same way we all did: the more, the better. To put it in precise terms, psychologists argued that any variation in the practice schedule that makes the target skill—whether in skating, algebra, or grammar—more immediate, more frequent, and more accurate improves learning. Brute-force repetition does that, and everyone who truly masters a skill has done at least some of it, usually lots. That’s the part they tend to remember later on, too—the repetition—and not other innovations or alterations they might have incorporated along the way.
One of the first hints that there might be another way came in a 1978 experiment by a pair of researchers at the University of Ottawa. Robert Kerr and Bernard Booth were trained in kinetics, the study of human movement. Kineticists often work closely with trainers and coaches, and they’re interested in the factors that contribute to athletic ability, injury recovery, and endurance. In this case, Kerr and Booth wanted to know how two distinct kinds of practice affected a simple, if somewhat obscure, skill: beanbag tossing. (It was an inspired choice, as it turned out; it’s a skill that most of us have tried, at a kid’s birthday party or some amusement park game, but that no one works on at home.) They recruited thirty-six eight-year-olds who were enrolled in a twelve-week Saturday morning PE course at a local gym and split them into two groups. The researchers ran both groups through a warm-up session of target practice to get the kids used to the game—and an awkward game it was, too. The children were asked to toss small golf-ball-sized beanbags from a kneeling position at bull’s-eyes on the floor. But they did so while wearing a harness that held a screen blocking their eyes. They took each shot blindly, removed the screen to see where it landed—then took the next shot.
On an initial trial, the two groups scored equally well, displaying no discernible difference in skill level.
Then they began regular practice sessions. Each child had six practice sessions, taking twenty-four shots every time. One group practiced on one target, a bull’s-eye that was just three feet away. The other group practiced on two targets, one that was two feet away and another that was four feet away, alternating their shots. That was the only difference.
At the end of the twelve-week course, the researchers gave the children a final test on performance—but only on the three-foot target. This seems unfair. One group was practicing on the three-foot target the whole time, and the other not at all. The group that practiced at three feet should have had a clear advantage. Yet it didn’t turn out that way. The kids in the mixed-target group won this competition, and handily. Their average distance away from the (three-foot) target was much smaller than their peers on the final test. What was going on? Kerr and Booth ran the same experiment again in twelve-year-olds, just to make sure the finding held up. It did. Not only that, but the result was even more dramatic in the older kids. Was it luck? Did the better groups have a few ringers? Not at all, reported Kerr and Booth. “A varied practice schedule may facilitate the initial formation of motor schema,” they wrote, the variation working to “enhance movement awareness.” In other words: Varied practice is more effective than the focused kind, because it forces us to internalize general rules of motor adjustment that apply to any hittable target.
A big idea—if true.
It might have been a fluke, given the strangeness of the task: blind beanbag tossing. Not that it mattered at the time, in part because no one was paying attention. The beanbag experiment was as obscure as they come. (So much so that it disappeared entirely from the website of the journal in which it originally appeared, Perceptual and Motor Skills; it took editors weeks to find it when I asked.) Yet even if the study had made the nightly news, it’s not likely to have changed many minds, certainly not among the academics studying memory. Kinetics and cognitive psychology are worlds apart in culture and in status. One is closer to brain science, the other to gym class. A beanbag study with a bunch of eight-year-olds and twelve-year-olds wasn’t about to alter centuries of assumptions about how the brain acquires new skills. At least not right away.
• • •

Psychologists who study learning tend to fall into one of two camps: the motor/movement, or the verbal/academic. The former focuses on how the brain sees, hears, feels, develops reflexes, and acquires more advanced physical abilities, like playing sports or an instrument. The latter investigates conceptual learning of various kinds: language, abstract ideas, and problem solving. Each camp has its own vocabulary, its own experimental paradigms, its own set of theories. In college, they are often taught separately, in different courses: “Motor and Perceptual Skills” and “Cognition and Memory.”
This distinction is not an arbitrary one. Before we go any further, let’s revisit, briefly, the story of Henry Molaison, the Hartford man whose 1953 surgery for epilepsy severely damaged his ability to form new memories. After the surgery, Molaison’s brain could not hold on to any describable memories, such as names, faces, facts, and personal experiences. The surgeon had removed the hippocampus from both hemispheres of his brain; without those, Molaison could not move short-term memories into long-term storage. He could, however, form new motor memories. In one of the experiments described in chapter 1, Molaison learned to trace a star while watching his drawing hand in a mirror. He became more and more adept at this skill over time, even though he had no memory of ever practicing it.
A major implication of the Molaison studies was that the brain must have at least two biological systems for handling memory. One, for declarative memories, is dependent on a functioning hippocampus. The other, for motor memories, is based in different brain organs; no hippocampus required. The two systems are biologically distinct, so it stood to reason that they’re functionally distinct, too, in how they develop, strengthen, and fade. Picking up Spanish is not the same as picking up Spanish guitar, and so psychology has a separate tradition to characterize each.
In the early 1990s, a pair of colleagues at UCLA decided to try something radical: They would combine the two traditions—motor and verbal—into a single graduate seminar, which they called “Principles of Motor and Verbal Learning.” The two researchers—Richard A. Schmidt, a motor-learning specialist, and the ever-present Robert Bjork, a verbal-learning expert—thought students would gain a better understanding of the main distinctions between their respective fields and how each type of learning is best taught. “Dick and I just assumed we’d lay out what the differences were between motor and verbal, nothing more than that,” Bjork told me. “But as we got deeper into it, the whole project changed course.”
An odd signal echoed down through the literature, they saw. For starters, they stumbled upon the neglected beanbag study, and took its conclusions at face value, as valid. They then searched the literature to see if they could find other studies in which mixed or interrupted practice sessions led to better performance over time than focused ones. If the beanbag result was solid, and Kerr and Booth were correct in arguing that it revealed a general principle of learning, then it should show up in other experiments comparing different practice techniques.
And so it did, in papers by researchers who weren’t familiar with Kerr and Booth’s work at all. In 1986, for instance, researchers at Louisiana State University tested how well thirty young women learned three common badminton serves. The short serve, the long, and the drive each has a distinct trajectory and takes some practice to hit well. To make a short serve, the player has to hit the shuttlecock just over the net (no more than fifty centimeters, or a foot and a half) so that it lands in the front third of the opposing court. A long serve passes at least two and half meters (about eight feet) above the net and lands in the back third of the opposite court. A drive splits the difference and darts downward to the midline on the other side. The researchers—Sinah Goode and Richard Magill—judged the serves by two criteria: where they landed and where they passed over the net. They split the women into three groups of ten, each of which practiced according to the same schedule, for three days a week over three weeks, thirty-six serves at a time. The sessions themselves were different, however. Group A performed blocked practice, rehearsing only one type of serve per session: doing thirty-six short ones on one day, for instance, thirty-six long ones the next session, and thirty-six drives the next. Group B performed serial practice, trying the serves in a given order—short, then long, then drive—repeatedly. Group C practiced randomly, trying any serve they wanted but no more than two of the same ones in a row.
By the end of the three weeks, each participant had practiced each serve the same number of times, give or take a few for those in the random group.
Goode and Magill wanted not only to compare the relative effectiveness of each type of practice schedule. They also wanted to measure how well the participants’ skills transferred to a new condition. Transfer is what learning is all about, really. It’s the ability to extract the essence of a skill or a formula or word problem and apply it in another context, to another problem that may not look the same, at least superficially. If you’ve truly mastered a skill, you “carry it with you,” so to speak. Goode and Magill measured transfer in a subtle, clever way. On their final test of skill, they made one small adjustment: The participants served from the left side of the court, even though they’d practiced only on the right. During the test, the examiner called out one skill after another: “Hit me a drive … Okay, now a short serve … Now give me a long one.” Each participant hit each serve the same number of times on the final test—six—though never two of the same kind in a row. Goode and Magill then rated each serve, according to its arc and placement, on a scale from 0 to 24.
The winner? Team Random, by a long shot. It scored an average of 18, followed by the serial group, at 14. The blocked practicers, who’d focused on one serve at a time, did the worst, with an average of 12—and this despite having appeared, for most of the three weeks, to be improving the most. They were leading the pack going into Week 3, but come game time, they collapsed.
The authors weren’t entirely sure what caused such a dramatic reversal. Yet they had a hunch. Interfering with concentrated or repetitive practice forces people to make continual adjustments, they reasoned, building a general dexterity that, in turn, sharpens each specific skill. Which, by the way, is exactly what the beanbag study concluded. But Goode and Magill then took it one step further. All that adjusting during a mixed-practice session, they wrote, also enhances transfer. Not only is each skill sharper; it’s performed well regardless of context, whether indoors or out, from the right side of the court or the left. “The general goal of practice is to transfer to a game,” the pair concluded. “A game situation varies from event to event, making random testing the best condition to appraise the effectiveness of practice.”
Schmidt and Bjork knew that this experiment, like the beanbag toss, proved nothing on its own; it was just one study. But there was a scattering of still others—of keyboard ability, of videogame skills, of precise arm movements—and they all had one thing in common: Whenever researchers scrambled practice sessions, in one form or another, people improved more over time than if their practice was focused and uninterrupted.
One way to think about this is in terms of practice versus performance. During practice we have a measure of control. We can block out or avoid distractions, we can slow down if needed, and most important, we decide which skill or move or formula we want to rehearse before actually doing it. We’re in charge. Performance is another story. Growing up, all of us knew kids who were exceptional in practice but only mediocre come game time. And vice versa, kids who looked awkward in drills and then came alive when it mattered, during competition, or performing in front of an audience. You can practice the step-over soccer move a thousand times in your front yard, but doing it at full speed with two opposing players running at you is much harder. It’s no longer a single move anymore, practiced in isolation, but one step in an ever-changing, fast-paced dance.
The incorporation of these random demands is what made Kerr and Booth’s observation plausible, and Schmidt and Bjork knew well enough that the principle wasn’t only applicable to physical skills. Digging out verbal memories on a dime requires a mental—if not physical—suppleness that doesn’t develop in repetitive practice as fast as it could. In one previous experiment, Bjork and T. K. Landauer of Bell Laboratories had students try to memorize a list of fifty names. Some of the names were presented for study and then tested several times in succession; other names were presented once and tested—but the test came after the study session was interrupted (the students were given other items to study during the interruption). In other words, each student studied one set of names in an unperturbed session and the other set in an interrupted one. Yet thirty minutes later, on subsequent tests, they recalled about 10 percent more of the names they’d studied on the interrupted schedule. Focused, un-harried practice held them back.
“It has generally been understood that any variation in practice that makes the information more immediate, more accurate, more frequent, or more useful will contribute to learning,” Schmidt and Bjork wrote. “Recent evidence, however, suggests that this generalization must be qualified.”
“Qualified” was a polite way to say “reconsidered” and possibly abandoned altogether.
It’s not that repetitive practice is bad. We all need a certain amount of it to become familiar with any new skill or material. But repetition creates a powerful illusion. Skills improve quickly and then plateau. By contrast, varied practice produces a slower apparent rate of improvement in each single practice session but a greater accumulation of skill and learning over time. In the long term, repeated practice on one skill slows us down.
Psychologists had been familiar with many of these findings, as isolated results, for years. But it was Schmidt and Bjork’s paper, “New Conceptualizations of Practice,” published in 1992, that arranged this constellation of disparate pieces into a general principle that can be applied to all practice—motor and verbal, academic as well as athletic. Their joint class turned out not to be devoted to contrasts, after all, but to identifying key similarities. “We are struck by the common features that underlie these counterintuitive phenomena in such a wide range of skill-learning situations,” they concluded. “At the most superficial level, it appears that systematically altering practice so as to encourage additional, or at least different, information processing activities can degrade performance during practice, but can at the same time have the effect of generating greater performance capabilities.”
Which activities are those? We’ve already discussed one example, in chapter 4: the spacing effect. Breaking up study time is a form of interference, and it deepens learning without the learner investing more overall time or effort. Another example, explored in chapter 3, is context change. Mixing up study locations, taking the books outside or to a coffee shop, boosts retention. Each of these techniques scrambles focused practice, also causing some degree of forgetting between sessions. In their Forget to Learn theory, Robert and Elizabeth Bjork called any technique that causes forgetting a “desirable difficulty,” in that it forces the brain to work harder to dig up a memory or skill—and that added work intensifies subsequent retrieval and storage strength (learning).
But there’s another technique, and it goes right back to the long-lost beanbag study. Remember, the kids who did best on the final test hadn’t practiced on the three-foot target at all. They weren’t continually aiming at the same target, like their peers, doing a hundred A-minor scales in a row. Nor were they spacing their practice, or changing rooms, or being interrupted by some psychologist in a lab coat. They were simply alternating targets. It was a small variation, only a couple of feet, but that alteration represents a large idea, and one that has become the focus of intense study at all levels of education.
• • •

Let’s leave the beanbags and badminton behind for now and talk about something that’s more likely to impress friends, strangers, and potential mates: art. I’m not talking about creating art, I’m talking about appreciating it. One of the first steps in passing oneself off as an urbane figure (so I’m told) is having some idea who actually created the painting you’re staring at. Remarking on Manet’s use of light while standing in front of a Matisse can blow your cover quickly—and force a stinging retreat to the information desk for some instructional headphones.
Yet learning to identify an artist’s individual touch, especially one who has experimented across genres and is not among history’s celebrities, a van Gogh or a Picasso or an O’Keeffe, is not so easy. The challenge is to somehow feel the presence of the artist in the painting, and there’s no simple recipe for doing so. What’s the difference between a Vermeer, a de Heem, and a van Everdingen, for example? I couldn’t pick any one of these Dutch masters out of a lineup, never mind identify the creative signatures that separate one from the others. “The different subjects chosen by Vermeer and de Heem and van der Heyden and van Everdingen are at once different ways of depicting life in 17th-Century Holland and different ways of expressing its domestic quality,” wrote the American philosopher Nelson Goodman in one of his essays on artistic style. “Sometimes features of what is exemplified, such as color organizations, are ways of exemplifying other features, such as spatial patterns.”
Got all that? Me neither.
Goodman famously argued that the more elusive and cryptic an artist’s style, the more rewarding it was for the viewer: “An obvious style, easily identified by some superficial quirk, is properly decried as a mere mannerism. A complex and subtle style, like a trenchant metaphor, resists reduction to a literal formula.” And there’s the rub. Art appreciation is a world removed from biology, playing music, German 101, and the epic poets. There are no word pairs or chemical bonds to study, no arpeggios or verses or other basic facts, no obvious verbal or motor “tasks” to measure. The ability contains an element of witchcraft, frankly, and learning scientists had traditionally left the study of artistic styles to the likes of academics like Goodman.
That all changed in 2006, when Robert Bjork and postdoctoral student Nate Kornell, now at Williams College, decided to test whether a form of interrupted study affected aesthetic judgment in addition to retention. The idea came from a story that one of Bjork’s colleagues had told him, about taking a trip to Italy with her teenage daughter. She—the mother—was excited by the opportunity to visit great museums, such as the Uffizi and Accademia in Florence, the National and Borghese in Rome, as well as the vast Vatican collection, but she worried that the experience would be lost on her daughter, if not actively resisted. She told Bjork that she knew her daughter would get so much more out of the trip if she learned to identify Italian painters’ styles—and had devised a flashcard game that taught her to do just that.
Kornell and Bjork did essentially the same thing in their experiment. They chose a collection of paintings by twelve landscape artists, some of them familiar (Braque, Seurat), but most by artists unfamiliar to the participants, like Marilyn Mylrea, YeiMei, and Henri-Edmond Cross. They then had a group of seventy-two undergraduates study the paintings on a computer screen. Half of the students studied the artists one at a time. For example: They saw one Cross after another for three seconds each, with the name of the painter below the image:

After six Crosses, they saw (let’s say) six works by Braque, again for three seconds each with the artist’s name below; then six by YeiMei; and so on. Kornell and Bjork called this blocked practice, because the students studied each artist’s works in a set.
The other half of the participants studied the same paintings for the same amount of time (three seconds per piece), also with the artist’s name below. But in their case, the paintings were not grouped together by artist; they were mixed up:

Both groups studied a total of six paintings from each of the twelve artists. Which group would have a better handle on the styles at the end?
Kornell and Bjork had the participants count backward from 547 by threes after studying—a distraction that acted as a palette cleanser, a way to clear short-term memory and mark a clean break between the study phase and the final test. And that test—to count as a true measure of performance—could not include any of the paintings just studied. Remember, the participants in this study were trying to learn painting styles, not memorize specific paintings. If you “know” Braque, you should be able to identify his touch in a painting of his you’ve never seen before. So Kornell and Bjork had the students view forty-eight un-studied landscapes, one at a time, and try to match each one to its creator, by clicking on one of the twelve names. The researchers weren’t sure what to expect but had reason to suspect that blocked study would be better. For one thing, no one understands exactly how people distinguish artistic styles. For another, similar studies back in the 1950s, having subjects try to learn the names of abstract drawings, found no differences. People studying the figures in blocked sets did every bit as well as those studying mixed sets.
Not this time. The mixed-study group got nearly 65 percent of the artists correct, and the blocked group only 50 percent. In the world of science, that’s a healthy difference, so the researchers ran another trial in a separate group of undergraduates to double-check it. Once again, each student got equal doses of blocked and mixed study: blocked for six of the artists, mixed for the other six. The result was the same: 65 percent correct for those studied in mixed sets, and 50 percent for those studied in blocks. “A common way to teach students about an artist is to show, in succession, a number of paintings by that artist,” Kornell and Bjork wrote. “Counterintuitive as it may be to art history teachers—and our participants—we found that interleaving paintings by different artists was more effective than massing all of an artist’s paintings together.”
Interleaving. That’s a cognitive science word, and it simply means mixing related but distinct material during study. Music teachers have long favored a variation on this technique, switching from scales, to theory, to pieces all in one sitting. So have coaches and athletic trainers, alternating endurance and strength exercises to ensure recovery periods for certain muscles. These philosophies are largely rooted in tradition, in a person’s individual experience, or in concerns about overuse. Kornell and Bjork’s painting study put interleaving on the map as a general principle of learning, one that could sharpen the imprint of virtually any studied material. It’s far too early to call their study a landmark—that’s for a better historian than I to say—but it has inspired a series of interleaving studies among amateurs and experts in a variety of fields. Piano playing. Bird-watching. Baseball hitting. Geometry.
What could account for such a big difference? Why any difference at all? Were the distinctions between styles somehow clearer when they were mixed?
In their experiment, Kornell and Bjork decided to consult the participants. In a questionnaire given after the final test, they asked the students which study method, blocked or interleaved, helped them learn best. Nearly 80 percent rated blocked study as good or better than the mixed kind. They had no sense that mixed study was helping them—and this was after the final test, which showed that mixing provided a significant edge.
“That may be the most astounding thing about this technique,” said John Dunlosky, a psychologist at Kent State University, who has shown that interleaving accelerates our ability to distinguish between bird species. “People don’t believe it, even after you show them they’ve done better.”
This much is clear: The mixing of items, skills, or concepts during practice, over the longer term, seems to help us not only see the distinctions between them but also to achieve a clearer grasp of each one individually. The hardest part is abandoning our primal faith in repetition.
Math scores, however, don’t lie.
• • •

Despite its leadership in technical innovation and discovery, the United States has long lagged in math education, usually ranking around ninth or tenth in the world—as measured by performance in eighth graders—far behind countries like South Korea and Finland. Experts and officials are perpetually debating how to close that gap, and in the late 1980s the nation’s premier organization of math teachers—the National Council of Teachers of Mathematics—convened a meeting of leading educators to review and reshape how the subject was taught. It was a gargantuan job and, like so many grand-scale efforts, became bitterly contentious. The central disagreement was over teaching philosophy: Do students learn most efficiently in classes that emphasize the learning of specific problem-solving techniques, like factoring and calculating slope? Or do they benefit more from classes that focus on abstract skills, like reasoning and number sense—knowing, for example, that 2/3 + 3/5 is greater than 1, without having to find a common denominator? The former approach is bottom-up; the latter is top-down.
This being education, the debate was quickly politicized. The top-down camp became “progressives” who wanted children to think independently rather than practice procedures by rote. (This group included many younger teachers and university professors with doctorates in education.) The bottom-up camp became “conservatives” who saw value in the old ways, in using drills as building blocks. (Its core was made up of older teachers and professors of math and engineering.) The math wars, as they were known, caused confusion among many teachers. Math education was virtually devoid of decent research at the time, so neither side had the ammunition to win the argument. The typical experiment involved academics or outside experts descending on a class or school with a novel math, history, or writing curriculum and announcing “improvements” that were hard to interpret, given that the measures (the tests) were often new themselves, and few experiments tracked the teachers’ commitment to the program.
Teachers, then as now, see enough new approaches come and go over time that many become constitutionally skeptical. Plus, this clash over math was (and is) about philosophies, and in math of all subjects it is results that matter, not theories. “One of the things you see that’s so baffling, when you’re a new teacher, is that kids who do great on unit tests—the weekly, or biweekly reviews—often do terribly on cumulative exams on the same material,” Doug Rohrer, who was a high school math teacher in Palo Alto, California, in the late 1980s, told me. “The kids would often blame the test or even blame me explicitly, saying I gave them trick questions.” What made those questions so tricky, explained Rohrer, was that “math students must be able to choose a strategy—not just know how to use it—and choosing a strategy is harder when an exam covers many kinds of problems.” For practical teaching issues like this one, the math wars debate was irrelevant.
Rohrer toyed with the idea of developing a different curriculum, one that rejected the idea of teaching in blocks (two weeks on proportions, say, then two weeks on graphs) and instead mixed problems from previously studied topics into daily homework to force students to learn how to choose appropriate solution strategies rather than blindly apply them. To solve a problem, you first have to identify what kind of problem it is. Rohrer was lying on his futon in his studio apartment one day, staring at the ceiling, and thought, Okay, maybe it’s time to write a textbook of mixed problems. He soon found out that someone already had.
That someone was a retired Air Force officer turned math teacher in Oklahoma City. In the 1970s, John H. Saxon was teaching math at Rose State College and growing increasingly exasperated with the textbooks the college used. The books’ approach left students fuzzy on the basics, and quick to forget what they’d just studied. So one day Saxon decided to write out some problem sets of his own, with the goal of building algebra skills differently—i.e., more incrementally—than the standard curriculum. His students improved fast, and soon he was developing entire lesson plans. Between 1980 and 1990, Saxon authored or coauthored twelve math textbooks for kindergarten through high school, plus a couple of college texts. His central innovation was a process of “mixed review.” Each homework assignment included some new technique—solving simultaneous equations, for example—along with a number of problems from previous lessons, say, solving equations for x. Saxon believed that we grasp a new technique more clearly when using it alongside other, familiar ones, gradually building an understanding of more abstract concepts along the way. His books built a following, mostly among private schools, homeschoolers, and some public districts, and he soon became a lightning rod in the math debate. Saxon was a bottom-up man. He thought the reformers were dangerous and they returned the compliment.
Rohrer wasn’t sure what he thought about the math wars or, for that matter, about Saxon. He does remember picking up the Saxon books and looking at the chapters. They were different, all right. The lessons, in Rohrer’s view, were not in logical order. Yet the problems were mixed, from all sorts of different lessons—precisely the approach he thought would help his own students.
He let it drop. Rohrer was ready to walk away from math teaching altogether, and entered graduate school in experimental psychology. It was in 2002—eight years after he finished his degree—that he again began to think about learning. For one thing, he’d read the 1992 Schmidt-Bjork paper on motor and verbal learning. And he returned to the central problem he’d had while teaching high schoolers. His students didn’t need to remember more. Their weakness was distinguishing between problem types—and choosing the appropriate strategy. Mixing problem types (he had not yet heard the term interleaving) looked like it might address just this weakness.
We’ve done well so far to avoid doing any real math in this book, but I think it’s time to break the seal. In the past decade, Rohrer and others have shown in a variety of experiments that interleaving can improve math comprehension across the board, no matter our age. Let’s take a look at one of those studies, just to show how this technique works. We’ll keep it light. This is fourth grade geometry, and a little review never hurt anyone. In 2007, Rohrer and Kelli Taylor, both at the University of South Florida, recruited twenty-four fourth graders and gave each a tutorial on how to calculate the number of faces, edges, corners, and angles in a prism—given the number of base sides. The tutorial is self-explanatory and perfectly doable, even for people with math allergies. In the diagrams below, b is the number of base sides:

Half the children performed blocked study. They worked eight “face” problems (FFFFFFFF), then eight “edge” problems (EEEEEEEE), eight “corner” problems, and eight “angle” problems in a row, with a thirty-second break in between, all in the same day. The other half worked the same number of each type of problem, only in randomly mixed sets of eight: FCEAECFA, for example, followed by CAAEFECF. The tutorials were identical for each group, and so were the problems. The only difference was the order: sequential in one group and mixed in the other. The next day the children took a test, which included one of each type of problem. Sure enough, those in the mixed-study—interleaved—group did better, and it wasn’t close: 77 to 38 percent.
One fairly obvious reason that interleaving accelerates math learning in particular is that tests themselves—the cumulative exams, that is—are mixed sets of problems. If the test is a potpourri, it helps to make homework the same. There’s much more going on than that, however. Mixing problems during study forces us to identify each type of problem and match it to the appropriate kind of solution. We are not only discriminating between the locks to be cracked; we are connecting each lock with the right key. “The difficulty of pairing a problem with the appropriate procedure or concept is ubiquitous in mathematics,” Rohrer and Taylor concluded. “For example, the notorious difficulty of word problems is due partly to the fact that few word problems explicitly indicate which procedure or concept is appropriate. The word problem, ‘If a bug crawls eastward for 8 inches and then crawls northward for 15 inches, how far is it from its starting point?’ requires students to infer the need for the Pythagorean theorem. However, no such inference is required if the word problem appears immediately after a block of problems that explicitly indicate the need for the Pythagorean theorem. Thus, blocked practice can largely reduce the pedagogical value of the word problem.”
Rohrer puts it this way: “If the homework says ‘The Quadratic Formula’ at the top of the page, you just use that blindly. There’s no need to ask whether it’s appropriate. You know it is before doing the problem.”
The evidence so far suggests that interleaving is likely applicable not just to math, but to almost any topic or skill. Badminton. History (mix concepts from related periods). Basketball (practice around the free throw line, not repeatedly from the line). Biology. Piano. Chemistry. Skateboarding. Blindfolded beanbag throwing, for heaven’s sake. Certainly any material taught in a single semester, in any single course, is a ripe target for interleaving. You have to review the material anyway at some point. You have to learn to distinguish between a holy ton of terms, names, events, concepts, and formulas at exam time, or execute a fantastic number of perfect bow movements at recital. Why not practice the necessary discrimination skills incrementally, every time you sit down, rather than all at once when ramping up for a final test? As mentioned earlier, many musicians already do a version of mixed practice, splitting their sessions between, say, thirty minutes of scales, thirty minutes of reading new music, and thirty minutes of practicing familiar pieces. That’s the right idea. Chopping that time into even smaller pieces, however—of fifteen minutes, or ten—can produce better results. Remember: Interleaving is not just about review but also discriminating between types of problems, moves, or concepts.
For example, I still take classes when I can in Spanish and Spanish guitar. Every time I look at a list of new vocabulary words, I take that list and combine it with a list of at least as many older words. I do more kinds of mixing with the guitar (maybe because there’s more to mix than words and reading). I do one scale, two or three times, then switch to a piece I know. Then I go back and try again the portions of that just played piece—let’s say it’s Granados’s Spanish Dance Number 5—that I messed up. Play those two times, slowly. Then I’m on to a (different) scale, followed by a few bars of a totally new piece I’m working on. Enough for one pass. I take a break and play a few riffs from the first tune I ever learned, “Stairway to Heaven” (somehow it never gets old), and after that I’m ready to dive into Spanish Classical.
That is interleaving. And it’s sure to be highly individual, far more effective for some subjects or skills than for others. The important thing to know is that you’re essentially surrounding the new material or new skill set with older stuff, stuff you already know but haven’t revisited in a while, whether it’s a Jimmy Page solo or a painting by Georges Braque.
As I read it, the science suggests that interleaving is, essentially, about preparing the brain for the unexpected. Serious climbers and hikers have a favorite phrase: It’s not an adventure until something goes wrong. By wrong they mean wrong wrong. A rope snaps; the food supply flies overboard; a bear crawls into the tent. I think interleaving prepares us for a milder form of wrong. Every exam, every tournament, every match, every recital—there’s always some wrinkle, some misplaced calculator or sudden headache, a glaring sun or an unexpected essay question. At bottom, interleaving is a way of building into our daily practice not only a dose of review but also an element of surprise. “The brain is exquisitely tuned to pick up incongruities, all of our work tells us that,” said Michael Inzlicht, a neuroscientist at the University of Toronto. “Seeing something that’s out of order or out of place wakes the brain up, in effect, and prompts the subconscious to process the information more deeply: ‘Why is this here?’ ”
Mixed-up practice doesn’t just build overall dexterity and prompt active discrimination. It helps prepare us for life’s curveballs, literal and figurative.