To Get the Real Story You Need to Go to Primary Sources

The other morning a colleague sent me the tweet from Daniel Pink “The secret to learning is overlearning…” which pointed to Cari Romm’s (2017) New York Magazine post To Truly Learn Something, Study Until You’ve Mastered It — and Then Keep Studying. Since I am a learning theorist and am always searching for “the secret to learning” and since Pink’s book Drive: The Surprising Truth About What Motivates Us is still one of my favorite books on motivation I started down the rabbit trail by reviewing Romm’s article in the Science of Us section of the blog. The reason I used the terms “started down the rabbit trail” and also used “reviewed” rather the “read” is very significant because in order to get to the truth or the actual facts about what these various people were stating I ended up looking at several other magazine/blog posts and then a few journal articles and went back to a couple of books to get the full story and really see what the facts are. While I am referring to this process of going down the rabbit trail what I am really referring to is simply doing the due diligence of analytical thinking and getting to the facts by going back to the primary sources to see what is really being said. Let me explain why this is so important and why we need to encourage everyone to verify what is being said and written.

Getting back to Romm’s post about the secret to learning, the headline alone would suggest that the article is about learning. Romm’s opening statement also points to and questions deliberate practice and elite performance:

…On its own, deliberate practice isn’t enough to turn you into an elite performer, whether you’re talking about boosting your athletic prowess or learning to play the violin.

Since I have been studying Anders Ericsson’s research into deliberate practice for many years I was intrigued by this opening statement and immediately reviewed the short post and followed the link to the post that Romm had pointed to in her opening. Before I deal with this second stop on the rabbit trail I need to explain that Romm’s generalization did not line up with the findings of the article Overlearning hyperstabilizes a skill by rapidly making neurochemical processing inhibitory-dominant in Nature Neuroscience (2017) she referred to and while she used the term learning what the article was referring to was actually training and memorization. This is where we get into a problem that can be resolved with clearly defining terms. How is the term “learning” really being used?

The learning that the authors of the Nature article referred to was in the context of a learned response to a stimulus. They also referred to training and memorization and their primary conclusion was that after a training event or session the learned stimulus-response needs to be stabilized or reinforced in order to prevent it from being disrupted by a new learned response. To prevent this loss you need to spend a minimum of 20 more minutes after you have reached the training plateau to reinforce the effect of training – which they referred to as overlearning. The authors of the Nature article (2017) were researching how people responded to a visual-recognition task by asking the participants to identify patterns in images and then measured the concentrations of excitatory and inhibitory neuro-transmitter levels in the visual areas of the brain. In a nutshell these researchers have identified the biological reaction in the brain that reinforces a conditioned response by increasing the excitatory neuro-transmitters and they have generalized that overlearning rapidly and strongly hyperstabilizes this biological reaction (Shibata et al., 2017). While they have also gone as far as to generalize that overlearning will help you retain your training or memorization they do qualify that their work has only gone as far as exploring this within the visual context.

While there are elements of data to support Romm’s headline the generalization – to truly learn something you need to study until you master it and then keep on studying is not correct. A more accurate claim would be – to truly memorize something you need to study until you master it and then keep on studying. There is a big difference between memorizing something and learning something. Learning is making meaningful connections by connecting new information or ideas with existing information or ideas to come to know something new. While memorization plays a role in the learning process it is only part of the process and all too often is used by people to simply regurgitate information. Richard Feynman (2014) reminds us that there is a difference between knowing the name of something and knowing something.

I firmly believe that Romm has no malicious intent in or intentionally wanted to mislead people but this where it is our responsibility to analyze and assess the author’s argument for validity. While there are elements of truth in her generalization of benefits of studying the assertions of the opening statement “deliberate practice isn’t enough” are actually misleading and unfortunately false.

This leads us further down the rabbit trail. Rather than look at primary sources and Anders Ericsson’s actual research into deliberate practice Romm points and links to another Science of Us blog post 10,000 Hours of Deliberate Practice Aren’t Going to Get You Olympic Gold by Drake Baer (2017). Unfortunately for Romm, Baer isn’t much better at research and doesn’t bother by going to the primary sources either to really find out the what he is referring to as deliberate practice or the 10,000-hour rule. Baer mistakenly points to Galdwell’s book Outliers and suggests that deliberate practice is simply a matter of putting in 10,000 of hard work and links to his own 2013 FastCompany article that confirms that you just need to put in the time. To be fair to Baer he does suggest in that you need to work on the hard parts to get better but only refers to a BrainPickings article (Popova, 2013) rather than a primary source. To add an appeal to authority Baer also points to the Karl Smith’s (2016) Scientific America blog post No One Wins Gold for Practicing the Most which also gets it wrong. While Smith does point to Macnamara, Hambrick, and Oswald’s (2014) research article Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis in Psychological Science to support his argument I am relatively certain he hasn’t read the full article or even looked at Ericsson’s original research or most recent work because he incorrectly defines deliberate practice and overemphasizes the 10,000 hour aspect.

If you look at Ericsson’s research or his latest book Peak: Secrets from the new science of expertise (2016) which summarizes all his work on deliberate practice you will find that the 10,000-hour rule that Gladwell popularized is actually false. Ericsson (2016) actually stated that depending on the discipline and various other factors a rudimentary level of expertise could be reached after one put in 7500 – 15,000 hours. This is a big range and Gladwell generalized this idea by simply picking the 10,000 point because it would be easier to remember. Ericsson (2016) also points out that this is just the starting point of expertise and many world-class performers have put in more than 20,000 more hours to be the best. The 10,000-hour rule is not a rule but a popular myth and authors like Romm and Baer mistakenly refer to this myth.

But there is an even bigger problem with the arguments of bloggers Romm, Baer, Smith and the researchers Macnamara, Hambrick, and Oswald. They all define deliberate practice incorrectly. Deliberate practice is not just putting in the time or working harder or pushing oneself further, nor is it just using structured practice. Ericsson (2016) is quite clear when he states that just practicing countless hour after hour, in the same way, will not help one improve and in many instances this repetitive practice, even if it has some form of structure, can potentially degrade performance. It is not the total hours or the fact that there is a structure that matters it is how one practices in those hours and what that structure is that matters.
According to Ericsson (2016) deliberate practice involves the following four components:

1. Goals – you have to have a clear vision of what you are working toward or hope to accomplish. Watching or visualizing the activity performed perfectly either in a video of yourself or another expert will help you get to your goals.
2. Focus – you have to break down the activity into smaller chunks and slow down the process to get a higher degree of control and precision. Paradoxically you have to slow down to get smooth enough before you can get faster and better.
3. Feedback – you have to analyze your performance and look for ways to improve. Most experts have learned to continually error correct and look at and analyze what they are doing with an eye to continuous improvement. This is where coaches, good teachers and even video recordings of your performance come into play. Most novices will require a coach to provide the necessary feedback because they often don’t even know what they need to improve. A cycle of feedback and continuous error correction is the key to deliberate practice
4. Exit your comfort zone – you have to push yourself beyond your comfort zone order to make improvements. The key is to push just enough to be slightly uncomfortable but not so much that you will fail immediately. Experts have learned what that 3-4 % improvement feels like and to know when they are going to far out of their comfort zone to reach new levels of performance.

Ericsson also points out that experts have a deep set of mental representations of their discipline that make it easy for them to do things that look magical to the average person. Experts have done the mental reps that give them the highest levels of mental representation that enable them to operate at the highest level. This is a combination of mental and physical training at the highest level and is much more than working or practicing hard for 10,000 hours. So at this point in the rabbit trail, I hope one can see that these first few authors really shouldn’t be trusted. It appears that the bloggers Romm and Baer may be more interested in building their following with catchy headlines to promote their writing then they are with the facts. If we can’t trust these authors then who can we trust—the academics? Smith is a Ph.D. candidate who published in the Scientific America blog and the researchers Macnamara, Hambrick, and Oswald are publishing in peer-reviewed journals but can we trust their findings just based their credentials and a perceived higher quality of the publication. Unfortunately, not. Smith didn’t bother looking at the primary sources to get a clear definition of deliberate practice and was too willing to simply run with the notion that deliberate practice involves harder work. If you compare the notion of harder work with the 4 components of deliberate practice listed above it is clear that deliberate practice is much more than hard work.

When you review the work of Macnamara, Hambrick, and Oswald (2014) you will find that they have gone to the primary sources but unfortunately, you will also see that they define deliberate practice as

engagement in structured activities created specifically to improve performance in a domain. (p. 1608)

which is not an accurate definition to use in their meta-analysis. Such a vague definition of deliberate practice not only cast doubts on the authors’ findings that deliberate practice only explained 26% of the variance in performance for games, 21% for music, 18% for sports it calls into question their entire research. The key to deliberate practice is the details of the purposeful goals, focus, feedback while pushing the limits. This is much more than just structure. While Macnamara, Hambrick, and Oswald do confirm that deliberate practice is still important they posit that it is not as important Ericsson argues.

This is where one has to be careful in examining the data, the research methods and exactly what the researchers are looking for. While I have stated earlier that I am calling their findings into question I will also state that it appears that their research is accurate. Let me explain, if you use a very loose definition of deliberate practice and simply point to structured activity then you will get the results that they point to. This is what they found in their meta-analysis. However, if you use the authentic definition of deliberate practice from the primary sources I would argue that there would be a very different result. Macnamara, Hambrick, and Oswald did a very thorough job on some aspects of their research like their methods, their analysis, and coding of the information but their research question was based on an inaccurate or overly broad definition of deliberate practice (Ericsson, 2016b).

So at this point of the rabbit trail, we find out that many of the claims made by numerous authors are simply wrong because they did not go to the original sources and kept referring to other authors who also failed to go to the original sources. And when we finally found some authors who did go to the original sources their claims could not be trusted either because these authors did not use the same definitions that the original authors used.

Who can you trust? Trust yourself. We have the responsibility to verify what we read by reading critically and thinking analytically while looking at the evidence. While I referred to this process of going down the rabbit trail it really is just a matter of seeing if an author has supported what they are saying and can corroborate their statements with external sources. Ideally, the external source should be primary sources. There will always be differences of opinions and biases but if you are objective enough and can look at the facts you should be able to discern what is accurate regardless of your bias. Admitting your bias is also is a good way of assuring your reader that you are attempting to be objective—we all have biases.

In summary, the tweet the other morning led to the above explanation and the examination of the following sources, and the following conclusions. Contrary to the errant claims of several authors who demonstrated very poor research skills the actual facts show:

• deliberate practice will help you become an elite performer,
• overlearning is great for memorization but memorization itself shouldn’t be mistaken for learning,
• the 10,000-hour rule isn’t a rule but a pop culture myth and an interesting rap song.
• accurate definition of terms is crucial to valid and reliable research.

This whole process took much more time than I had hoped or expected but if you really want to know then you have to do the due diligence and look at all the facts. There is no a quick fix. The most efficient way is to go back to the primary sources and see what is really being claimed. In the information age, there is an abundance or overload of available information so the need to do this is greater than ever before. Anyone can put anything up on the Internet so we have to be even more diligent than ever before. Unfortunately, the notion of trusted sources is something that we cannot rely on upon anymore, at least not completely. I will go as far as to suggest that there are some sources that I may be more inclined to initially trust but I would still verify. Stating what those sources are is a whole other argument and post. There are just far too many examples of faulty research being exposed and if you consider my example above, all it takes is a definition of terms to be ignored and the results of the research will be inaccurate. Furthermore, we need to be willing to heed the warnings of Chuck Klosterman (2016) who asks the question What If We’re Wrong? If we look how our understanding of science and the world around has progressed in the last several centuries then we should be willing to admit that there may be some things that we hold to be true today that may be false 10, 20, 50 or more years into the future.

We do live in the most amazing time to be a learner. All the world’s information is available to us in the palms of our hands. Because so much information is available we must not only be prepared but be willing to take the time that it takes to critically and analytically assess all the information we are taking in.

References

Baer, D. (2013, October 29). Why “Deliberate Practice” is the only way to keep getting better [Magazine]. Retrieved June 9, 2017, from https://www.fastcompany.com/3020758/leadership-now/why-deliberate-practice-is-the-only-way-to-keep-getting-better

Baer, D. (2016, August 8). 10,000 hours of deliberate practice aren’t going to get you Olympic gold [Blog]. Retrieved June 9, 2017, from http://nymag.com/scienceofus/2016/08/deliberate-practice-isnt-going-to-get-you-olympic-gold.html

Ericsson, A., & Pool, R. (2016). Peak: Secrets from the new science of expertise. New York, NY: Eamon Dolan/Houghton Mifflin Harcourt.

Ericsson, K. A. (2016). Summing up hours of any type of practice versus identifying optimal practice activities: Commentary on Macnamara, Moreau, & Hambrick (2016). Perspectives on Psychological Science, 11(3), 351–354.

Klosterman, C. (2016). But what if we’re wrong?: Thinking about the present as if it were the past. New York, NY: Blue Rider Press.

Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014). Deliberate practice and performance in music, games, sports, education, and professions a meta-analysis. Psychological Science, 25(8), 1608–1618.

Popova, M. (2013, October 17). The psychology of getting unstuck: How to overcome the “OK Plateau” of performance & personal growth. Retrieved June 9, 2017, from https://www.brainpickings.org/2013/10/17/ok-plateau/

Richard Feynman : Knowing the Name of Something. (2014). [Video file] Retrieved from https://youtu.be/lFIYKmos3-s

Romm, C. (2017, January 31). To truly learn something, study until you’ve mastered It — and then keep going [Blog]. Retrieved June 9, 2017, from http://nymag.com/scienceofus/2017/01/to-truly-learn-something-study-way-more-than-you-need-to.html

Shibata, K., Sasaki, Y., Bang, J. W., Walsh, E. G., Machizawa, M. G., Tamaki, M., … Watanabe, T. (2017). Overlearning hyperstabilizes a skill by rapidly making neurochemical processing inhibitory-dominant. Nature Neuroscience, 20(3), 470–475.

Smith, K., J. (2016, August 5). No one wins gold for practicing the most. Retrieved June 9, 2017, from https://www.scientificamerican.com/article/no-one-wins-gold-for-practicing-the-most/