Back in February of 2018 I posted Reading, Watching, & Listening where I finally completed a list of what I am reading, watching, and listening to in response to the questions I get from many of my students, friends, and colleagues about where I find my ideas. I promised to update this list on a regular basis and I am finally getting around to that promise. As part of that promise I am moving the contents of the 2018 blog post to my Reading, Watching, & Listening page which, I will continue to make this a permanent part of the Learning section of my website. I will keep the 2018 blog post up to serve as an archive and to see how my selection of resources has changed over time.
Archives For Research
If you ever wondered how scarce educational dollars are all too often wasted on foolish technology purchases all you need to do is continue to not look at the research. As soon as you read above statement you immediately thought I made a mistake and should have stated that all you need to do is look at the research. You would be right from a grammatical perspective that I meant to say “look at the research” but the emphasis I am trying to make is that there is overwhelming evidence that almost 90% of educational leaders are making technology purchases without looking at the research.
Dr. Michael Kennedy, an associate professor at the University of Virginia along with a team of thirteen researchers at the Edtech Research Efficacy Symposium in 2017 asked 515 educational leaders from 17 states the following question:
When making purchasing and/or adoption decisions regarding a new technology-based product (assume for academic instruction) for your district or school, how important is the existence of peer-reviewed research to back the product?
It is important to note that the survey participants were categorized as, 24 percent district technology supervisors, 22 percent assistant superintendents, 7 percent superintendents, 27 percent teachers, and 10 percent principals. Equally important is that 75 percent of this group were directly responsible for EdTech purchases for their school or were consulted on purchase decisions.
Only 11 percent of these decision-makers listed research being in place to confirm the efficacy of the product they were planning to purchase. In contrast what was rated “extremely important” or “very important” were the prioritized factors of ‘fit’ for their school, price, functionality, and alignment with district initiatives. This should be extremely concerning to parents, most teachers, and taxpayers who are funding our educational system. If technology supervisors, superintendents, assistant superintendents, principals, and select teachers are making decisions on what technology to purchase that doesn’t include support for its effectiveness in the learning environment then they should be held accountable when it doesn’t work.
Unfortunately, the research which so many of these leaders are willing to ignore shows that that technology all too often doesn’t make a difference or worse it can hinder learning:
…students who use computers very frequently at school do a lot worse in most learning outcomes, even after accounting for social background and student demographics (OECD 2015, p. 5).
This not new research. On the contrary, we have known for a very long time that just adding technology to the classroom does not have any significant impact on learning. In the early 1990’s Thomas Russell and several other researchers pointed to the results of a meta-analysis of the research into technology use in distance education and found that there is no difference between technology-based instruction or classroom instruction (1999).
I must be very clear that I am not opposed to using computers, smartphones, tablets or any other technology to enhance learning. I would challenge you to find a bigger user and proponent of the effective use of technology to enhance learning. The key is to focus on the learning first and then look to technology to further enhance and amplify the learning environment and the learning experience. We also have to be careful that we don’t just try to bolt technology onto an antiquated classroom that emphasizes the 19th-century information transfer model of standardized curriculum and testing which research as also shows doesn’t enhance learning. That OECD (2015) report I referred to earlier also suggest that:
Technology can support new pedagogies that focus on learners as active participants with tools for inquiry-based pedagogies and collaborative workspaces. For example, technology can enhance experiential learning, foster project-based and inquiry-based learning pedagogies, facilitate hands-on activities and cooperative learning (p. 6)
The OECD (2015) report also pointed to John Hattie’s research into what contributes to student achievement and confirms that:
Computers were more effective when they are used to extend study time and practice, used to give students control over the learning situation (pacing of material) and when used to support collaborative learning (p. 163).
Technology is a potentially powerful tool that should be used to enhance creation, collaboration, inquiry, investigation, communication. Ideally we want to give our learners choice ownership, and voice through authentic learning opportunities. Technology helps us to do this more effectively. We need to make our technology purchases based on research and the research shows that we aren’t doing this.
The fact that most of our educational leaders are making technology purchases based on price, fit, or other factors that support their confirmation bias we should not be surprised when the Ed Tech industry ignores the research. They don’t need to support their product claims with evidence or research because it is clear that they can sell their products without it. All they need to do is have a well-tuned sales pitch and a good salesperson that will give these educational leaders just what they think is important or what want.
Are you part of the 11 percent that is using research to make informed decisions about your technology purchases that will enhance learning. Or are part of 89 percent that is ignoring the research and potentially ignoring the learning?
References
Kennedy, M. (2017) Role of federal funding and research findings on adoption and implementation of technology-based products and tools. Edtech Research Efficacy Symposium. Retrieved from http://symposium.curry.virginia.edu//wp-content/uploads/2017/07/The-Goals-and-Roles-of-Federal-Funding-for-EdTech-Research_FINAL-1.pdf
OECD (2015), Students, Computers and Learning: Making the Connection, OECD Publishing, Paris. http://dx.doi.org/10.1787/9789264239555-en
Russell, T. L. (1999). The no significant difference phenomenon: A comparative research annotated bibliography on technology for distance education: As reported in 355 research reports, summaries and papers. North Carolina State University.
The other morning a colleague sent me the tweet from Daniel Pink “The secret to learning is overlearning…” which pointed to Cari Romm’s (2017) New York Magazine post To Truly Learn Something, Study Until You’ve Mastered It — and Then Keep Studying. Since I am a learning theorist and am always searching for “the secret to learning” and since Pink’s book Drive: The Surprising Truth About What Motivates Us is still one of my favorite books on motivation I started down the rabbit trail by reviewing Romm’s article in the Science of Us section of the blog. The reason I used the terms “started down the rabbit trail” and also used “reviewed” rather the “read” is very significant because in order to get to the truth or the actual facts about what these various people were stating I ended up looking at several other magazine/blog posts and then a few journal articles and went back to a couple of books to get the full story and really see what the facts are. While I am referring to this process of going down the rabbit trail what I am really referring to is simply doing the due diligence of analytical thinking and getting to the facts by going back to the primary sources to see what is really being said. Let me explain why this is so important and why we need to encourage everyone to verify what is being said and written.
Getting back to Romm’s post about the secret to learning, the headline alone would suggest that the article is about learning. Romm’s opening statement also points to and questions deliberate practice and elite performance:
…On its own, deliberate practice isn’t enough to turn you into an elite performer, whether you’re talking about boosting your athletic prowess or learning to play the violin.
Since I have been studying Anders Ericsson’s research into deliberate practice for many years I was intrigued by this opening statement and immediately reviewed the short post and followed the link to the post that Romm had pointed to in her opening. Before I deal with this second stop on the rabbit trail I need to explain that Romm’s generalization did not line up with the findings of the article Overlearning hyperstabilizes a skill by rapidly making neurochemical processing inhibitory-dominant in Nature Neuroscience (2017) she referred to and while she used the term learning what the article was referring to was actually training and memorization. This is where we get into a problem that can be resolved with clearly defining terms. How is the term “learning” really being used?
The learning that the authors of the Nature article referred to was in the context of a learned response to a stimulus. They also referred to training and memorization and their primary conclusion was that after a training event or session the learned stimulus-response needs to be stabilized or reinforced in order to prevent it from being disrupted by a new learned response. To prevent this loss you need to spend a minimum of 20 more minutes after you have reached the training plateau to reinforce the effect of training – which they referred to as overlearning. The authors of the Nature article (2017) were researching how people responded to a visual-recognition task by asking the participants to identify patterns in images and then measured the concentrations of excitatory and inhibitory neuro-transmitter levels in the visual areas of the brain. In a nutshell these researchers have identified the biological reaction in the brain that reinforces a conditioned response by increasing the excitatory neuro-transmitters and they have generalized that overlearning rapidly and strongly hyperstabilizes this biological reaction (Shibata et al., 2017). While they have also gone as far as to generalize that overlearning will help you retain your training or memorization they do qualify that their work has only gone as far as exploring this within the visual context.
While there are elements of data to support Romm’s headline the generalization – to truly learn something you need to study until you master it and then keep on studying is not correct. A more accurate claim would be – to truly memorize something you need to study until you master it and then keep on studying. There is a big difference between memorizing something and learning something. Learning is making meaningful connections by connecting new information or ideas with existing information or ideas to come to know something new. While memorization plays a role in the learning process it is only part of the process and all too often is used by people to simply regurgitate information. Richard Feynman (2014) reminds us that there is a difference between knowing the name of something and knowing something.
I firmly believe that Romm has no malicious intent in or intentionally wanted to mislead people but this where it is our responsibility to analyze and assess the author’s argument for validity. While there are elements of truth in her generalization of benefits of studying the assertions of the opening statement “deliberate practice isn’t enough” are actually misleading and unfortunately false.
This leads us further down the rabbit trail. Rather than look at primary sources and Anders Ericsson’s actual research into deliberate practice Romm points and links to another Science of Us blog post 10,000 Hours of Deliberate Practice Aren’t Going to Get You Olympic Gold by Drake Baer (2017). Unfortunately for Romm, Baer isn’t much better at research and doesn’t bother by going to the primary sources either to really find out the what he is referring to as deliberate practice or the 10,000-hour rule. Baer mistakenly points to Galdwell’s book Outliers and suggests that deliberate practice is simply a matter of putting in 10,000 of hard work and links to his own 2013 FastCompany article that confirms that you just need to put in the time. To be fair to Baer he does suggest in that you need to work on the hard parts to get better but only refers to a BrainPickings article (Popova, 2013) rather than a primary source. To add an appeal to authority Baer also points to the Karl Smith’s (2016) Scientific America blog post No One Wins Gold for Practicing the Most which also gets it wrong. While Smith does point to Macnamara, Hambrick, and Oswald’s (2014) research article Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis in Psychological Science to support his argument I am relatively certain he hasn’t read the full article or even looked at Ericsson’s original research or most recent work because he incorrectly defines deliberate practice and overemphasizes the 10,000 hour aspect.
If you look at Ericsson’s research or his latest book Peak: Secrets from the new science of expertise (2016) which summarizes all his work on deliberate practice you will find that the 10,000-hour rule that Gladwell popularized is actually false. Ericsson (2016) actually stated that depending on the discipline and various other factors a rudimentary level of expertise could be reached after one put in 7500 – 15,000 hours. This is a big range and Gladwell generalized this idea by simply picking the 10,000 point because it would be easier to remember. Ericsson (2016) also points out that this is just the starting point of expertise and many world-class performers have put in more than 20,000 more hours to be the best. The 10,000-hour rule is not a rule but a popular myth and authors like Romm and Baer mistakenly refer to this myth.
But there is an even bigger problem with the arguments of bloggers Romm, Baer, Smith and the researchers Macnamara, Hambrick, and Oswald. They all define deliberate practice incorrectly. Deliberate practice is not just putting in the time or working harder or pushing oneself further, nor is it just using structured practice. Ericsson (2016) is quite clear when he states that just practicing countless hour after hour, in the same way, will not help one improve and in many instances this repetitive practice, even if it has some form of structure, can potentially degrade performance. It is not the total hours or the fact that there is a structure that matters it is how one practices in those hours and what that structure is that matters.
According to Ericsson (2016) deliberate practice involves the following four components:
- Goals – you have to have a clear vision of what you are working toward or hope to accomplish. Watching or visualizing the activity performed perfectly either in a video of yourself or another expert will help you get to your goals.
- Focus – you have to break down the activity into smaller chunks and slow down the process to get a higher degree of control and precision. Paradoxically you have to slow down to get smooth enough before you can get faster and better.
- Feedback – you have to analyze your performance and look for ways to improve. Most experts have learned to continually error correct and look at and analyze what they are doing with an eye to continuous improvement. This is where coaches, good teachers and even video recordings of your performance come into play. Most novices will require a coach to provide the necessary feedback because they often don’t even know what they need to improve. A cycle of feedback and continuous error correction is the key to deliberate practice
- Exit your comfort zone – you have to push yourself beyond your comfort zone order to make improvements. The key is to push just enough to be slightly uncomfortable but not so much that you will fail immediately. Experts have learned what that 3-4 % improvement feels like and to know when they are going to far out of their comfort zone to reach new levels of performance.
Ericsson also points out that experts have a deep set of mental representations of their discipline that make it easy for them to do things that look magical to the average person. Experts have done the mental reps that give them the highest levels of mental representation that enable them to operate at the highest level. This is a combination of mental and physical training at the highest level and is much more than working or practicing hard for 10,000 hours. So at this point in the rabbit trail, I hope one can see that these first few authors really shouldn’t be trusted. It appears that the bloggers Romm and Baer may be more interested in building their following with catchy headlines to promote their writing then they are with the facts. If we can’t trust these authors then who can we trust—the academics? Smith is a Ph.D. candidate who published in the Scientific America blog and the researchers Macnamara, Hambrick, and Oswald are publishing in peer-reviewed journals but can we trust their findings just based their credentials and a perceived higher quality of the publication. Unfortunately, not. Smith didn’t bother looking at the primary sources to get a clear definition of deliberate practice and was too willing to simply run with the notion that deliberate practice involves harder work. If you compare the notion of harder work with the 4 components of deliberate practice listed above it is clear that deliberate practice is much more than hard work.
When you review the work of Macnamara, Hambrick, and Oswald (2014) you will find that they have gone to the primary sources but unfortunately, you will also see that they define deliberate practice as
engagement in structured activities created specifically to improve performance in a domain. (p. 1608)
which is not an accurate definition to use in their meta-analysis. Such a vague definition of deliberate practice not only cast doubts on the authors’ findings that deliberate practice only explained 26% of the variance in performance for games, 21% for music, 18% for sports it calls into question their entire research. The key to deliberate practice is the details of the purposeful goals, focus, feedback while pushing the limits. This is much more than just structure. While Macnamara, Hambrick, and Oswald do confirm that deliberate practice is still important they posit that it is not as important Ericsson argues.
This is where one has to be careful in examining the data, the research methods and exactly what the researchers are looking for. While I have stated earlier that I am calling their findings into question I will also state that it appears that their research is accurate. Let me explain, if you use a very loose definition of deliberate practice and simply point to structured activity then you will get the results that they point to. This is what they found in their meta-analysis. However, if you use the authentic definition of deliberate practice from the primary sources I would argue that there would be a very different result. Macnamara, Hambrick, and Oswald did a very thorough job on some aspects of their research like their methods, their analysis, and coding of the information but their research question was based on an inaccurate or overly broad definition of deliberate practice (Ericsson, 2016b).
So at this point of the rabbit trail, we find out that many of the claims made by numerous authors are simply wrong because they did not go to the original sources and kept referring to other authors who also failed to go to the original sources. And when we finally found some authors who did go to the original sources their claims could not be trusted either because these authors did not use the same definitions that the original authors used.
Who can you trust? Trust yourself. We have the responsibility to verify what we read by reading critically and thinking analytically while looking at the evidence. While I referred to this process of going down the rabbit trail it really is just a matter of seeing if an author has supported what they are saying and can corroborate their statements with external sources. Ideally, the external source should be primary sources. There will always be differences of opinions and biases but if you are objective enough and can look at the facts you should be able to discern what is accurate regardless of your bias. Admitting your bias is also is a good way of assuring your reader that you are attempting to be objective—we all have biases.
In summary, the tweet the other morning led to the above explanation and the examination of the following sources, and the following conclusions. Contrary to the errant claims of several authors who demonstrated very poor research skills the actual facts show:
- deliberate practice will help you become an elite performer,
- overlearning is great for memorization but memorization itself shouldn’t be mistaken for learning,
- the 10,000-hour rule isn’t a rule but a pop culture myth and an interesting rap song.
- accurate definition of terms is crucial to valid and reliable research.
This whole process took much more time than I had hoped or expected but if you really want to know then you have to do the due diligence and look at all the facts. There is no a quick fix. The most efficient way is to go back to the primary sources and see what is really being claimed. In the information age, there is an abundance or overload of available information so the need to do this is greater than ever before. Anyone can put anything up on the Internet so we have to be even more diligent than ever before. Unfortunately, the notion of trusted sources is something that we cannot rely on upon anymore, at least not completely. I will go as far as to suggest that there are some sources that I may be more inclined to initially trust but I would still verify. Stating what those sources are is a whole other argument and post. There are just far too many examples of faulty research being exposed and if you consider my example above, all it takes is a definition of terms to be ignored and the results of the research will be inaccurate. Furthermore, we need to be willing to heed the warnings of Chuck Klosterman (2016) who asks the question What If We’re Wrong? If we look how our understanding of science and the world around has progressed in the last several centuries then we should be willing to admit that there may be some things that we hold to be true today that may be false 10, 20, 50 or more years into the future.
We do live in the most amazing time to be a learner. All the world’s information is available to us in the palms of our hands. Because so much information is available we must not only be prepared but be willing to take the time that it takes to critically and analytically assess all the information we are taking in.
References
Baer, D. (2013, October 29). Why “Deliberate Practice” is the only way to keep getting better [Magazine]. Retrieved June 9, 2017, from https://www.fastcompany.com/3020758/leadership-now/why-deliberate-practice-is-the-only-way-to-keep-getting-better
Baer, D. (2016, August 8). 10,000 hours of deliberate practice aren’t going to get you Olympic gold [Blog]. Retrieved June 9, 2017, from http://nymag.com/scienceofus/2016/08/deliberate-practice-isnt-going-to-get-you-olympic-gold.html
Ericsson, A., & Pool, R. (2016). Peak: Secrets from the new science of expertise. New York, NY: Eamon Dolan/Houghton Mifflin Harcourt.
Ericsson, K. A. (2016). Summing up hours of any type of practice versus identifying optimal practice activities: Commentary on Macnamara, Moreau, & Hambrick (2016). Perspectives on Psychological Science, 11(3), 351–354.
Klosterman, C. (2016). But what if we’re wrong?: Thinking about the present as if it were the past. New York, NY: Blue Rider Press.
Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014). Deliberate practice and performance in music, games, sports, education, and professions a meta-analysis. Psychological Science, 25(8), 1608–1618.
Popova, M. (2013, October 17). The psychology of getting unstuck: How to overcome the “OK Plateau” of performance & personal growth. Retrieved June 9, 2017, from https://www.brainpickings.org/2013/10/17/ok-plateau/
Richard Feynman : Knowing the Name of Something. (2014). [Video file] Retrieved from https://youtu.be/lFIYKmos3-s
Romm, C. (2017, January 31). To truly learn something, study until you’ve mastered It — and then keep going [Blog]. Retrieved June 9, 2017, from http://nymag.com/scienceofus/2017/01/to-truly-learn-something-study-way-more-than-you-need-to.html
Shibata, K., Sasaki, Y., Bang, J. W., Walsh, E. G., Machizawa, M. G., Tamaki, M., … Watanabe, T. (2017). Overlearning hyperstabilizes a skill by rapidly making neurochemical processing inhibitory-dominant. Nature Neuroscience, 20(3), 470–475.
Smith, K., J. (2016, August 5). No one wins gold for practicing the most. Retrieved June 9, 2017, from https://www.scientificamerican.com/article/no-one-wins-gold-for-practicing-the-most/
