Archives For evaluate

Walking the Talk
Changing or improving the world one learner at a time is not an end but a process. Once you start this process it requires continual effort. Fortunately, it has a compounding effect, and small but consistent efforts will maintain and grow its impact.

I have been endeavoring to walk the talk in all aspects of my life for many decades. Whether it was the technology-related courses I developed in the early ’90s, my move into online learning at the University of Alberta in 1995, my related research into web-based instruction that went into my doctoral dissertation, or the dozens of online and blended learning courses that I have developed since then, I have always strived to apply all the theories, approaches and principles that I advocate in my own practice. A continual and iterative process of analysis and evaluation that leads to synthesis and creation is simply part of the learner’s mindset and is what I have been using in the development of courses for the new Applied Digital Learning (ADL) Master which is an update or revision of the Masters of Digital Learning and Leading (DLL) which I co-developed at Lamar University in 2014-2015. One of my goals for revising the DLL courses and converting them to ADL courses has been to strip away unnecessary information, activities, processes, or anything else that will impede the learner’s choice, ownership, and voice as they create their innovation plans and then create significant learning environment where their plans can be realized. Another goal is to point to how key foundational concepts like Applied Learning, Assessment Of/For/As Learning, Connecting the Dots Vs Collecting, the Dots, Change of Focus, CLSE, COVA, and the Learner’s Mindset will play a role in their growth and development.

Desiring the Quick Fix
Over the past several years my colleagues and I have observed and analyzed the tendency for many educators who come into the DLL program to look for a quick fix to innovation. They generally see technology as a tool that has a hyped potential for a quick fix so they often expect to be told what procedure, process, or product they should implement as part of their innovation plan. While the application or applying of a model or simulation may initially appear to be what they want when they try to apply these products or processes to their circumstance, they quickly find that this often doesn’t work. The educational technology (Edtech) literature for the past several decades is filled with examples of how the application of technology in a learning setting makes no significant difference and has little impact on learning outcomes and that the focus needs to be the learning, not the technology if we want to make a difference (Reich, 2020; Cuban, 2001; Russel, 1999; Wenglinsky, 1998). The research is clear. Edtech is not a quick fix or silver bullet (Thibodeaux, Harapnuik, Cummings, & Wooten, 2017) and the naive notion that one can implement it better than the last group that failed is continually repeated in all levels of classrooms across the nation. This is why the SAMR model or other quick fixes do not work (Harapnuik, 2017). Why does this thinking continually persist?

Start with a Change in Thinking about your Thinking
In analyzing this situation I looked at what type of thinking or level of thinking that was being used. According to the revised Bloom’s taxonomy when people are attempting to carry out a procedure or implement a process or apply an existing model to a new but similar situation they are using lower ordering thinking in their hopes of applying existing information to their situation. The category of Applying is at the top of the lower order thinking within Bloom’s taxonomy but it is still considered lower-order thinking and only facilitates information transfer because there is no analysis, evaluation, or creation which are at the higher-order and are essential to deeper learning.

Inverted Bloom's Taxonomy

The inverted Bloom’s taxonomy is one of my favorite perspectives because it combines the higher-order thinking into a continuum and reveals that analyzing, evaluating, and creating must be conducted in conjunction. In the Applying section, the notion of using information in a new but similar situation seems to fit what I was seeing in many of my students in the DLL program.

Because I practice what I preach, I continued to analyze and evaluate my ideas further before I created this post. I also went to the original or primary sources of the Bloom taxonomy (Anderson & Krathwohl, 2001) to make sure I wasn’t missing any key information in the dozens of abbreviated versions of Bloom’s taxonomy that a quick Google search revealed. While the summary in the inverted Bloom taxonomy was useful and it was compatible with many other summaries of Bloom’s taxonomy, you will find online the original perspective from Anderson and Krathwohl (2001) was much more effective in addressing my concerns.

Applying: Carrying out or using a procedure through executing or implementing. Applying is related and refers to situations where learned material is used through products like models, presentation, interviews, and simulations.

This summary confirms what I have seen with many students who just want to be told what procedure or model they need to execute or implement and believe that all they have to present an existing model to their colleagues, simulate the new approach, and their innovation process is complete. To be fair to many of these students, this is what they know and simply what their administrators, schools, districts, or other organizations ask them to do. This ongoing process of identifying a standard to be met, finding the approved or accepted procedure or process being used in the organization to meet this standard, and finally applying a standardized test or other information transfer confirmation tools to confirm that the standard has been met by the students is the norm. For rudimentary knowledge, simple situations, and information transfer this application process does work well and our education system has been relying on this model for over a century. As we move further into the digital information age we are realizing that our challenges are much more complex and require much more than doing what we have done in the past. To address these more significant challenges we need to move beyond applying existing information or processes in a new but similar fashion.

Moving to Higher Order Thinking
We need to move into analyzing, evaluating, and creating new solutions to ever-increasing challenges that we and our learners will face in the future. We also need to look beyond convenient summaries, quick fixes, or “Coles Notes” solutions and go back to primary sources to get the full picture. If we want to address the ever-increasing complexities in the challenges we face in the 21st Century then we must use higher-order thinking. We must continually analyze and evaluate what we are doing as we begin creating innovations that will enhance learning. Anderson and Krathwohl (2001) explanation of the following three higher thinking levels offers the best starting point for our own analysis, evaluation, and creation of a novel way of integrating these ideas.

Analyzing: Breaking material or concepts into parts, determining how the parts relate or interrelate to one another or to an overall structure or purpose. Mental actions include differentiating, organizing, and attributing as well as being able to distinguish between components.

Evaluating: Making judgments based on criteria and standards through checking and critiquing.

Creating – Putting elements together to form a novel coherent whole or make an original product.

Analyzing, Evaluating, and Creating Lead to Deeper Learning and Learner’s Mindset
While the inverted Bloom’s taxonomy is useful for helping us to see the linear relationship between analyzing, evaluating, and creating and also see how the higher-order thinking is separated from lower-order thinking, it doesn’t convey the interrelatedness between analyzing, evaluating, and creating. It also doesn’t show how the interrelation between analyzing, evaluating, and creating contributes to deeper learning.
Analyze-Evaluate-Create-Deeper-Learning

The Venn diagram (Harapnuik, 2021) reveals how analyzing, evaluating, and creating come together and at that convergence point is where the learner engages in deeper learning and has then moved into the Learner’s Mindset.

This deeper learning and the adoption of a Learner’s Mindset is realized when you create a significant learning environment in which you give your learner’s choice, ownership, and voice through authentic learning opportunities (CSLE+COVA). I have been applying this approach in all the learning environments that I have created and most recently have applied this to the DLL and ADL programs as well as in the Provincial Instructor Diploma Program (PIDP).

In my grandiose goal of changing the world one learner at a time, I am sharing this approach with as many people that I can. It is my hope that you too will begin the ongoing process of analysis, evaluation, and creation. Through a continual and iterative process of analysis of your learning environment, the new concepts, theories, and ideas you are exploring combined with your goal of bringing learning innovation to your organization, you too can begin to explore and evaluate how best to synthesis your findings and ideas into an innovation plan which will create the changes you desire and prepare your learners for life.

Please remember that this is only one part of a bigger picture so explore the following:

Applied Learning
Assessment Of/For/As Learning
Connecting the Dots Vs Collecting, the Dots
Change of Focus
CLSE
COVA
Feedforward
Learner’s Mindset

References

Anderson, L. W., Krathwohl, D., & Bloom, B. S. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.

Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching and assessing: A Revision of bloom’s taxonomy of educational objectives (Abridge Edition). Addison.

Cuban, L. (2001). Oversold and underused: Computers in the classroom. Harvard university press.

Harapnuik, D.K. (2021). Analyze-evaluate-create-deeper-learning-cropped.png. [Image] https://www.harapnuik.org/wp-content/uploads/2021/01/Analyze-Evaluate-Create-Deeper-Learning-cropped.png

Harapnuik, D.K. (2017). Reconsider the use of the SAMR model. [Blog]. Retrieved from https://www.harapnuik.org/?p=7235

Reich, J. (2020). Failure to disrupt: Why technology alone can’t transform education. Harvard University Press.

Russell, T. L. (1999). The no significant difference phenomenon: A comparative research annotated bibliography on technology for distance education: As reported in 355 research reports, summaries and papers. North Carolina State University.

Thibodeaux, T. N., Harapnuik, D. K., Cummings, C. D., & Wooten, R. (2017). Learning all the time and everywhere: Moving beyond the hype of the mobile learning quick fix. In Keengwe, J. S. (Eds.). Handbook of research on mobile technology, constructivism, and meaningful learning. Hershey, PA: IGI Global.

Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics. ETS Policy Information Center. https://www.ets.org/Media/Research/pdf/PICTECHNOLOG.pdf