The following are the links to articles, blog posts, YouTube videos, TED Talks, and books that were used or referenced in the Train the Trainer course:
You will find this video, learning philosophy and links to many of my presentations, and my favorite blog posts on my blog About page.
The Head Won’t Go Where the Heart Hasn’t Been
Benjamin Bloom argued that we need to address all the domains and find a balance. We often overemphasize the cognitive domain, relegate the psychomotor to the trades or other overtly physical disciplines and limit the affective domain to ethical or values issues. This limitation will severely limit the change that is necessary for learning. The blog post The Head Won’t Go Where the Heart Hasn’t Been post includes a more detailed explanation of the importance of the affective domain and has links to the Behavioral Science (BS) Guys video How to Change People Who Don’t Want to Change and also a related TED Talk Why TED Talks don’t change people’s behaviors.
The Power of belief — mindset and success | Eduardo Briceno | TEDxManhattanBeach
The power of believing that you can improve | Carol Dweck
You will find a very useful Fixed vs Growth Mindset graphic and a short comparison of how the Fixed Vs Growth Mindset is equivalent to the Print Vs Digital Information Age on the blog post Fixed Vs Growth Mindset = Print Vs Digital Information Age
The Way of the Essentialist involves doing less, but better, so you can make the highest possible contribution. Greg McKeown’s book Essentialism: The Disciplined Pursuit of Less will help you understand how less can be more.
The following are the links to articles, blog posts, YouTube videos, TED Talks, and books that were used or referenced in the Train the Trainer and/or PIDP 3220 Delivery of Instruction course:
You will find this video, learning philosophy and links to many of my presentations, and my favorite blog posts on my blog About page.
The Head Won’t Go Where the Heart Hasn’t Been
Benjamin Bloom argued that we need to address all the domains and find a balance. We often overemphasize the cognitive domain, relegate the psychomotor to the trades or other overtly physical disciplines and limit the affective domain to ethical or values issues. This limitation will severely limit the change that is necessary for learning. The blog post The Head Won’t Go Where the Heart Hasn’t Been post includes a more detailed explanation of the importance of the affective domain and has links to the Behavioral Science (BS) Guys video How to Change People Who Don’t Want to Change and also a related TED Talk Why TED Talks don’t change people’s behaviors.
Fixed VS Growth Mindset
Growth Mindset vs. Fixed Mindset
The Power of belief — mindset and success | Eduardo Briceno | TEDxManhattanBeach
The power of believing that you can improve | Carol Dweck
You will find a very useful Fixed vs Growth Mindset graphic and a short comparison of how the Fixed Vs Growth Mindset is equivalent to the Print Vs Digital Information Age on the blog post Fixed Vs Growth Mindset = Print Vs Digital Information Age
The Way of the Essentialist involves doing less, but better, so you can make the highest possible contribution. Greg McKeown’s book Essentialism: The Disciplined Pursuit of Less will help you understand how less can be more.
One of the advantages of posting your ideas to your own journal, blog, or ePortfolio on an ongoing and long-term basis is that you can go back and evaluate and analyze your thinking and then continue to refine and synthesize your ideas. As new data or information comes to light or your research confirms or contradicts your hypothesis you can update your synthesis. The following is a synthesis from the post How to Change the World One Learner at a Time from January 2021 which is an update to a 2015 post Changing the world, one learner at a time as well as many other ideas that I have posted over the years. The higher-order thinking that I referred to in the Owning Your Learning Process video above is also a key function of the Learner’s Mindset which is achieved by a change in thinking about learning, a change in the approach to learning, and a change in the learning environment.
The change in thinking that I refer to requires a move away from lower-order thinking that dominates our society and results in the desire for a quick fix to all our challenges. I often refer to this quick-fix thinking in education because I spend most of my time in this discipline. For example, the educational technology (Edtech) literature for the past several decades is filled with examples of how the application of technology in a learning setting makes no significant difference and has little impact on learning outcomes and that the focus needs to be the learning, not the technology if we want to make a difference (Reich, 2020; Cuban, 2001; Russel, 1999; Wenglinsky, 1998). The research is clear. Edtech is not a quick fix or silver bullet (Thibodeaux, Harapnuik, Cummings, & Wooten, 2017) and the naive notion that one can implement it better than the last group that failed is continually repeated in all levels of classrooms across the nation (Harapnuik, 2021). This is why worksheets and fill-in-the-blank questions even when they are digitized in things like the SAMR model or other quick fixes do not result in deeper learning (Harapnuik, 2017).
When I originally explored why this reliance on lower-order thinking continually persists I naively assumed that we could simply move up to higher thinking to high-order thinking because it incorporates the lower levels and it also has the potential to offer so much more. Unfortunately, the move to higher-order thinking involves more than just the desire to operate at that level. Besides being much easier than higher-order or deeper thinking, lower-order thinking offers a sense of security because it is what our educational system has prepared most people to do. Standardized testing and the competency-based system of education that uses this form of summative assessment exist primarily in the realm of applying, understanding, and remembering which fall into the lower-order thinking in Bloom’s Taxonomy.
While we do have pockets of outcome-based instruction where students are given choice, ownership, and voice through authentic learning opportunities or project-based learning, for the most part, our system relies on information transfer and competency-based instruction which resides in the realm of lower-ordered thinking which can be easily measured. The philosopher, Steven Hicks (2021) argues that our current education system is one that teaches compliance, and rather than learning that life is about solving problems our students are instructed that authorities have all the answers. We use the rhetoric of Dewey and say we want children to grow to be self-reliant, creative problem-solving adults but we have the reality of Thorndike that promotes the information transfer standardized model of education that can be easily measured and allows us to sort our students into the fixed norms of the industrial age (Labaee, 2005). I have listed several obstacles to higher-order thinking but I think the biggest challenge is that most people don’t really understand the difference between the two levels. Furthermore, many don’t realize that learners are seldom asked to move beyond showing they can remember information, can understand how information is used, and how that information is applied in a different yet similar situation.
Bloom’s Taxonomy
According to the revised Bloom’s taxonomy when people are attempting to carry out a procedure or implement a process or apply an existing model to a new but similar situation they are using lower-ordering thinking in their hopes of applying existing information to their situation (Anderson & Krathwohl, 2001). The category of Applying is at the top of the lower-order thinking within Bloom’s taxonomy but it is still considered lower-order thinking and only facilitates information transfer because there is no analysis, evaluation, or creation which are at the higher order and are essential to deeper learning. Drawing a diagram, making a chart, applying an existing process, and solving a formula are all lower-level skills that do not require higher-order thinking and this is typically as far as our education system goes.
I prefer to use the inverted Bloom’s taxonomy because it combines higher-order thinking into a continuum and reveals that analyzing, evaluating, and creating must be conducted in conjunction. The notion of using the information in a new but similar situation detailed in the Applying section seems to match the level of thinking that many students are comfortable with. But, don’t take my word for this. In the following 3 Learner’s Stories podcast Applied Digital Learning (ADL) students reflect on their learning journey and discuss what they have learned and what they would do differently if there were able to start the ADL program now. One of the most consistent laments is that ADL students wished they would have trusted the ADL process sooner and moved away from expecting to be told what to do and simply giving the instructors what they wanted.
This desire and comfort level of being told what to do and being given a checklist prescription of what is required to complete an assignment falls directly into the lower-order thinking that most of our learners are accustomed to. The original definition from Anderson, Krathwohl, and Bloom’s (2001) of Bloom’s taxonomy aligns with what I have seen with many students:
Applying: Carrying out or using a procedure through executing or implementing. Applying is related and refers to situations where learned material is used through products like models, presentation, interviews, and simulations.
Many just want to be told what procedure or model they need to execute or implement and believe that all they have to present an existing model to their colleagues or simulate the applied approach, and their innovation process is complete. To be fair to many of these students, this is what they know and simply what their administrators, schools, districts, or other organizations ask them to do. Applying an existing model, presenting a summary, and even creating a simulation or a model is the norm. This ongoing process of identifying a standard to be met, finding the approved or accepted procedure or process being used in the organization to meet this standard, and finally applying a standardized test or other information transfer confirmation tools to confirm that the standard has been met by the students is what most educators are engaged in on a daily basis. For rudimentary knowledge, simple situations, and information transfer this application process does work well and our education system has been relying on this model for over a century. As we move further into the digital information age we are realizing that our challenges are much more complex and require much more than doing what we have done in the past. To address these more significant challenges we need to move beyond applying existing information or processes in a new but similar fashion.
Moving to Higher Order Thinking
We need to move into analyzing, evaluating, and creating new solutions to ever-increasing challenges that we and our learners will face in the future. We also need to look beyond convenient summaries, quick fixes, or “Coles Notes” solutions and go back to primary sources to get the full picture. If we want to address the ever-increasing complexities of the challenges we face in the 21st Century then we must use higher-order thinking. We must continually investigate, explore, analyze and evaluate what we are doing as we begin creating innovations that will enhance learning. Anderson and Krathwohl’s (2001) explanation of the following three higher thinking levels offers the best starting point for our own analysis, evaluation, and creation of a novel way of integrating these ideas.
Analyzing: Breaking material or concepts into parts, determining how the parts relate or interrelate to one another or to an overall structure or purpose. Mental actions include differentiating, organizing, and attributing as well as being able to distinguish between components.
Evaluating: Making judgments based on criteria and standards through checking and critiquing.
Creating – Putting elements together to form a novel coherent whole or make an original product.
Analyzing, Evaluating, and Creating Leads to Deeper Learning and Learner’s Mindset
While the inverted Bloom’s taxonomy is useful for helping us to see the linear relationship between analyzing, evaluating, and creating and also see how higher-order thinking is separated from lower-order thinking, it doesn’t convey the interrelatedness between analyzing, evaluating, and creating. It also doesn’t show how the interrelation between analyzing, evaluating, and creating contributes to deeper learning.
The Venn diagram (Harapnuik, 2021) reveals how analyzing, evaluating, and creating come together and at that convergence point is where the learner engages in deeper learning and has then moved into the Learner’s Mindset.
This deeper learning and the adoption of a Learner’s Mindset is realized when you create a significant learning environment in which you give your learner choice, ownership, and voice through authentic learning opportunities (CSLE+COVA). I have been applying this approach in all the learning environments that I have created and most recently have applied this to the DLL and ADL programs, the Provincial Instructor Diploma Program (PIDP), and all other aspects of my professional and personal life.
In my original post that I referenced at the beginning of this post, I made the grandiose goal of changing the world one learner at a time. A year later, I am still sharing this approach with as many people as I can. It is my hope that you too will begin the ongoing process of analysis, evaluation, and creation. Through a continual and iterative process of analysis of your learning environment, the new concepts, theories, and ideas you are exploring combined with your goal of bringing learning innovation to your organization, you too can begin to explore and evaluate how best to synthesize your findings and ideas into an innovation plan which will create the changes you desire and prepare your learners for life.
Please remember that this is only one part of a bigger picture and this synthesis will be continually evaluated and analyzed so explore the following and provide your feedback to help this ongoing process:
Anderson, L. W., Krathwohl, D., & Bloom, B. S. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching and assessing: A Revision of Bloom’s taxonomy of educational objectives (Abridge Edition). Addison.
Cuban, L. (2001). Oversold and underused: Computers in the classroom. Harvard University Press.
Reich, J. (2020). Failure to disrupt: Why technology alone can’t transform education. Harvard University Press.
Labaree, D. F. (2005). Progressivism, schools and schools of education: An American romance. Paedagogica Historica, 41(1–2), 275–288. https://doi.org/10.1080/0030923042000335583
Russell, T. L. (1999). The no significant difference phenomenon: A comparative research annotated bibliography on technology for distance education: As reported in 355 research reports, summaries, and papers. North Carolina State University.
Thibodeaux, T. N., Harapnuik, D. K., Cummings, C. D., & Wooten, R. (2017). Learning all the time and everywhere: Moving beyond the hype of the mobile learning quick fix. In Keengwe, J. S. (Eds.). Handbook of research on mobile technology, constructivism, and meaningful learning. Hershey, PA: IGI Global.
Even though constructivist learning theorists for many decades promoted the benefits of self-directed learning or autodidactism it wasn’t until the COVID crisis of 2020 and the mass forced remote learning that most educators had realized that too many students were not suited or prepared to learn online. Why? Justin Reich (2020) points to research in his book, A Failure to Disrupt: Why Technology Alone Can’t Transform Education, which shows that the learners who are most successful in an online or blended environment that requires self-pacing and personal motivation are those who are already successful in school. These self-directed, self-motivated, and academically prepared learners will succeed in any learning environment because they know how to learn and assess the quality of their own work. The problem that we face is that the vast majority of students are dependent on their teachers to direct their learning and to administer standardized testing. If autodidactic learners are able to learn in any type of environment then we should be asking how do we help our learners become autodidacts and adopt a learner’s mindset. I have explored this notion further in the post, We Need More Autodidacts and the related Learner’s Mindset Discussion.
Our research in the Digital Learning and Leading (DLL) program at Lamar University, our experience in the School of Instructor Education at Vancouver Community college over the past several years, and several decades of related research and experience in a wide variety of learning environments have confirmed that if you create a significant learning environment where you give your learners choice, ownership, and voice through authentic learning opportunities (CSLE+COVA) you can incorporate assessment FOR/AS learning which can help shift a learner toward a learner’s mindset. We have also learned through our experience and research that incorporating feedforward or educative formative assessment will also help to continue that shift toward the learner’s mindset. By giving learners choice over most aspects of their learning experience and through the use of authentic learning opportunities and ePortfolios, our students over the past several years have incorporated many aspects of the assessment as learning perspective which are essential to the learner’s mindset.
Unfortunately, all too often there is a very different learning environment that our students experience in the courses and programs I have developed and instructed than the type of the learning environment that my students are able to create for their learners in their organizations. Finding the right balance between assessment of learning, assessment for learning, and assessment as learning is one more factor that plays a significant role in the learning environment. In much the same way that we have explored and differentiated the role of choice, ownership, and voice through authentic learning opportunities we have to do the same for assessment OF/FOR/AS learning.
Rather than add to the decades of literature on assessment OF/FOR/AS learning I will draw upon the key ideas and summarize the salient points that are most important to contributing to a significant learning environment.
For those who prefer a more typical written definition the New South Wales (Australia) Education Standards Authority (2017) provide a good summary of “assessment for, as, and of learning”
Assessment of learning assists teachers in using evidence of student learning to assess achievement against outcomes and standards. Sometimes referred to as ‘summative assessment’, it usually occurs at defined key points during a teaching work or at the end of a unit, term or semester, and may be used to rank or grade students. The effectiveness of assessment of learning for grading or ranking purposes depends on the validity, reliability, and weighting placed on any one task. Its effectiveness as an opportunity for learning depends on the nature and quality of the feedback.
Assessment for learning involves teachers using evidence about students’ knowledge, understanding, and skills to inform their teaching. Sometimes referred to as ‘formative assessment’, it usually occurs throughout the teaching and learning process to clarify student learning and understanding.
Assessment as learning occurs when students are their own assessors. Students monitor their own learning, ask questions and use a range of strategies to decide what they know and can do, and how to use assessment for new learning.
The following assessment OF/FOR/AS learning table is a compilation of from a wide variety of resources that goes a bit further than simple definitions (Chappuis et al., 2012; Fenwick & Parsons, 2009; McNamee & Chen, 2005; Rowe, 2012; Schraw, 2001; Sparks, 1999):
Assessment
Of Learning
For Learning
As Learning
Type
Summative
Formative
Formative
What
Teachers determine the progress or application of knowledge or skills against a standard.
Teachers and peers check progress and learning to help learners to determine how to improve.
Learner takes responsibility for their own learning and asks questions about their learning and the learning process and explores how to improve.
Who
Teacher
Teacher & Peers
Learner & Peers
How
Formal assessments used to collect evidence of student progress and may be used for achievement grading on grades.
Involves formal and informal assessment activities as part of learning and to inform the planning of future learning.
Learners use formal and informal feedback and self-assessment to help understand the next steps in learning.
When
Periodic report
Ongoing feedback
Continual reflection
Why
Ranking and reporting
Improve learning
Deeper learning and learning how to learn
Emphasis
Scoring, grades, and competition
Feedback, support, and collaboration
Collaboration, reflection, and self-evaluation
If we want to encourage our learners to become more autodidactic it would then seem reasonable to shift from assessment of learning to assessment for learning and ultimately get to assessment as learning. We see this perspective from Lorna Earl (2012) in her highly cited text Assessment as Learning: Using Classroom Assessment to Maximise Student Learning.
Earl’s assessment pyramids are featured in many different sources and her argument that the traditional assessment of learning is the dominant form of assessment is widely accepted. Even though she calls for a balance in the use of assessment of/for/as learning her revised assessment pyramid that replaces assessment of learning with assessment as learning as the base of the pyramid still doesn’t represent a realistic balance nor an effective way to incorporate assessment into the learning environment.
Rather than view assessment of/for/as learning as hierarchical it may be more effective to view assessment of/for/as learning more holistically as more of an interplay of assessment within the learning environment. The National Forum for the Enhancement of Teaching and Learning in Higher Education in Ireland (2017) offers a wonderful perspective on assessment of/for/as learning that emphasizes the interplay of the different types of assessment and the key roles that the assessment and the people involved play.
While some learning theorists may desire to craft a potential learning environment that uses assessment as learning, the reality we face, and that our learners face is not theoretical. We live in a world where we use credentialing exams and other forms of standardized testing and while we have seen a recent move toward implementing formative feedback most educators’ reality reveals that assessment of learning dominates. Moving toward assessment for learning and assessment as learning will only be possible if we look at the bigger picture. We need to help educators to recognize that we are not asking for a full pendulum swing away from assessment of learning to assessment as learning with assessment for learning somewhere in the middle. We are acknowledging that an interplay of all three is not only realistic it will be the most productive approach to improving the learning environment.
We must also acknowledge that our teaching and learning environment are dramatically influenced by the assessments we use. If we consider assessment of/for/as learning as an integral part of the learning environment and we look to fully integrate assessment as part of the learning process then we do our learners justice by helping them to experience a balance in the assessment of/for/as learning. If we model an integrated approach to assessment of/for/as learning then we will be equipping our learners so that they too can integrate assessment of/for/as learning into their own learning environments that they create for their learners.
While this more focused examination of assessment of/for/as learning may provide a novel perspective for some, we have been incorporating the assessment of/for/as learning inter-relationship in the creation of our significant learning environments and when we give learners choice, ownership and voice through authentic learning. This assessment as learning perspective is a practical way to move into what the researcher Mizerow would argue is transformational learning. Mizerow (2000 & 2010) argues that you do not learn things until you tell someone about what you have learned. The transformation to deeper learning happens in the reflective process and the sharing of your learning process with others.
The entire shift toward the learner’s mindset includes the shift toward assessment as learning and you and the following posts and video are a few examples of how we have been supporting and exploring how to help learners become self-directed or autodidactic.
Contribution to Your Learning and the Learning Community
The assessment as learning model is realized in the ADL program and courses through the Contribution to Your Learning and the Learning Community collaboration and reflection component of each course.
In addition to viewing the Assessment As Learning Video posted at the top of the page, ADL students are required to also view the following video and then consider their contribution to their learning and their learning community.
Contribution to Your Learning and the Learning Community
“At some point, to be powerful performers in life as well as self-directed learners, students must learn how to assess the quality of their own work” Creating Significant Learning Experiences by Fink, L. D. (2013, p. 103).
This critical reflection allows you to evaluate your ability to be a self-directed learner by getting you to self-assess your contributions to your own learning and to the learning of your classmates. Learning to self-assess is an important part of your being a self-directed and lifelong learner.
You will be self-assessing your contributions to your learning and to the learning community at the end of each course.
Directions: (Expectations)
For each course, you are to select a numerical score from the self-assessment marking guide and then write a rationale (min 500-800** words) that supports and justifies the numerical score you have selected. This rationale will address what is working, what you can do better, and will highlight your contributions to your learning and to the learning of the community. Please provide specific details in your rationale. The rationale must also list the 3-5 members of your base group community with whom you consistently collaborate. The rationale must list a numerical score, include links to your work, and be submitted as a link to a post on your ePortfolio.
**Accelerated ADL option: For those who are currently taking two ADL courses at the same time, you can combine the two separate course Contributions to Your Learning Community reflections into one unified reflection on the condition that you reflect on and articulate how your collaborations impact the connecting of ideas from the two courses. This is much more than just stating that you did combine the two reflections; you need to explain how you combined your collaborations and what was the impact of doing so.
Note: If your rationale lacks specific details and does not support your score (too high or too low) you may be asked to redo the rationale before the score is adjusted and is recorded.
Self-Assessment Marking Guide
Score 90-100
Key Contributions
Contributed to and helped build your core collaboration group.
Provided peer feedback to your core group members.
Revised all assignments and reflected on revisions in this contribution to learning activity.
Completed ALL of the course readings, videos and supporting resources.
Met the various course activity deadlines indicated in the calendar.
Supporting Contributions
Took leadership responsibility in your base group and the course.
Contributed to your learning and the learning of your colleagues by participating in ALL activities.
Active contributions in the various course forums.
You posted in a timely fashion so others can respond to your posting.
Your postings reflect breadth and depth of thinking with research to support your thinking and is cited using APA.
Additional postings were made that did not require research but were rather to contribute to the learning.
Score 80 – 89.5
All of the key contributions were met.
One of the supporting contributions was not met.
Score 70-79.5
One of the key contributions was not met or and one or more of the supporting contributions was not met.
Score 0 – 69.5
More than one of the key contributions were not met and more than one of the supporting contributions was not met.
This guide is provided in each of the ADL courses.
Alberta Education. (2003). Types of classroom Assessment http://www.learnalberta.ca/content/mewa/html/assessment/types.html
Assessment OF/FOR/AS Learning. (2017, March). [National Forum]. The National Forum for the enhancement of teaching and learning in higher education. https://www.teachingandlearning.ie/our-priorities/student-success/assessment-of-for-as-learning/
Chappuis, J., Stiggins, R. J., Chappuis, S., & Arter, J. (2012). Classroom assessment for student learning: Doing it right-using it well. Pearson Upper Saddle River, NJ.
Earl, L. M. (2012). Assessment as learning: Using classroom assessment to maximize student learning. Corwin Press.
Earl, L. M., & Manitoba School Programs Division. (2006). Rethinking classroom assessment with purpose in mind: Assessment for learning, assessment as learning, assessment of learning. Manitoba
Education, Citizenship and Youth. https://www.edu.gov.mb.ca/k12/assess/wncp/index.html
Fenwick, T. J., & Parsons, J. (2009). The art of evaluation: A resource for educators and trainers. Thompson Educational Publishing.
McNamee, G. D., & Chen, J.-Q. (2005). Dissolving the Line between assessment and teaching. Educational Leadership, 63(3), 72–76.
Mezirow, J. (2000). Learning as transformation: Critical perspectives on a theory in progress. Jossey-Bass Publishers. San Francisco, CA.
National Forum for the Enhancement of Teaching and Learning in Higher Education. (2017, March 30). Expanding our Understanding of Assessment and Feedback in Irish Higher Education. Retrieved from https://www.teachingandlearning.ie/publication/expanding-our-understanding-of-assessment-and-feedback-in-irish-higher-education/.
NSW Education Standards Authority. (n.d.). Assessment For, As and of Learning. Retrieved December 7, 2020, from https://educationstandards.nsw.edu.au/wps/portal/nesa/k-10/understanding-the-curriculum/assessment/approaches
Rowe, J. (2012). Assessment as learning—ETEC 510. http://etec.ctlt.ubc.ca/510wiki/Assessment_as_Learning
Schraw, G. (2001). Promoting general metacognitive awareness. In Metacognition in learning and instruction (pp. 3–16). Springer.
Sparks, D. (1999). Assessment without victims: An interview with Rick Stiggins. Journal of Staff Development, 20, 54–56.
Dwayne Harapnuik will be your facilitator – dharapnuik@vcc.ca
Office Hours – By Appointment. If you wish to meet with me please use my scheduling site https://calendly.com/dr-h/15-min to schedule a meeting. If none of the days or times work for you please let me know what will.
