This is part two of a three part series reviewing Svenja Heitmann Dissertation “On Optimising the Use of Practice Quizzing: Benefits Increase by Adapting and May Depend on Learners’ Achievement Motives and the publications that followed.
Remember, in the first post, Heitmann et al, (2018) reported that:
…the adaptive testing (personalised retrieval practice) group performed better on the posttest than the participants in the non-adaptive testing group. This was not only the case for familiar questions, but also for questions that the learners were not familiar.
Heitmann et al, (2018)
That this benefit was consistent for learning time and question familiarity – reporting a medium effect size when compared to the much preferred student strategy of note-taking.
Heitmann et al, (2018) also reported a significant mediation effect via “perceived cognitive demand.” That the adaptive group outperformed the learners in the non-adaptive group “not only because of their higher performance in the learning phase, but also because their mental resources were less exhausted due to the adaptation of the test questions to their level of knowledge.” That is the learners “profited from the freed up capacity for the execution of beneficial learning processes.”
What does this mean for teachers?
Heitmann et al., (2018) research supports the notion that learning outcomes can be improved by adapting the complexity of quiz question to the learner’s current state of knowledge (although limitations are associated with such a conclusion). What we might refer to as adaptive learning or personalisation.
Declaration: Personalised, spaced retrieval practice is a learning approach I have been championing for sometime. Of course, the paper caught my attention.
This led Heitmann to take a closer look at two adaptive approaches, one investigating the motivational prerequisites or achievement motives and the second, learners perceptions of cognitive load. Now this line of enquiry doubled peaked my interest!
Declaration: I have long been championing the importance of student motivation and “lower failure rates” when adopting retrieval practice in class and the research of Luke Eglington.
You can understand my enthusiasm, here is a dissertation researching retrieval practice, personalisation and motivation. There is also a reference to perceived cognitive load as an adaptive measure. (It left me wondering how different is perceived cognitive load or confidence based assessment?). I was keen to read on. After the first reading, I asked if I could squeeze in one more paragraph into the forth-coming book.
Heitmann et al., (2022) – achievement motivation and perceived cognitive load
Having read and signposted various motivational moderators of retrieval practice, highlighted the importance expectancy value theory (e.g., Eccles & Wigfield, 2020; Wigfield, 1994) in previous posts and called for lower failure rates when quizzing- Heitmann’s dissertation and this paper was of particular interest.
Remember – retrieval practice or quizzing, is not the preferred learning strategy of most students. Rivers, (2020) reported 58% of students adopt ineffective / low utility learning techniques (rereading 43%, copying notes 11%, highlighting 4%). Might Heitmann et al., (2022) provide clues to addressing this motivational indifference towards retrieval practice for learning? Or boost the benefits of retrieval? Maybe both?
Achievement Motivation
Heitmann et al., (2022) head down the classic achievement-motivation literature (McClelland et al., (1953) hope of success and fear of failure route, as two separate achievement motives, Their position:
…quizzing activates learners’ achievement motives. Learners with high hope of success could interpret quiz questions as welcome challenges to test their ability, and the provision of feedback could provide a standard with which they could measure their improvement. By contrast, learners with high fear of failure could interpret quiz questions as unwelcome obstacles and might be apprehensive of the shame and humiliation they expect to experience if they fail to correctly answer the quiz questions. Therefore, the higher learners’ hope of success, the more appealing learners should find quizzing; the higher learners’ fear of failure, the less appealing they should find quizzing.
Heitmann et al., (2022)
Achievement motives and the resultant appeal of quizzing might therefore influence both learners’ performance on the quiz questions and the number of quiz questions answered. Both hope of success and fear of failure have been shown to influence academic performance (Steinmayr et al., 2019).
Here is my position. Success-motivation are bidirectional and that “managing” failure rates is critical to developing a commitment to test-enhanced learning #retrieval practice. Second, with relation to expectancy value theory, learners need to see value in the task, the learning, and also in the approach, retrieval practice.
Perceived cognitive load
Second Heitmann et al., (2022) looked at the learners’ perceptions of cognitive load during quizzing. This aligns with the notion that “effortful retrieval” is crucial for substantial benefits of quizzing to occur. Could cognitive load ratings be used for adapting question sequence? Very high cognitive load ratings would indicate that learners have not yet acquired the knowledge necessary to master a question and might benefit more from quiz questions of lower complexity; very low cognitive load ratings would indicate that learners have already acquired the knowledge necessary to master a question and might benefit more from quiz questions of higher complexity. Wisely, Heitmann et al., (2022) preference caution with any metacognitive judgement, due to the common biases and heuristics of learners self-assessment.
Again, I will add my position, that retrieval practice with metacognitive judgement (be that confidence of perceived cognitive load), with feedback, promotes metacognitive accuracy. Students accurately knowing, what they know and do not yet know, represents a crucial academic advantage. So to the present study.
Method summary
Pretest – to measure the participants’ knowledge on the topic.
As per the first paper, e-lecture, sub topics and quizzing groups. (a) non-adaptive quizzing (n = 54), (b) adaptive quizzing (n = 53), or (c) note-taking (n = 53), four levels of complexity. This time, the increase in complexity was therefore rooted in increasing element interactivity, meaning that, in order to answer the quiz questions, learners would have to integrate more and more idea units that were provided in the e-lecture.
After each question, participants were asked to rate the cognitive load they felt by answering the question “How demanding did you find the question?” on a 12-point scale, providing the basis for the adaptation in the adaptive quizzing condition. Having rated the cognitive load of a quiz question, learners in both quizzing conditions were then given feedback on their answer to the question. Note-taking as before.
Achievement motives were measured with the revised 10-item version of the Achievement Motives Scale.
Learning outcomes were assessed via a posttest one week later.
Results summary
Would increasing hope of success scores benefit from quizzing compared to note-taking as would decreasing fear of failure scores?
Learners in both practice quizzing conditions attained higher learning outcomes than those in the note-taking condition. Quizzing benefits were moderated by hope of success scores but not fear of failure scores compared to note-taking. (However not in the field experiment covered in the third post).
What of quizzing and adaptive quizzing given the learners perceived cognitive load?
The adaptation of quiz questions via cognitive load ratings substantially increased the quizzing effect. (Also translating to the field experiment).
What do we learn?
Quizzing and the motivational orientation matters. Of course, if students can experience success, one would suggest it matters even more so. Heitmann et al., (2022) conclude that for learners with relatively low hope of success “sweeping recommendations for quizzing should be considered with (more) caution.” I would suggest that is an important reflection for teachers in “challenging” or low-aspiration settings.
Alternatively, should we would advise teachers to lower the failure rate? What we are dealing with here is the appeal of quizzing to learners – there has to be value in the investment of quizzing.
Second, increasing the fit of the provided quiz questions to learners’ perception of cognitive load can increase the benefits of quizzing. The paper offers further evidence of the benefits of adapting learning material (such as quiz questions) to learners’ cognitive resources. Likewise, learners in the cognitive demand-based adaptive condition performed better in the posttest than learners in the non-adaptive condition, and that learners in both practice quizzing conditions had higher learning outcomes than learners in the note-taking condition.
Of the two adaptations – the cognitive demand-based adaptive mechanism’s adaptation effect was larger.
The perceived cognitive load is important. Getting that pitched for x30 students… now that requires supremely skillful teaching. But get it right and the students recognise it – certainly, students who learn with RememberMore report that assurance, that they find personalised retrieval rewarding and motivating which in turn gives them confidence in class. “This app (RememberMore) helps me feel more confident going into class.”
Now – having been using RememberMore in class for approaching three years, I was interested to read about Heitmann experiments in the field?
Heitmann, S, Grund A, Berthold K, Fries S and Roelle J (2018) Testing Is More Desirable When It Is Adaptive and Still Desirable When Compared to Note-Taking. Front. Psychol. 9:2596. doi: 10.3389/fpsyg.2018.02596
Heitmann, S., Grund, A., Fries, S., Berthold, K., & Roelle, J. (2022). The quizzing effect depends on hope of success and can be optimized by cognitive load-based adaptation. Learning and Instruction, 77, 101526. https://doi.org/10.1016/j.learninstruc.2021.101526
Heitmann, S., Obergassel, N., Fries, S., Grund, A., Berthold, K., & Roelle, J. (2021). Adaptive practice quizzing in a university lecture: A pre-registered field experiment. 139 Journal of Applied Research in Memory and Cognition. Advance online publication. https://doi.org/10.1016/j.jarmac.2021.07.008
McClelland, D. C., Atkinson, J. W., Clark, R. A., & Lowell, E. L. (1953). The achievement motive. Appleton-Century-Crofts.
Steinmayr, R., Weidinger, A. F., Schwinger, M., & Spinath, B. (2019). The importance of students‘ motivation for their academic achievement – Replicating and extending previous findings. Frontiers in Psychology, 10, 1730. https://doi.org/10.3389/fpsyg.2019.01730
Pingback: Adaptive quizzing in the field – Edventures
Pingback: Overt or covert retrieval – of both – Edventures
Pingback: Bahrick to classroom – three phases (part 2) – Edventures
Pingback: Bahrick to classroom – three phases (part II) – Edventures