Dunlosky 2013 Mark II

For an educator with a professional intesting in test enhanced learning, one of the seminal papers is Dunlosky et al, (2013) Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Dr John Dunlosky has, in the past, said that “anyone could have written it,” however he and his colleagues did. And it is the measure of his humility that he discounts it’s importance. Cited over 3020 times, it is one of the papers that spark the interest of many educators, and continues to inform thousands of revision assemblies ever year, given it’s accessible and practical nature.

In the monograph, Dunlosky et al, (2013) discussed 10 learning techniques in detail and offer recommendations about their relative utility. That is the extent to which techniques are robust and generalize widely.

Just two techniques received a high “Utility” rating, “Practice testing” and “Distributed practice” (spaced practice) because they were report to “benefit learners of different ages and abilities and have been shown to boost students’ performance across many criterion tasks and even in educational context.”

Conversely, summarization, highlighting, the keyword mnemonic, imagery use for text learning, and rereading were rated as low utility for numerous reasons.

It was followed up by a much lesser cited, but in my opinion, equally informative “Teacher-Ready Research Review” Practice tests, spaced practice, and successive relearning: Tips for classroom use and for guiding students’ learning. (just 77 citations). Here Dr John Dunlosky and Dr Katherine Rawson share pointers on how to use “practice testing and spaced practice” (formerly referred to as distrubuted practice) in the classroom and introduced how successive relearning can be used by your students to guide their learning outside of the classroom.

Donoghue and Hattie (2021)

8 years on, we have “A Meta-Analysis of Ten Learning Techniques,” from Donoghue and Hattie (2021). 242 studies, 1,619 effects, 169,179 unique participants from the research published between 1929 and 2014 and the same conclusion:

The most effective techniques are Distributed Practice and Practice Testing.

Donoghue and Hattie (2021)

Two important moderators, presence of feedback and the prior attainment, that is the benefits were lower for higher ability students. As you might expect, they have already found, and use, strategies that work.

After removeing all non-empirical studies and any studies that did not report sufficient data to calculte an effect size, 242 studies being included, many with multiple effect sizes. Yes, most were undergraduates (65%), of average abilities (86%), from North America (86%).

Three sets of moderators were coded:

• Attributes of the article: quality of the journal (h-index), year of publication and sample size.
• Attributes of the students: ability level of the students (low, average, and high), country of the study and grade level.
• Attributes of the design – 91% experimental compared to control group. 93% learning outcomes were classified as Surface. The post-tests were classified as Near, (74%) predominantly completed in 1 day or less, 91% within 7 days.

Results

The effects of the learning technique have not changed over time (from 1929 to 2015). English and Reading (33.7%), and Science (20.8%) accounted for over 50% of the effect sizes, with “considerable variation in the effect sizes.”

“Near transfer,” performance on a similar task, was much greater than “Far transfer” transfer was to another dissimilar context. Surface learning effects, greater than Deep processing. Importantly, their conclusion highlighted that the majority of studies were based on Surface learning of factual, academic content, measure learning almost immediately after the technique has been used, and only measure Near transfer. Primary students had lower effects on interleaved practice, mnemonics, self-explanation, and practice testing. Both primary and secondary students had lower effects on Underlining. Only six investigated.” Low ability students benefited, The High ability students had negative effects for Interleaved Practice and Summarization. There were no overall differences relating to the effects less than a day, >1 day and <1 week, > 1 week and <1 month, <1 month and less than 6 months.

What is worth knowing?

Donoghue and Hattie (2021) confirm the major findings in Dunlosky et al, (2013a) however some of the low effects were very close estimates to the moderate effects and not low enough to be dismissed. Even the lowest learning techniques (Underlining and Summarization) are sufficiently effective to be included in a student’s toolbox of techniques. However – the counter point would be, why invest time in less effective strategies?

The “preponderance” of Surface and Near learning studies, as with Dunlosky et al’s review, means we should not generalise the benefits when Deep and Far learning is desired. All well and good, but we know that is plenty of evidence reporting the durability of retrieval practice and successive relearning post 2014.

One key point that Donoghue and Hattie (2021) highlight, as we often do here on the blog, is the importance of corrective feedback. To which they add:

It is not the frequency of testing that matters, but the skill in using practice testing to learn and consolidate knowledge and ideas.

Donoghue and Hattie (2021)

(But as @Jon Hutchinson reminded me via Twitter, “frequency” is important, during both the encoding phase and retrieval mode, (e.g., Pyc and Rawson 2009; Vaughn and Rawson 2011). however possibly not as important as sometimes reported. First, the underline the importance of feedback, then with broader reference to the paper, they note that most of the findings are taken from highly-fidelity research settings – not from low-fidelity classrooms.)

Furthermore, Donoghue and Hattie (2021) also point to the importance of dispositions and motivations. Citing three inputs and outputs: “knowing, dispositions, and motivations,” more colloquially, “the skill, the will, and the thrill” (Hattie and Donoghue, 2016). That motivation and emotions are involved in the selection, persistence with, and effectiveness of using the learning techniques, so attention to these matters is imperative for many students.

Their final encouragement – to research the value in teaching these strategies to both teachers and students. Can these techniques be taught, and if so, how? Well, I think Biwer et al, (2020) and McDaniel and Einstein (2020) have that covered.

The obvious limitation: the current study uses the same papers as Dunlosky and therefore stops at 2014.

Mark II

I’d secretly hoping for the sequel 2014-2022. Mark II. I’d love to see where pretesting or potentiated learning and successive relearning fit in? As a teacher that employs As a teacher that employs pretesting, retrieval / successive relearning, I’d love to see how the impact extends, into the “Far” and the “Deep”.

Donoghue, G. and Hattie, J., (2021). A Meta-Analysis of Ten Learning Techniques. Frontiers in Education, 6.

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013).  Improving students’ learning with effective learning techniques:  Promising directions from cognitive and educational psychology.  Psychological Science in the Public Interest, 14, 4-58.

Dunlosky, J., Rawson, K. A., (2015). Practice tests, spaced practice, and successive relearning: Tips for classroom use and for guiding students’ learning. Scholarship of Teaching and Learning in Psychology. 1. 72-78. 10.1037/stl0000024.