48,478 students’ data from 222 studies and 573 effects – Testing Effect

Not that creative a post title I know however I have been reading and re-reading Yang et al., (2021) for the third time. It is a hefty piece of research. It draws together a number of components connected to the Testing Effect, Retrieval Practice, Success Relearning and meta-cognition.

On Wednesday, hot off the press (Online First Publication, March 8, 2021), I read “Testing (Quizzing) Boosts Classroom Learning: A Systematic and MetaAnalytic Review.” Twice. It is a hefty summary. I was glad for the “Public Significance Statement.” A statement that lays down two stern markers in the first sentence.

Testing (class quizzing) yields a variety of learning benefits¹, even though learners, instructors, and policymakers tend to lack full metacognitive insight into the virtues of testing². The current metaanalysis finds a reliable advantage of testing over other strategies in facilitating learning of factual knowledge, concept comprehension, and knowledge application in the classroom. Overall, testing is not only an assessment of learning but also an assessment for learning.

Yang et al., (2021).

They go on to make a third key point in the “Rationale of the Current Meta-Analytic Review” where they contrast laboratory research and classroom practice. The divergences they highlight, they conclude bring the “ecological validity” of the research into question.

Concluding that it is possible that “different mechanisms may contribute to testing effects in the laboratory and classroom.”

Nevertheless onwards… 48,478 students’ data, extracted from 222 independent studies, 573 effects, were interrogated to investigate the magnitude, boundary conditions, and psychological underpinnings of test-enhanced learning in the classroom. So forth time through – what do we learn about the “reliable advantage of testing over other strategies?”

Yang et al (2021) suggest the classroom testing effect may originate from both direct (i.e., testing consolidates studied information) and indirect forward (i.e., testing boosts new learning) benefits of testing. In fact, I am banking on it!

In educational settings, testing is usually regarded as an evaluative instrument to assess learning and comprehension, or to gauge learners’ ongoing progress toward learning objectives. However “a large body of research” has “repeatedly” and “convincingly” demonstrated that retrieval practice (i.e., retrieving information from memory) can more effectively consolidate long-term retention of studied information (Roediger & Karpicke, 2006a) and facilitate mastery of new information (Yang, Potts, & Shanks, 2018) by comparison with other strategies.

No here is the first hurdle: The effects are in fact both backward AND forward.

Backward in that testing consolidates retention of studied information) is a robust phenomenon across different educational materials in both the laboratory and the real classroom.

Forward in that many experiments have documented that testing on studied information can also facilitate acquisition of new information or test-potentiated new learning.

What underlies this benefit? It is certainly more than additional exposure, with testing almost twice as effective as no activity (g = 0.93) and more effective than restudy (g = 0.51). Testing is more effective with feedback (g = 0.73) than without (g = 0.39).

That a level of difficulty is preferential or “desirable” even, with free recall (g = 0.82) more effective than cued recall (g = 0.72), both more effective than recognition recall (g = 0.36) eg multiple choice.

I will add that reaching the answer, being successful, is also important for all manner of reasons – though not directly commented upon it is inferred by the coverage of feedback.

Research Questions

With 19 important questions about the classroom testing effect to cover, each summarised and each explored in depth, I am going to take a break and share these in clusters of two or three. That is, with the exception of the first.

Q.1 Does (and If So, to What Extent) Classroom Testing Boost Student Attainment?

Testing, by comparison with other strategies “significantly boosts student learning achievement to a medium extent” (g = 0.499). Although g = 0.499 is a medium-sized effect according to conventional descriptors, it is a notably large effect by the standards of educational interventions. Also note that 81% of these finding were significant at p=0.01.

In addition, “in real-world [educational] settings, a fifth of a standard deviation [0.20 SD] is a large effect” (Dynarski, 2017; Yeager et al., 2019). Against these comparators, the relatively large effect of testing on classroom learning is noteworthy and of considerable practical importance.

It is also worth restating that the significant enhancing effect of testing inside the classroom (g = 0.514) mitigates the challenge that class quizzing displaces other class activities and impairs attainment.

Testing produced significant learning gains in elementary school (g = 0.328), middle school (g = 0.597), high school (g = 0.655), and university/college (g = 0.486) classes.

Male and female learners obtain comparable learning benefits from testing. The connection – Classroom offers instant testing opportunities.

18 more questions to go…

Also released in the same week Agarwal, et al (2021). Retrieval Practice Consistently Benefits Student Learning: a Systematic Review of Applied Research in Schools and Classrooms. Summarised “Yet more evidence for Retrieval Practice.”