Part I covered the background research, that the “more popular option” of premade flashcards may lead to users “missing out on learning experiences that might occur when making one’s own flashcards.” Of course, there is more to it than that.
What Pan et al. (2022) were essentially asking was “does generating flashcard content elicit productive learning processes?” whilst taking into consideration the time-cost of generating the flashcards.
This study exemplifies two important and related principles from the science of learning. First, the added convenience that some learning technologies offer may make things easier, but are not necessarily helpful in the long run. Second and more broadly, some learning techniques that require more effort — and may seem onerous because they increase the chance of making mistakes, are more time consuming, and can be frustrating — are actually more effective than other, less-onerous methods (also known as “desirable difficulties”).
Dr Steven Pan – https://www.growkudos.com/publications/10.1037%25252Fmac0000083/reader
Sections in green refer back to the questions from part I.
User-generated digital flashcards yield better learning than premade flashcards – Pan et al. (2022)
Six experiments. Participants read two text passages then spent 20–25 min using premade flashcards or creating them. To-be-learned information in each condition was identified for each participant and there were precise instructions regarding content generation and method of practice. Crucially, time on task was controlled.
Now this is critical. I agree. And I did double-check.
Yes, no time added to make flashcards; the same 20 min in total was allotted for premade and selfmade conditions. Time is constant.
Dr Steven Pan
Learning was measured on a 48-hr delayed criterial test featuring definition and application questions (measuring memory and transfer of learning, respectively). The delayed test used to address the “durability of learning over time.”
Pan et al. (2020) highlighting the dangers of many suboptimal learning techniques (e.g. cramming) that “yield comparable or even better performance” in the short term compared to more effective methods (e.g., distributed practice) on a delayed test (Bjork, 1994) in the long term.
“Durability of learning?” 48-hr? I was not overly convinced by the delay, 2 days is still relatively ‘near’ testing. In reality, most learning is assessed over a unit of learning, weeks rather than days, or over an examination period, in that case 1-2 or more years. Again I asked.
Yes, it is true that the durability of learning could be assessed using a test that is delayed much longer than 48 hrs, which we might consider for future research.
Dr Steven Pan
How might an extended timeframe impact on the learning gains? Second, how might it impact the balance of time invested in user-generated flashcards versus time invested retrieving or relearning them?
Now in the week where AI and education has hit the headlines, what if that user-gen investment could be assisted?* We will come back to that at the end of the post.
Flashcard generation method
A range of generating flashcards methods was investigated (Experiment 1: word-by-word transcription; Experiment 2: copying-and-pasting; Experiments 3A, 4A, and 4B: paraphrasing; Experiment 3B: generating examples).
Ultimately, each experiment addressed a practical issue facing many students: Given a fixed amount of time*, is it better for learning to create flashcards or use premade flashcards? (That issue was investigated in a manner that manipulated flashcard type while keeping other factors carefully controlled or constant.)
Let’s not forget the problems outlined by Cecilia Astolfi, Teacher of Physics at Brentwood School, in her Impact Magazine case study. Students struggling to select the crucial content, effectively design cards and then use them.
Method
62 undergraduate students. (Age = 20.6 years, 74% female).
How might that level of prior learning expertise help or hinder the use of flashcards and the ability to select the appropriate knowledge?
Two sessions. Participants read a text passage, used digital flashcards to master the meanings of key terms, each of which represented a fact or concept. This was repeated with a second passage. Text passages were just over 500 words in length, five to six short paragraphs. Readability score of 15–16.
I would suggest that flashcards were used terminally, under controlled conditions, to encode the knowledge just read, possibly some consolidation, given the near time frame of 48 hrs.
Within each passage, there were 10 italicised key terms (e.g. “metaphysical painting”), each with a corresponding one-sentence definition (e.g. “a style of painting using representational but incongruous imagery to produce disquieting effects on the viewer”).
For each of the 10 key terms, two types of multiple-choice criterial test questions, each with four possible answer choices (40 questions in total across both passages).
40 items of knowledge would be relatively low compared to the knowledge required for an examination at 16 (UK GCSE) . Also signposting the need to ‘maintenance’ knowledge over time.
One passage each was assigned to the user-generated and the premade conditions. 48-hr delayed criterial test from both text passages.
Two metacognitive questions: Judgements of learning and predicting future test performance.
62 undergraduate students. (Age = 20.6 years, 74% female).
Premade flashcards gains
As participants were able to use the full 20 min to practice, participants viewed each flashcard significantly more times (about 1.6 more repetitions per card) in the premade versus user-generated conditions.
This exposure is clearly an advantage, one that would reduce as the timeframe extends beyond 48hrs.
Results
User-generated flashcards yielded significantly better delayed criterial test performance than premade flashcards. Controls for time-on-task and methods of content generation and practice. Different methods of generating content varied in effectiveness.
Paraphrasing and copying-and-pasting (that required between 13%-52% of the entire first session) enhanced overall performance, generating examples enhanced definition recall only, and word-for-word transcription (arguably the most passive) was the least effective.
In addition, when the quality of premade flashcard content was manipulated, a user-generated advantage still occurred, but to a larger extent relative to low-quality premade flashcards.
Most participants did not exhibit a strong metacognitive awareness of the benefits of generating flashcard content even after doing so.
What do we learn?
A user-generated advantage was repeatedly observed despite deficits in study time, practice time, and number of repetitions per flashcard. The advice is crystal clear: “use flashcard-making features.”
Generating flashcard content benefits learning by eliciting extra processing (rereading, mental elaboration, depth of processing, or even improved attention of the to-be-learned information.)
It is possible that generating content increases the rate of successful recall during retrieval practice, which in turn improves learning.
Pan et al. (2022: 12)
Extra exposure (afforded by premade flashcards) did not always improve learning.
Given the same amount of time and the same methods of practice, user-generated flashcards are likely to be more beneficial.
Pan et al. (2022: 13)
For users that endorse benefits of generating flashcards, this research provides compelling support.
Reasons to be cautious
Guidance for flashcard use is rare. Alignment between flashcard and exam content varies. (e.g. strong alignment for instructor-provided flashcards and weak alignment for premade flashcards made by students lacking foreknowledge of the content to be tested is clearly a variable to be considered).
Yes, alignment could be key in how effective the flashcards are.
Dr Steven Pan
And certainly, learners need to be able to identify the critical knowledge to know.
Dr Steven Pan
Takeaway
Given the same amount of time and the same methods of practice, user-generated flashcards are likely to be more beneficial for learning.
Discussing timeframes, Dr Steven Pan surmised:
Overall, though, I suspect that the critical finding — making flashcards is better than using those provided by someone else — will replicate.
Dr Steven Pan
Final thought – for the 60% who prefer digital, there is the potential gains of personalisation.
Side note
In a week where OpenAI’s release of Chat GPT caused a wave of education news articles, the ripples reached flashcards shores. With Polar, simply highlight some text and click the button and it will take the text you selected and automatically convert it into a question and answer flashcard.