Why Quiz? Effective quizzing routines (Part 2)

In part 1, I shared the benefits of quizzing and of routines. Now to the practicalities of quizzing in the classroom – with a little help from classroom.remembermore.app and plenty of discussion with Sarah Cottingham @overpractised. (See the link at the end of the post.)

Three steps to quizzing

Most of the work and expertise is invested prior to teaching, defining the knowledge and designing the questions. Step three takes us into (and beyond) the classroom.

1. Define the knowledge (breadth and weight, distribution and sequence of knowledge) – deck definition
2. Design the questions / flashcards, categories and tags – deck design
3. Deploy (routines)
   • Breadth and weight
   • Failure rates
   • Use of feedback – corrective and elaborative interrogation
   • Self-assessment and metacognitive monitoring
   • Quiz expectancy
4. Employing personalised, spaced retrieval practice – remembermore.app

Deck definition

“Where the information being learned has a framework or structure that can be used to organise both the learning and the retrieval then memory is often considerably improved.” — Michael Eysenck, The Blackwell Dictionary of Cognitive Psychology (1994).

Not to over complicate things – quizzing is rooted in curriculum design. Any quiz needs two components: defined knowledge, what it is the learner should know and questions to assess whether that knowledge is accessible. What you may not know, is that RememberMore gives your defined knowledge a “framework or structure,” and with that, organisation for the learner and flexibility for the educator.

The breadth and weight, (distribution and sequence of knowledge)

First you need to know what the learners need to know (specification minus what the pupils already know) and define what information is essential. (Potentially what knowledge needs correcting, particularly any knowledge that acts as a gate-keeper to progress (threshold concepts) or knowledge that inhibits understanding or thinking about a topic eg subject specific vocabulary).

Now that you have defined the weight and breadth of knowledge required – you may wish to consider the sequence.

• 1000 knowledge Cards could be separated into:
  • 2 decks of 500 Cards
  • 5 decks of 200 Cards

Categories and 🏷️Tags

Deck design

The structure of decks can be further separated into Categories. Categories are large, broad, structural divisions of information. Categories can be further separated using tags. For example:

• a text could be categorised by chapter, tagged by chapter number.
• a play could be categorised by act, tagged by act and scene
• a subject could be categorised by paper, tagged by topic

A deck can have multiple categories each with multiple tags. Quizzes can then be quickly designed to target specific knowledge. You can now see for yourself the deck “framework or structure.” There are more features and learning opportunities however let’s leave “Define” there for now.

Understandably, designing knowledge cards benefits from expertise and experience. (Also extending cards with “notes” for elaborative interrogation or questioning.) Writing decks collaboratively is a worthwhile department activity and offers obvious efficiencies.

Remember, the aim of quizzing and #retrievalpractice is “successful retrieval.” Categories and tags also help cue retrieval. (Removing tags is also a way to increase card challenge). Otherwise here are a few tips for writing effective cards.

💡Tips

• Use single point prompts and responses. (Avoid multi-part responses. These are very hard to “free-recall.” As you would expect they require x recall attempts)
• Two short Q-A are preferable to one long, single Q-As.
• If you need to explore a process or a sequence – design individual cards as steps.
• Road test your cards. RememberMore learners offer give great feedback. It takes but a moment to amend a card. 
• Vocabulary in RememberMore is best presented as word, word class and phonetical spelling. Followed by the definition. Notes are used for examples. We have found that this format encourages pupil oracy.
  • Eucharist (n) Eu·​cha·​rist Spiritual communion with God (Also called Holy Communion or Lord’s Supper)

RememberMore metrics will inform you about card difficulty and therefore your teaching. You can always modify a card or add cards to target knowledge gaps.

I have thoroughly enjoyed creating the decks for Classroom. Having not taught the Cold War in 18 months, it has really helped to strengthen and refresh my subject knowledge and really get to grips with the big picture and key events. It’s also given me lots of inspiration for teaching it again, especially in terms of highlighting the links across the events. I’m really looking forward to using Classroom and then RememberMore for all units with my classes too.

Teacher and deck designer

Deploy (routines)

Teaching with Classroom.remembermore.app (with RememberMore) is as much about defining the deck and designing great knowledge cards, as it is building effective quizzing routines. The routine is where the rubber hits the road.

Remember, investing in building a deck helps you prioritise what knowledge is essential, it secures and consolidates your own knowledge as well as surfacing connections within the knowledge (the latter possibly the most beneficial to me as an English teacher). It is also important to highlight Pep’s final benefit of routines, of “free[ing] up teachers mental capacity,” for teaching. Having prepared questions, knowing you have instant, flexible quizzes available to you at any time, offers real assurance to teachers and frees up capacity to really listen to the students responses and underline where meaning is being made or “track lost detail”.

Retrieve ➡️ Reveal ➡️ Repeat. ⤵️ Reorder. 🔄 Refresh. Refine. 

Deploying the routine is the final step. Here we highlight two distinct phases of learning. When introducing knowledge for encoding and when relearning or retrieving knowledge.

Use the left side of the model for encoding knowledge. OVER-learn and secure this routine before increasing the weight of cards or extending the breadth of knowledge or 🏷️Tags. Moving towards the right side for retrieval and relearning 

💡Tips

Encoding	Retrieval and relearning
Outline why the class is committed to quizzing. (Knowledge KBCP model)	Recognise success or where learning surfaces in the lesson. (Knowledge and Beief – KBCP model)
Set expectations for quiz performance. Low scores outline the need for pupils to direct their attention in class.	Ask pupils to forecast their performance on a quiz (this tells you a lot about the pupils investment)
Display a timer to manage a common finish time.	Set more cards in the same time – (Belief – BCP model)
Pupils answer all questions – always. (Answer generation effect)	Relearning is far more efficient. More cards in the same timeframe. Extend the breadth as well as the weight of knowledge.
Overt quizzing – primarily for recording knowledge. Pupils correct or complete answers (highlight additions)	Move to oracy or covert quizzing for relearning. Use elaborative questioning to deepen understanding.* Time costs for elaborative questioning are high.
Pupils self-assess using 2, 1, 0 marking scheme (over extensive feedback cycles, repeat or reorder the quiz)	Pupils self-assess using 2, 1, 0 marking scheme and check against forecasts Peer marking using 2, 1, 0 marking scheme and check against forecasts
Promote learning, tracking is optional.	Track an “expected” quiz, eg weekly or fortnightly. Box plot attainment (Belief – BCP model).

Self-assessment and metacognitive monitoring

When people avoid tests, and choose restudy instead, we hypothesise that they are not trying to avoid taking a test per se, but that they are trying to avoid failure; they do not want to get the answer wrong.

Vaughn and Kornell, (2019)

As a part of the quizzing process you may wish to use self-assessment or peer-assessment of quizzes as learning or a combination – and also include teacher assessment too. Again, marking benefits from routines.

With the questions and answers (cue-response) displayed:

• 2/2 marks for a precise / exact answer – I find myself using the phrase “correct and accurate.”
• 1/2 mark for a correct answer or partial answer – pupil corrects or completes the response and highlights the corrections or additions made.
• 0/2 if incorrect. Remember – there should be no missing answers. Pupils add the correct answer.

Although I quiz every lesson, I hold a weekly quiz where I collect and track self-assessed scores. These are then presented as a box-plot chart back to the class. The box plot preserving pupil anonymity. Again, very powerful for developing belief and commitment to quizzing.

Why quiz at the start of term? We have offered strong 11 reasons – that is before we move onto using the same content for homework or self-study.

One final point. Asking pupils to forecast their quiz score on the tracked quiz can be very revealing. We always recognise forecast accuracy and challenge low and accurate forecasts. Why? If pupils correctly forecast low scores, they knew they should have invested more ahead of the quiz.

Next steps

The next step is to use Remembermore.app for self-study. Here the benefits of retrieval can be enhanced by “adapting the sequence of test questions to learners’ knowledge,” Heitmann et al, (2018), a worthy an effect size of 0.29.

Sarah picks up on our discussions here around 17 minutes in her presentation for “A Learners Toolkit.”