“It’s just guesswork.”

Well yes, it can be.

Here’s a question for you:

What is the capital of New Zealand? Is it:

1. Auckland
2. Tokyo
3. Canberra
4. Wellington

OK, so assuming a minimum level of education we can rule out Tokyo and Canberra as capitals of other countries. That means that the chance of getting the answer correct – if you’re not sure between the two remaining options – isn’t 25% but 50%.

Two of the ‘distractors’ (the technical term for false options) are effectively wasted here, because they are very obviously incorrect. So why put them in? Just because the MCQ format required 4 options?

This is clearly a poorly written question – but other types of less than effective question design may be less obvious.

How about this question from a Fire Safety course:

What are the necessary ingredients for a fire to start and continue?

1. Oxygen
2. Heat
3. Fuel
4. Oxygen, heat and fuel

There is a clue in the way the question is constructed: the fact the it is plural ‘ingredients’.  Even if you have not read the module content, and did not already know the answer, you could figure out from the wording that it is option 4.

These are examples of what not to do. Now let’s look at some simple guidelines on best practice. This isn’t intended to be a comprehensive list; there are many in-depth guides out there if you’re interested in the academic research to back this up.

• Make MCQ options roughly equal in length as often a longer option is a clue that it is the correct choice.
• Avoid silly / plainly incorrect options, unless you’re writing first round questions for Who Wants To Be A Millionaire?
• There is no evidence that having lots of options (say >5) leads to more reliable assessment.
• Make sure that the questions you include have been covered in the training content.
• Avoid negative questions (eg which of the following is NOT true?)
• Take into account those of your learners (whether employees or students) who are neurodivergent or for whom English is not their first language.
• Think carefully about what is a reasonable pass mark. (See the separate blog post on this topic.)
• Think about how many attempts you are going to let the learner have to pass the test before they are deemed to have failed.
• If your system allows, choose the setting for random presentation of options. (CourseBeam can do this.)
• If your system allows, base your assessment on choosing a random selection of MCQs from a question bank. (CourseBeam can do this.)
• If you are creating a question bank, ensure that all questions are at the same level so all learners have an equal chance of passing.
• If your system records which questions were answered incorrectly, look for patterns across your cohort of learners. If you can see that the same questions were answered incorrectly again and again, you may want to address this in your training content.

Finally, here’s a question from an exciting new course we’re working on at the moment for the music industry. If the user has paid attention during the main part of the module, the answer should be clear; but if not, there are no hidden clues or ‘wasted’ distractors.

MCQs may have gained a bad reputation but that’s down to how they have been used. Designed with care, there is no reason not to use them alongside other question styles.