Tips for educators: How to write multiple-choice questions

(a) Not at all difficult. I randomly open the dictionary and choose the 3 weirdest words on the page to use as the distractors. Boom! Easy.

(b) Challenging, but doable.

(c) Difficult. In fact, so very, very difficult that I sweat blood and cry angry glass-shard tears of frustration. Every. Single. Time.

The multiple choice quiz: it should be easy enough to write, right? The questions themselves seem so streamlined, so simple - just a question, some distractors, and the correct answer.

But effective multiple choice questions are notoriously difficult and time-consuming to construct.

There are many ways to give away the correct answer. Students can read more into the question than you intended. It can be hard to find plausible distractors.

So, below are our top tips for designing effective, clear Multiple-Choice Questions, as well as a handy cheat sheet at the end to download, print and keep nearby for reference.

Hopefully these tips will help you to answer (b) - Challenging, but doable - next time we ask!

Multiple-choice questions

Let’s start with a basic definition and some terms.

According to Vanderbilt University’s Center for Teaching (Brame, 2013) a “multiple choice item consists of a problem, known as the stem, and a list of suggested solutions, known as alternatives. The alternatives consist of one correct or best alternative, which is the answer, and incorrect or inferior alternatives, known as distractors.”

Within a single multiple-choice question, educators need to knit many objectives and considerations, while at the same time maintaining clarity and keeping the learner at the forefront of their mind.

To quote The Learning Management Corporation: 

Essentially, if your test-takers get a question wrong it should only be because they don't understand the concept being tested, not because the question itself is confusing.

1 Write the stem (the problem or question)

The stem should be meaningful by itself and should present a definite problem.

Too often, multiple choice items have a short, insubstantial stem that doesn’t give the learner enough information or context.

The stem needs to be a single, self-contained problem that uses the main idea in the question itself. A handy tip is to ask yourself, “will the learner know what the problem is without reading the response options?”

More tips:

• Ensure the question is relevant to your courses learning objectives.
• Try to use: “which statement” rather than “which of the following”.
•  Higher level MCQs can start with a vignette (i.e. a set of information that sets up the question), from which one or more questions can refer to.
• Make sure all of the context is in the stem/question.
• Avoid true/false. Could use categories question instead.
• Put as much of the wording as possible into the stem rather than the distractors

2 Write the correct answer

The answer should be clear and concise.

According to Haladyna (2004), the correct choice is undeniably the one and only right answer.

That sounds easy, right? However, you might be surprised at how often a choice you thought was perfectly clear and concise can come across as confusing, even to someone who knows as much about the material as you do! A good way to test this is to ask a colleague to answer the questions and to give you feedback on their clarity and feasibility.

! Be aware that you aren’t giving away the correct answer accidentally!

 

3 Write the distractors

Use plausible distractors (wrong-response options).

Okay. So, this is the tricky bit. A distractor is an unquestionably wrong answer. But deciding on suitable and plausible distractors is often the most challenging part of multiple choice question writing.

Ideally, each distractor should to be plausible to a test taker who has not yet learned the skill or knowledge that the test is trying to measure, but clearly wrong for those who do possess the knowledge. In other words, a good distractor should be selected by low achievers and not by high achievers.

Have 3–4 choices (including the wright answer) If you have to think hard for a while to come up with one, chances are that distractor will be obviously wrong. All MCQs in a test do not have to have the same number of distractors. Do not use only 2 choices (e.g. true/false). [Fluid and Electrolyte Balance - Fluid Movement page has example of when can be done]

Distractors should be plausible to those who are uninformed, yet should be clearly wrong to those who understand the concept being tested (i.e. don't write obviously wrong distractors).

Ensure your distractors are grammatically correct.

Make the distractors approx. the same length as the correct answer.

Do not use absolutes (e.g. "never", "always", "all" etc...).

Try to avoid "usually".

Do not use "all of the above". If this is the correct answer, the student only needs to know if two are correct; or if this is the wrong answer the student only needs to know one of the distractors is wrong in order to know this option is also wrong.

Do not use "none of the above". If this is the correct answer, then all that is being tested is the students ability to detect wrong answers (i.e. this does not test whether the student knows the actual correct answer or not).

Order all answer choices alphabetically/numerically.

Do not introduce new terms that do not appear in the background, for example if something was only referred to by its description, use that description/alternatively, go back to the background page and introduce the term there.

Use either a single item or a full sentence.

Look back at what the question is asking - do the answer choices make sense in terms of the question?

 

4 Review and revise

Encourage higher-order thinking.

Something that’s useful to consider when reviewing and revising your questions is the extent to which you want to promote higher-order thinking in your testing and learning outcomes.

There are many definitions of exactly what higher-order thinking means. Brookhart (2010) identifies definitions of higher-order thinking as falling into three categories: (1) those that define higher-order thinking in terms of transfer, (2) those that define it in terms of critical thinking, and (3) those that define it in terms of problem-solving.

In general, higher-order thinking is thinking on a level higher than just memorizing facts or repeating information back in exactly the same way it was told. Higher-order thinking requires students to do something with the facts: apply them, analyze them, organize them, align them to concepts and connect these to other concepts, question and critique them, and use them to creatively solve problems. Higher-order thinking encourages students to be independent, active and confident in the way they process and use the learned information. 

You can promote and test for higher-order thinking within your multiple-choice questions.

To help with this, it’s useful to identify, organize, and clarify your educational goals, by using a framework like Bloom’s Taxonomy (1956).  The 2001 revision of Bloom’s Taxonomy (Anderson and Krathwohl, 2001) provides a dynamic, action-orientated framework to classify and describe the cognitive processes and levels by which thinkers encounter and work with knowledge.

Very briefly, these levels are:

The idea is to think about how you can use these levels to test for understanding and ability that extends beyond the basics of remembering and understanding the information.

Some examples:

Remembering	When did WWI begin?	Requires the student to simply recall when the war began.
Understanding	How did the killing of Archduke Ferdinand provide the spark for the war to begin?	Requires the student to understand why the assassination caused the war to begin.
Applying	Which of the following "hotspots" in the world today might be thrown into war with the assassination of a world leader?	Requires the student to understand the situation in a number of places in the world, and apply what they know about the causes of conflict to those situations.

Remembering	What are the symptoms of a cold?
Understanding	A patient has the following symptoms: fever, blocked nose, headache, cough. What ailment do they most likely have?
Applying	What tests would you conduct to determine if the patient has the flu?
Analyzing	How would you determine if the patient has a cold, the flu, or allergies?
Evaluating	[example of a process to test for the flu] what is the best justification for using this process to test for the flu?

 

Here are some other ways to insert higher-order thinking into your multiple-choice questions:

 

Novel material:

Using novel material (e.g. questions based on examples/situations) can be a good way to test for understanding and higher level concepts.

Lowest level example (remembering)	Which is the best definition of a metaphor?
Higher level example (understanding)	Which of the following is a metaphor?

 

Vignettes:

It can be useful to start with a short paragraph that provides context or a set of information that sets up the question or questions. Another possibility is to present a diagram or illustration and ask questions about it.

Tip: Real-world scenarios work well!

Example (applying)

You arrive at the scene of an accident where people are panicked and yelling. Three people appear to be injured. What action will you take first?

Example (analyzing)

A 32-year-old man has a 4-day history of progressive weakness in his extremities. He has been healthy except for an upper respiratory tract infection 10 days ago. His temperature is 37.8 C (100 F), blood pressure is 130/80 mm Hg, pulse is 94/min, and respirations are 42/min and shallow. He has symmetric weakness of both sides of the face and the proximal and distal muscles of the extremities. Sensation is intact. No deep tendon reflexes can be elicited; the plantar responses are flexor. Which of the following is the most likely diagnosis?

 

The level of difficulty of an item can depend on what has been taught in the course:

For example: The pegword method is a good approach for learning lists of items. What is the essential feature of the pegword method that makes it an effective mnemonic device?

• Analyzing - if the basic ideas behind mnemonic strategies and the information processing theory that lies behind it have not been taught
• Understanding - if the pegword approach has been taught, but not explained
• Remembering - if the features and why they work have been taught explicitly

 

Answer choices can also play a factor in determining the difficulty of the question:

The difficulty of the question can be determined by how far away the distractors are from the correct answer:

What is the population of Dunedin?
Easier	Harder
50,098 126,520 634,901 1032,820	124,569 126,520 129,784 130,062

 

We hope these tips help you when you next set out to write multiple-choice questions!

The Lt Team

 

 

Additional Resources

Case Study: Authoring Lessons in Lt - University of Sydney

Easy Authoring - How Lt helps you create modern, engaging lessons

How to engage students: Five easy ways to activate your lessons using Lt

Five Tips for Teachers: How to Activate your Lessons!

De-clutter your lab: Introducing our new education hardware kits

[Free Download] The Most Excellent Course Design Tool Ever - in association with HAPS

Erin Bowkett

Instructional Designer at ADInstruments

Sina Walker

Scientific Writer & Content Developer at ADInstruments

References:

Andersen, L. W. & Krathwohl, D. R. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives, Abridged Edition. New York: Longman.

Bloom, B. S. (1956). Taxonomy of educational objectives. The classification of educational goals. Handbook 1: Cognitive domain. New York: David McKay.

Brame, C., (2013). Writing good multiple-choice test questions. Retrieved [02-25-18] from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/

Brookhart, S., (2010), How to Assess Higher Order Thinking Skills in Your Classroom. Alexandria, VA: ASCD.

Haladyna, Thomas M., (2004). Developing and validating multiple-choice test items (3rd edition). Mahwah, N.J: Lawrence Erlbaum Associates.

Smith, J., (n/d). Writing Test Items. University of Otago, NP.

The Learning Management Corporation, (n/d). Writing Effective Questions. www.thelearningmanager.com.

Photo Credit: Alberto, G., https://flic.kr/p/9UnRWS