Creating multiple-choice test questions can be quite time-consuming and challenging. Have you ever found yourself struggling to come up with a “good” fourth answer option? Well, if this has been the case for you, you’re in for some good news! It turns out that trying to identify a plausible fourth option may not be worth it. Research on this topic suggests that you can stop this frustrating process and start spending your valuable time on other more valuable academic tasks.
Baghaei and Amrahi (2011) researched whether the number of answer options had a significant impact on test performance. In their experimental study, 180 undergraduate students were randomly assigned to one of the following three conditions: three answer options, four answer options, or five answer options. In the three and four answer option conditions, distractor items were randomly eliminated (so it wasn’t that the “worst” answer option was eliminated). The students in this study completed a multiple choice test comprised of thirty questions. All of the tests were exactly the same except for the number of distractor items. So here’s the main finding: no significant differences were found in terms of item difficulty. There were also no differences found in terms of test reliability. Thus, Baghaei and Amrahi (2011) concluded that three answer options are all you need. If the test characteristics are essentially the same, there doesn’t seem to be any reason to spend our time developing additional answer options.
Still not convinced? It is just one study after all. Well, you might be surprised to learn that this is not a new finding. In fact, it has been a consistent finding the in the research for years. Rodriguez (2005) conducted a meta-analysis on this topic, reviewing eighty years of research. In this study, twenty-seven previous studies met their criteria for inclusion and the same result was found: three answer options is all you need. Rodriguez (2005) argues that shifting to three answer options also increases the amount of content that can be tested. Because students don’t have to spend as much time reading four or five answer options, there will be more time during the test for students to read additional questions on different course content. Instead of spending your time on identifying more answer options, spend your time developing additional test questions. This is more productive, helping you assess the content of the course in a more comprehensive way.
Baghaei, P., & Amrahi, N. (2011). The effects of the number of options on the psychometric characteristics of multiple choice items. Psychological Test and Assessment Modeling, 53(2), 192-211.
Rodriguez, M. C. (2005). Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research. Educational Measurement: Issues and Practice, 24(2), 3-13.
How many answer options do you include on your multiple-choice tests? Have you tried varying those numbers? What were your results? Share your thoughts, or your questions for Dr. Harrington, in the comments section below.
Have you heard? Dr. Harrington is running Scholarly Teaching Institutes. These are opportunities to spend a full day exploring a pedagogical topic. This spring, there are two upcoming Institutes on Dynamic Lecturing, one in Orlando, Florida on February 19, 2016 and another in Hamilton, New Jersey on April 22, 2016. For more information and to sign up for her mailing list, visit www.scholarlyteaching.org.