Do Item Writing Best Practices Improve Multiple Choice Questions for University Students?

We conduct a randomized experiment using a series of three finance exams sat by over 1,000 students at an Australian university, to determine the extent to which multiple-choice questions (MCQs) are improved by item writing best practices. We identify deviations from best practices (i.e. “flaws”) in MCQs, correct these flaws in a random sample of MCQs and re-administer the exams. Regardless of question difficulty, editing the MCQs according to best practices provides greater clarity for students and increases the proportion of correct responses. We also observe a substantial improvement in MCQ performance related to the effective removal of non-performing distractors. The effect of MCQ editing is larger on those items that students find more difficult. The discriminatory power of MCQs edited according to best practices declines on average, but it increases among students with the lowest scores.