Summarizing multiple-choice tests using three informative statistics

Abstract

An observed score distribution for a multiple-choice test can often be cogently summarized using 3 statistics that are easily calculated functions of the first 3 moments of the distribution. These statistics can be interpreted as providing a mean ability in terms of probability of obtaining a correct response, a guessing threshold, and a reliability relative to binomial variability. The 3 statistics can also be used, as method-of-moments estimates of the parameters of an extended beta-binomial model, to produce a smoothed score distribution and an estimated score distribution if the test were modified to include infinitely many choices for each test item. Examples of applications to actual A..