Partial Scoring for Multiple Choice Tests

I read some time ago an article by Terence Tao on how to assign partial credit scores for multiple choice tests – the article can be found here.

Terence’s solution focused on True/False questions, but could be easily generalized to multiple choice questions, but involved as part of the solution giving negative marks to students, including negative infinity marks. Clearly this is an infeasible solution for the classroom, and one wonders if there is a function f:[0,1] \to [0,1] such that f(0)=0 and f(1)=1 which satisfy similar conditions as those outlined in the article. The main condition that needs to be satisfied is that the students expected score should be maximised at their subjective probability of the particular answer being correct.

We can use some of Terence’s work here, to cheat slightly. Let us institute a rule that a person cannot bid 1 or 0 for a particular answer, and secondly that they can only bid a value q\in\{0.01,0.02,...,0.99\}. Now, we can use the form f(p)=Alog(p)+B with the constraints that f(0.99)=1 and f(0.01)=0. This yields the equation f(p) = 0.2176log(p) + 1.0022.

Let us check the required conditions:

f(0.01)=0: f(0.01)=0.2176\times log(0.01) + 1.0022 = 0

f(0.99)=1: f(0.99)=0.2176\times log(0.99) + 1.0022=1

E[mark] = p(0.2176log(q)+1.0022)+(1-p)(0.2176log(1-q)+1.0022)

\frac{\partial E[mark]}{\partial q}=0.2176\frac{p}{q}-0.2176\frac{1-p}{1-q}=0 so that the expected mark is maximised when p=q.

So we have found a function which practically maps to [0,1], with the minor caveat that a student can never pick either 0 nor 1 for their answer.

There is, however, a large flaw with this function, which is a student can lock in a score of 87% simply by choosing 0.5 for both options. Clearly further work is required here!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s