How the MSRA is Marked

Ranking questions are different in that they provide marks for near misses. In ranking questions you will typically be given 4-5 options to rank. If you place the option in the correct rank you will get 4 marks, if you place the option either one slot above or below its true rank then you will still score 3 marks. Two slots away scores you 2 marks and so on. There is no negative marking.

When added up this gives you a maximum score of 20 per ranking question and a minimum score of 8.

There are actually many good reasons for standardising the score:

• The CPS paper will typically have a maximum score of ~86, whereas the PD paper will have a maximum score of around ~640. This is as PD questions have a maximum score of 8-20, whereas CPS questions will have a maximum score of just 1. Standardising these scores helps giving them equal weighting.
• Not everyone sits the same MSRA paper, even within the same MSRA window. In fact there are typically about 8 different papers, this is to help stop people sharing questions and giving later exam takers an unfair advantage. These papers can have a subtle difference in difficulty which can result in people performing worse on one MSRA paper compared with another. Standardisation attempts to correct for this so people don't get unfairly penalised for sitting a harder paper.
• It allows for comparison of performance across papers taken in different years.

Standardisation itself is a simple process and is done separately for each paper.

The average (mean) raw score is calculated across everyone that sat the paper, this is normally around ~79% for the PD paper and ~69% for the CPS paper. Candidates with the mean score are given a new standardised score of 250.

Scores are then allocated based on how many standard deviations your performance is from the mean, with +40 standardised points given for every 1 SD above the mean and -40 for every SD below the mean.

This is what your score represents, how many standard deviations your performance was from the mean.

The distribution of scores is very left skewed, especially for the Professional Dilemmas paper. This means that most candidates do reasonably well and will score close together with a long tail of candidates who score much more poorly.

Because of this candidates who score low, but maybe reasonable scores, will get a catastrophically low standardised score. For example if you get 60% of the full marks on the Professional Dilemma paper your standardised score will be 101.

This standardisation process is completed separately for both papers. You will receive a standardised score for each paper and one combined score that adds them both together with equal weighting.

Standardisation is done across all MSRA takers, not on a speciality-by-speciality basis. As such your MSRA score will be the same for each speciality training programme that you have applied for within the same recruitment round.

The SEM tells us where a candidates true score is likely to lie if any error were to be removed from the exam. It cannot tell us precisely what a candidate should have scored, or even if they were helped or hindered by error.

Another way to think of this is as follows: If we have 100 people sitting an exam and compared their scores, a perfect exam would rank the 60th most knowledgeable candidate 60th each time. In the real world, where exams have biases and errors, the exam may sometimes rank this candidate as the 50th best, or the 70th best depending on the variation of the exam sat. This range of possible values is the Standard Error of Measurement.

The better an exam is, the smaller its SEM should be.

The SEM of the Professional Dilemma paper is typically around 14 to 15 raw marks.

This is quite small in the context of the overall marks available, representing about 2.2% of all the marks available.

Because most candidates score very closely together however, this error ends up having a much larger effect on your standardised score. This is the random number machine aspect of the Professional Dilemma paper in action. You can see the effect this has in the chart below.