The conclusion has to be that the pass rate for Part 2 was increasing before statistical equating was introduced, and it continued to do so afterwards.
A formal test can show that, there being no difference in DIF for items used before statistical equating was introduced and after (Part 1: t(5224) = .054, p = .957; Part 2: t(6383) = 0.001, p = .999).999
The practical procedure for implementing statistical equating for the Part 1 examination, was developed in conjunction with a group of external psychometricians with previous expertise at IRT and statistical equating, and used the specialist IRT software Winsteps.
The present study provides a detailed case study of introducing statistical equating, and issues which may need to be considered with its introduction.
What the changes are not due to is an artefact resulting from the introduction of statistical equating, and indeed, it is in part the statistical and theoretical robustness of statistical equating which means that the increase in part rates can be accepted as real.
The changes can only be properly understood given a background of the methods used before and after statistical equating was introduced, and we will describe them first as a part of the introduction.
DIF analyses were therefore carried out for all diets of Part 1 and Part 2, using the program Bilog-MG, and diets compared before and after statistical equating.
Particularly clear in Figures 4 and 5 is that there is no obvious change in the pattern of DIF before and after statistical equating was introduced.
The red box indicates the diets for which statistical equating was used, and the green vertical arrows indicate the diet used as the base form, and the diet for which re-equating occurred.
Although DIF undoubtedly occurs in Parts 1 and 2 of MRCP UK), the extent of it did not seem to change after statistical equating was introduced, and nor did the balance of questions favouring UK or non-UK candidates.
