Table 3 shows the use of ECHO in all the judgement situations: Five GPs used information about EF in their diagnostic reasoning for five of the case vignettes, four GPs used it for four of the vignettes, one GP used it for three of the vignettes, one GP for two of the vignettes and two GPs for one of the vignettes (and in one case in the wrong direction).
Lens Model statistics could prove useful when comparing different options for the design of simulations used in clinical judgement analysis.
The issue of time in making comparative judgements was highlighted in a study by McMahon and Jones (2014), who investigated the use of comparative judgement in the assessment of a secondary school chemistry experiment.
In this paper we argue for the increased use of comparative judgement in the assessment of performance, particularly where creative expression is an integral component of that performance.
Several researchers have found that assessors needed little training in the use of comparative judgement systems and the process, including those in which judges had not had specific assessor training (see Heldsinger & Humphry, 2013; McMahon & Jones, 2014; Pollitt, 2012a).
The fifth step consists of sensitivity analyses to explore the effects of changing the data or judgements used in the original analysis.
The fifth step consists of sensitivity analyses to explore the effects of changing the estimates or judgements used in the original analysis.
These online systems, such as the Adaptive Comparative Judgement System (ACJS) (Pollitt, 2012b) and the Pair-Wise Web Software (Humphry et al. 2013 2015), are based on Thurstone's (1927) law of comparative judgement and have dealt with many of the issues, that in the past inhibited the use of comparative judgement in assessment (Jones & Alcock, 2012; Pollitt, 2012a).
The negative concept "non-P" used in the paraphrases of O- and A-judgements is more problematic.
No formal adjustments were made for multiple testing but P-values were reported to allow judgement to be used in the interpretation of the results.
