To overcome this obstacle, we designed a new method for eliciting probabilities from experts that combines the ideas of transcribing probabilities as fragments of text and of using a scale with both numerical and verbal anchors for marking assessments.
Within medical education considerable attention has been devoted to the development of teaching methods, but examiners are often expected to know how to mark assessments, being regarded as experts in their field.
Effects in the same direction and of similar magnitude were found in meta-analyses of undergraduate assessments only, postgraduate assessments only, machine marked written assessments only, practical clinical assessments only, assessments with pass/fail outcomes only, assessments with continuous outcomes only, and in a meta-analysis of white v Asian candidates only.
Separate meta-analyses of undergraduate assessments (d=−0.42; P<0.001), postgraduate assessments (d=−0.38; P<0.001), machine marked written assessments (d=−0.35; P<0.001), practical clinical assessments (d=−0.42; P<0.001), assessments with pass/fail outcomes (d=−0.59; P<0.001), and assessments with continuous outcomes (d=−0.38; P<0.001) all showed similar effects in the same direction.
9 51 52 The separate analysis of machine marked written assessments and practical assessments allowed us to investigate possible effects of examiner bias and verbal communication skills on ethnic differences in attainment.
In the past this part of the qualification has been assessed by teachers and there were claims in 2012 of a high degree of inconsistency in the way teachers in different schools marked the assessments.
Some GP supervisors also marked academic assessments.
Andrew Hall, the chief executive of the AQA exam board told the House of Commons education select committee he did not believe teachers were cheating in the way they marked controlled assessment modules, but he said they were in a position where "their judgments were influenced by the pressures of the accountability system".
We carried out a prospective cluster randomised controlled trial to assess the effects of including a brief self-affirmation intervention in the medical school curriculum, using high stakes machine-marked written assessments and OSCE (Objective Structured Clinical Examination) assessments as the outcome measures.
