The Log station was introduced into the formative, and subsequent summative, OSCEs with careful attention to student and assessor training, marking rubrics and the standard setting procedure.
Students and assessors were briefed on the station and standard setting procedure, and also had online access to the station information and aims, prior to the examinations.
These results are encouraging; it seems logical to have those surgeons involved in the content development perform the standard setting procedure.
PISA on the other hand has not performed a systematic standard-setting procedure, which may indeed be criticized.
"Procedural validity" is the appropriateness of the procedures used in standard setting, with poor procedure casting doubt on the validity of a pass mark but good procedure alone being unable to validate a pass mark.
Despite a requirement that hospitals abide by a standard set of procedures to prevent surgical mistakes like operating on the wrong patient or the wrong body part, such errors continue to occur far too often, researchers are reporting.
The CEFR is a popular basis for standard-setting procedures such as the one conducted for German NA.
DIALANG (Alderson, 2005) was something of a pioneer in devising and implementing standard-setting procedures in the late 1990s in order to estimate whether a given learner is at A1, A2, B1, B2 or even the C levels according to the CEFR and the associated scales, Can Do statements and proficiency descriptors.
Nonetheless, the lack of any measurable influence on item discrimination within this study may be of more practical relevance than our analysis of item difficulty, given the aforementioned weaknesses with regard to cohort effects and absence of standard-setting procedures.
