The results of this OSCE (reliability >0.8), which tested the application of clinical skills, demonstrated a significant difference between junior and senior residents for the overall global ratings, total checklist scores, as well as for the global ratings/checklist scores for each station.
The PI (KRA) and a clinical psychologist (STW) will oversee the ratings and checklist to provide corrective feedback to the study therapist as needed in real time.
For the other four checklist ratings, ratings for all indicators were considered collectively to assign an overall rating (High/Moderate/Low) for a given area or process, as detailed in Tables 3, 4, 5 and 6 and described in further detail below.
The study objectives were (1) to evaluate the reliability of the assessment form used by SPs and (2) to compare global ratings to checklists reproducibility.
The use of global ratings and checklists for observations has been widely used for the assessment of technical skills within surgery [ 12- 15] and has also proven reliable for echocardiography in a non-workplace-based Objective Structured Clinical Examination (OSCE) [ 8].
There was good agreement between therapist checklist ratings of delivery of target intervention components and independent ratings based on recorded sessions (κ=0.82).
Standardized inter-rater reliability for independent video raters was.79 for the assessment of step 4 of Peyton's four-step approach, and.75 for the evaluation of the final, sixth repetition when using binary checklist ratings, and.76 for the assessment of step 4 of Peyton's four-step approach, and.81 for the evaluation of the final, sixth repetition when using global performance ratings.
Structured performance ratings also include checklists, rating scales, and inventories but are completed by a trained evaluator to provide objective evidence of performance.
For expert binary checklist ratings, an ANOVA with the between-subject factor 'Group' (HFF vs. LFF) and within-subject factors 'Time' (T1 vs. T2) was conducted.
These children were randomly selected or oversampled based on Child Behavior Checklist ratings from a large population-based study (N = 6,172).
Regarding the exploratory analyses of task specific performance (as reflected by our binary checklist ratings), both groups benefited from the training but with no significant difference between the two groups.
