For inter-observer ratings, Kappa values were "almost perfect" for one item, "substantial" for one, "moderate" for five, "fair" for three, and "poor" for one (Table 2).
We rated our confidence in the effect estimates as moderate for five outcomes [ 23, 26- 28], as low for three outcomes [ 25, 30], and as very low for one outcome [ 29].
The intra-observer reproducibility was moderate for two physicians and was poor/average for the other two physicians.
Agreement was substantial for pain, moderate for four items, fair for three items and slight for two items.
Intensity of training was classified as low for one group, moderate for four groups, moderate to high for two groups, and high for seven groups.
For the eighteen groups (51.4%) in which data were provided, intensity of training was classified as low for three groups, moderate for eleven and high for four.
For intra-observer ratings, Kappa values were "almost perfect" for two of the assessments, "substantial" for one, "moderate" for six, "fair" for one, and "poor" for one.
First, the grouping was weak for eight scales (.01 < ICC < .10) and only moderate for two of the scales (ICC ≥ .16).16
Modulators were moderate for nine participants (lower score 7/16 and higher score 17/24, weighted scores 1.75/4, 1.83/4) and positive for a single participant (subject #8; weighted score: 3.17/4).
Evidence was assessed as strong if a paper was graded strong for population, ascertainment and analysis, strong– if graded as strong for two of the areas and moderate for one area, and moderate if graded as strong for one area and moderate for two areas, or as moderate for all three areas.
Convergent validity agreement was moderate (range for r = 0.55, 0.67) for cluster B disorders rated with DSM-IV criteria, and discriminant validity was moderate for eight of the ten personality disorders.
