Our findings show that Cochrane reviews are more likely to assess individual methodological components, 13 whereas non-Cochrane reviews were more likely to report using a quality assessment scale such as the Jadad scale 14 or other such scale, contrary to recommendations warning of the hazards of using such an approach.
Cochrane reviews were much more likely to assess individual methodological components (Cochrane: 90%; non-Cochrane: 34%), whereas non Cochrane reviews were more likely to report using a quality assessment scale (Cochrane: 9%; non-Cochrane: 38%); 20% of non-Cochrane reviews did not report the method used to assess risk of bias.
We examined study heterogeneity and implemented stratified analyses to assess intervention efficacy based on study quality as opposed to using a composite quality scale due to the potential problems in the interpretation of these scales [ 12].
This would combine the advantage of using a continuous quality scale, which records small shifts in perceived quality, with the advantage of a labeling procedure, which gives the possibility of recording multiple qualitative aspects of the same stimulus.
In a typical subjective quality assessment study, test subjects are asked to rate the test video using a given quality scale.
Due to the wide range of study designs considered in the review, it was difficult to use a structured quality scale.
Each randomised trial was scored for quality by using a three-item quality scale [ 30].
The methodological quality of included studies was assessed using a five-point quality scale [ 16] with minor adjustments, including the trial's design, double blinding, withdrawal rate, analysis and presentation.
Because of the stringency of this systematic review, we evaluated the methodological quality of the 12 included trials using a five-point quality scale, as described in the Methods section (Table 2).
We assessed the methodological quality of studies using Detsky quality scale.
