The frequency of positive and negative results was then compared between disciplines, domains and methodological categories.
Methodological categories are based on very general characteristics of the object of study and the parameters measured in each paper.
In one discipline, Plant and Animal Sciences, an additional 50 papers were analysed, in order to increase the statistical power of comparisons involving behavioural studies on non-humans (see below for details on methodological categories).
To avoid this (modest) sign of possible collinearity, methodological categories were reduced to the minimum number that previous analyses have shown to differ significantly in the frequency of positive results: purely physical and chemical, biological non-behavioural, and behavioural and mixed studies on humans and on non-humans [36].
Different combinations of these variables identified mutually exclusive methodological categories: Physical/Chemical (1-N, 2-N, 3-N, 4-Y); Biological, Non-Behavioural (1-Y, 2-Y/N, 3-N, 4-Y); Behavioural/Social (1-Y, 2-Y/N, 3-Y, 4-N), Behavioural/Social + Biological, Non-Behavioural (1-Y, 2-Y/N, 3-Y, 4-Y), Other methodology (1-Y/N, 2-Y/N, 3-N, 4-N).
The year 2010 looks like it was a peak in research using the CCHS surveys, and the number of publications starts to plateau after 2010 with the three methodological categories maintaining their relative ranks.
However, we consider 3D as a different data modality rather than a methodological category.
The effect of per capita academic productivity remained highly significant when controlling for expenditure and for characteristics of study: broad methodological category, papers testing one vs. multiple hypotheses, and pure vs. applied discipline (Table 1, Nagelkerke R2 = 0.051).
Methodological category was a significant predictor of positive results both when all disciplines and only pure disciplines were included (respectively, Wald = 37.943 and Wald = 33.834, df = 8, p<0.001), but not when only applied disciplines were included (Wald = 9.328, p = 0.315; power to detect a small, medium and large effect, respectively 0.18, 0.575 and 0.867) (Fig. 5).
Indeed, among the topologically suspicious links, only those that are characterized by at least one technique of each methodological category –henceforth referred to as well-characterized links (WCLs)–, can safely be considered as non-suspicious direct regulations.
We calculated χ on the basis of the three methodological quality categories of high, moderate, and low risk of bias.
