Crowdsourcing data analysis, a strategy by which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective analytic choices influence research results.
Other analytic choices could be utilized but a consistent approach should be applied throughout the implementation.
3) Provide greater methodological detail and justification for the analytic choices made, particularly with respect to the rationale for choosing eight states and collapsing across subjects.
Also, those P-values might be based on samples of different size, obtained using variable recruitment strategies, and might originate diverging analytic choices with respect to data cleaning and inclusions of covariates.
We have also conducted multivariate sensitivity analyses by developing multiple scenarios based on different combinations of analytic choices (e.g., discount rate or perspective), efficacy levels, vaccination program costs, etc.
It is essential for the progress of such studies to build up public databases of identified, high-quality images and commented source code, documenting analytic choices in full detail.
Based on UK10K whole-genome sequence data, we derive genome-wide significance thresholds ranging between 2.5 × 10−8 and 8 × 10−8 for our analytic choices in window-based testing, and thresholds of 0.6 × 10−8 1.5 × 10−8 for a combined analytic strategy of testing common variants using single-SNP tests together with rare variants analyzed with our sliding-window test strategy.
Although the Framework Method is a highly systematic method of categorizing and organizing what may seem like unwieldy qualitative data, it is not a panacea for problematic issues commonly associated with qualitative data analysis such as how to make analytic choices and make interpretive strategies visible and auditable.
Accordingly, to comparatively evaluate the health and economic impact of the two new vaccines (HPV and rotavirus), we further standardized on some analytic choices and scope of the costs, but kept the same model inputs and assumptions as described in the previously published articles [ 6, 17].
However, for a combined strategy including single-marker tests of common variants combined with window-based tests for rare variants, we found that single-variant results are the principle drivers of significance thresholds and hence that significance thresholds depend little on the analytic choices for the windows and more on the MAF thresholds chosen.
