Finally, as described in more detail below, we performed an out-of-sample validation using our data collected outside of CDC Region 2 (New York and New Jersey) to build the model used to estimate ILI in Region 2 based on Region 2 tweets.
The predictive models were subjected to extensive leave-one(or a group -out cross-validation as well as out-of-sample validation usingroup -outression data from additional cell lines and human tumors.
The calibration was performed using a training set of 50 samples and validation using a separate test set of 50 samples representing ER-, PR-, and Ki-67-stained routine breast cancer specimens.
To correct for optimism bias in the C statistic value (that is, over-fitting to a specific sample), we undertook internal validation using 100 bootstrap samples.
To estimate the classification proficiency, we performed internal cross-validation, using out-of-bag (OOB) samples, and external validation, using an independent test set.
Indeed, to focus on genes for which differential methylation and expression could be related, we applied larger sample cohorts (78 paired HCC samples) for a further validation using Illumina MiSeq-BSP (Fig. 2).
These issues include experimental design, sample size, data analysis and validation using an independent set of samples.
Such designs can be broadly classified into retrospective validation (using samples collected from a previously conducted randomized controlled trial (RCT)) and prospective validation (enrichment, all-comers, hybrid or adaptive analysis designs).
This observation requires validation using larger sample sets.
Although we have conducted internal validation using split sample approaches [ 3], we have not previously used an independent data set to assess performance.
The validity of such models has been seriously questioned due to their failure in clinical validation using independent samples.
