The 10% of remaining sites were used to compute the cross-validated likelihood.
Since the cross-validated likelihood depends on the randomized folding of the data, we repeated the regression 50 times and selected the model with the highest cross-validated likelihood.
The optimal value of the tuning parameter λ is found by maximizing the leave-one-out cross-validated likelihood.
In order to estimate the best threshold, SUPERPC computes a cross-validated likelihood ratio (LR) statistic using the first, the first two or the first three principle components.
The fits of these additional models to the data were measured using cross-validated likelihood (see Methods and Additional file 4, Table S3).
Note that, to compute the cross-validated likelihood under CAT, we have made an approximation (see methods), but one which leads to an underevaluation of the cross-validation score of CAT.
In keeping with the hypothesis that over-expression of these genes is predictive of recurrence, we constrained the regression coefficients to positive values, and selected the L1 penalty parameter by optimizing 10-fold cross-validated likelihood.
Among the three additional models considered, CAT + GTR nt + Γ4 [ 43] (see Methods) yielded the best fit to the nucleotide data set, and it outperformed GTR nt + Γ4 by 103 points of cross-validated likelihood (Additional file 4, Table S3).
The classifier had a median cross-validated likelihood ratio of 3.94, indicating that it was able to identify a subpopulation enriched in non-responders (~0.75:1 responder to non-responder ratio) compared to the full population (~2.5 1 responder to non-responder ratio), and place nearly one third of the non-responders in this subpopulation.
For each of the replicates, the cross-validated likelihoods were calculated under the test set, averaged over the posterior distribution of the learning set, discarding the first 500 sampled points as burn-in and using the remaining 1,000 generations.
In ridge regression, the amount of penalisation is driven by predictive performance in cross-validation (cross-validated partial likelihood).
