In the following, we perform a comparative benchmarking of FML-kNN, in terms of scalability and wall-clock completion time, using the synthetic dataset and the probabilistic classifier.
Using the synthetic dataset described above, we evaluated the accuracy and robustness of different identification approaches via receiver operating characteristic (ROC), area under curve (AUC), positive predictive value (PPV), and false discovery rate (FDR).
Finally, to enable a direct (and fair) comparison between our algorithm and another leading computational prediction algorithm, CHASM (Carter et al., 2009), we performed the same 2-fold cross-validation procedure used in (Capriotti and Altman, 2011) using the synthetic dataset.
In this scenario, we use the synthetic dataset.
To validate the radial current algorithm we used the synthetic dataset as employed for the phase A study (Venner-strøm et al., 2005, 2006; Moretto et al., 2006).
We used the synthetic dataset and we followed a 10 90% testing training split scheme, resulting in 10 M records being used as the testing set (R) and 90 M as the training set (S).
We used the synthetic dataset of black and white random noise square images with eight different image sizes from a low resolution of 32 × 32 pixels to a maximum resolution of 4096 × 4096 pixels proposed by Grana et al. [3].
We evaluated the performance of PTMClust against these two algorithms using the synthetic PTM dataset described above, which provides us with ground truth labels for the true modified amino acids, modification positions, modification masses, identities of the PTM groups and cluster assignment for each peptide.
Using the synthetic mutation dataset, we assessed the performance of these algorithms when tasked with discriminating between somatic driver and passenger mutations.
Complete timing results without parallelization are shown in Supplementary Section 8. We compare the empirical performance of RVD2 to other variant calling algorithms using the synthetic DNA dataset by the false discovery rate (FDR) and sensitivity/specificity.
