Criteria used for performing the predictive performance of the model were accuracy rate, positive predictive value, negative predictive value, false-positive rate, and false-negative rate computed from test data according to the LOOCV methodology.
The VPCs, which were used to assess the internal validity of the model, demonstrated that most of the observed CA-125 values and tumour size predictions were included within the 90% confidence interval boundaries of simulated CA-125 and tumour size predictions, suggesting good predictive performance of the model.
To evaluate the prospective predictive performance of the model, predictions were generated for 13 known inhibitors and inducers of CYP3A4 that were not included in the analysis dataset. Figure 2i shows the observed vs. predicted MDZ AUC ratio for these compounds.
In general, the predictive performance of the model increased after performing data sampling.
Finally, a categorical VPC for the frequently used dichotomized CTC count (<5 vs. ≥ 5) was performed and confirmed the acceptable predictive performance of the model (Supplementary Figure S3).
A multiple regression was performed to test the relation between the predictive performance of the model and the participants' accuracy for each of the 256 videos.
To determine the potential gain of the best performing model in day-to-day clinical practice, prospective data from a second cohort of 65 consecutive patients undergoing oesophagectomy between May 2005 and April 2006 were used in the model, and the predictive performance of the model was compared with prediction based on mean LOS.
To examine the predictive performance of the model the differences between the predicted and observed EQ-5D scores at the individual level were examined by computing the mean squared error (MSE) and root MSE (RMSE).
The sampling design strongly influenced the predictive performance of the models while the number of pseudo-absences had minimal effect on the predictive performance.
For the practical validation, potential applications of the neural network models were analyzed, and the predictive performance of the models was assessed using ecological expert knowledge.
The predictive performance of developed classification models was assessed using five-fold external cross-validation (5-CV) procedure considering Balanced Accuracy (BA) value [79] as a criterion of the predictive performance of the models.
