This suggests that the choice of error metric plays a crucial role in assessing the capabilities of any lossy compression technique in terms of accuracy.
Using the analytical mover as a reference, we demonstrate that the choice of error estimator in the Crank Nicolson mover has significant impact on the overall performance of the implicit PIC algorithm.
Moreover, with so few data points it is difficult to see how Schick et al. could justify their choice of error distribution and link function ahead of other viable alternatives; this is important because alternative but equally justifiable choices could lead to very different results, particularly with regard to the whereabouts of the upper confidence limits.
Following [19], the optimal loss function in a maximum likelihood sense for a particular choice of error distribution (P psi _i)) can be formulated as: begin{aligned} l psi _i) = - ln {P psi _i)} end{aligned} (24) Therefore, using (24) and removing all factors constant with respect to (psi _i), we can obtain the optimal loss functions associated to a given choice of noise distribution.
Four factors (choice of error metric, classifier, error estimator, and event balancing) were investigated by theoretical assessment, simulation study, and application to 7 human microarray datasets.
Fundamental to the assessment of predictive signal of molecular signatures is the choice of error metric that is used to quantify predictivity.
However the choice of classifier is not addressed as of crucial importance, the choice of error metric and estimator is not linked to statistical power, and event balancing as a source of bias and low power is not addressed.
However the simulation strengthens our hypothesis that the choice of error metric, classifier, event balancing and error estimator have large impact on study results and sheds light on the limitations of the analyses described in prior work [23].
