AIC, defined as deviance plus 2 times the number of predictors, measures the predictive power; a model is estimated to reduce out-of-sample prediction error if AIC decreases.
Abadie et al. (2015, p. 502): "Intuitively, the cross-validation technique selects the weights v m that minimize out-of-sample prediction errors".
In a practical situation, the embedding dimension (d) and the number of nearest neighbours (k) are estimated by minimizing the in-sample prediction errors, where (din left{ {2,3,ldots,8} right} ) and (kin left{ {1,2,ldots,20} right} ) is the percentage of neighbours with respect to the sample size.
We then assessed the size and stability of out-of-sample prediction errors for each payer category and determined the correlation of the PCRs with traditional CCRs.
Increasing the sample size did not affect the overall ranking of the sampling strategies by prediction error; the scenarios n = 12, n = 18 and n = 6 all resulted in a preference for MAXVAR.
Although the predicted series are convergent, with the increasing of the testing samples' number, prediction errors have a tendency to increase Fig. 3 Prediction results using single-RBF predictor with data start point at 1st.
In Fig. 2, with the increase of the testing samples' number, prediction errors are tended to increase.
Even though the prediction filters are updated only once per block of samples, quantization of the prediction error is performed on a sample-by-sample basis.
In each segment i (i = 1,..., N), there are three steps: (1) taking sample i out as temporary 'test set', which is not used to develop the calibration model, (2) developing the calibration model with the remaining (N-1) samples, (3) testing the developed model with sample i, calculating and storing the prediction error of the sample.
This provide some interesting insights on the small sample performance of the prediction error method.
We fit the model with one observation deleted from the data at each time and used that observation as the test sample to estimate the prediction error.
