Overall, the ensemble model results are skillful, with an all site average skill score of about 0.4.
The results are compared using multiple verification metrics and skill scores: relative mean error, Brier skill score and its decompositions, mean continuous ranked probability skill score and its decomposition, and the ROC score.
The Linear Error in Probability Space (LEPS) skill score, Brier Skill Score (BSS), relative operating characteristic, and reliability diagrams were used to evaluate the skill of downscaled probabilistic forecasts.
We also introduce the probabilistic performance measures sharpness, reliability and interval skill score for model comparison and for checking the reliability of the confidence bounds.
In terms of ensemble spread, both the Brier skill scores and continuous ranked probability skill score are improved appreciably after post-processing.
The Pearson correlation coefficient (R), mean squared error skill score (MSESS), and Bias were used to evaluate the skill of downscaled deterministic forecasts.
Prediction accuracies of the model are evaluated using three different statistical measures: the probability of detection, the false alarm ratio, and the true skill score or R score.
The model is required to be calibrated regularly to ensure higher degree of forecast accuracy in terms of Heidke skill score (HSS).
The model demonstrated good quantitative skill (Brier skill score, BSS, typically 0.65) with respect to beach face erosion, offshore accretion and morphological impacts of washover.
In comparing the GA forecasts with observations, we obtain values of the Hanssen Kuiper skill score ranging from 0.463 to 0.727, which are usual values for similar forecasts of space weather.
The model efficiency and the predictive performance were quantified using the root mean squared error (RMSE), Nash Sutcliffe model efficiency coefficient (NSE), anomaly correlation coefficient (ACC) and mean square skill score (MSSS).
