0.0001 kcal/(mol·Å) was chosen as the cutoff of the root-mean-squared gradient and maximum iterations was 1000 with their defaulted parameters.
The parameters which can be modified, aiming to optimize the training procedure with Levenberg Marquardt method of each network are the adaptive Marquardt value mu, the mu decrease and mu increase factors, the final target mean squared error, the minimum gradient value and the maximum mu value.
The parameters which can be modified to optimize each network's training procedure with the Bayesian method are the adaptive Marquardt mu, the mu decrease and mu increase factors, the final target mean squared error, the minimum gradient value and the maximum mu value.
Generally, training stops when one of the following conditions is fulfilled: the maximum epoch number is reached, the maximum training time is reached, the desired mean squared error is achieved, the gradient value becomes smaller than its minimum value and as mentioned above when the adaptive Marquardt overruns its maximum value.
A stochastic gradient descent with the mean squared error function was used as the learning algorithm.
The algorithm employs iterative mean squared error minimization using least squares curve fitting [64].
Root mean squared error is defined as the square root of the mean squared error, which is the average squared difference between the estimated treatment effect and the true parameter.
The results showed that for the same mean squared error (0.2204) the Levenberg Marquardt training algorithm is much faster than the gradient descent.
To train each ANN, we employed the scaled conjugate gradient method for learning the neuron weights and the mean squared error as a criterion function.
A modified powered exponential correlation function is introduced, which in conjunction with a gradient based space-filling design resulted in an averaged 8 times smaller mean squared error compared to other correlation design combinations.
The gradient, ∇ n, with respect to A n of the cost function, the mean squared error (MSE), is derived and used to adjust A n in the direction of decreasing MSE.
