Specifically, we explore the robustness of a learning-based ADP method, Sarsa, with a GARCH 1,1) demand model, and provide empirical comparison between Sarsa and two simulation-based ADP methods: Rollout and Hindsight Optimization (HO).
In particular, we present a parallel version of the rollout algorithm, an innovative heuristic method for solving NP-Hard combinatorial optimization problems, recently proposed by Bertsekas et al. [J. Heuristics 3 (1997) 245].
Ghosh also expressed worry on "apparently now active discussion of other culling methods for any wider rollout, such as gassing and snaring: both have strong experimental evidence bases calling into question whether they can be humane".
In the rollout-based schemes, the basis heuristics affects the performance of its corresponding rollout policy, and the differential training method improves the robustness to the approximation error.
It is obvious that, in a stochastic environment, the Monte Carlo method of computing the rollout policy is particularly sensitive to the approximation error, which is closely related to the number of trajectories.
To obtain a suboptimal solution, a problem-dependent heuristics is proposed first as the base policy, and then the reward of the base policy can be used by the rollout algorithm in a one-step lookahead method to approximate the value function.
By the differential training of the rollout policy, the approximation error caused by Monte Carlo method can be reduced a lot and the proposed suboptimal spectrum sensing and access scheme performs more robustly.
The OBR is assuming there will be another delay in the rollout of universal credit, the government's flagship new method of paying benefits.
Hopefully this relatively straightforward and inexpensive method of thought detection will see a rollout sometime soon in hospitals and long-term care facilities.
In the differential training method, we estimate the relative Q-factor difference rather than absolute Q-factor value, which is a suitable improvement of the recursively generating rollout policy in the context of Monto Carlo-based policy iteration methods.
