where κ m (a) denotes the amount of time slots included in the m-th last control interval, namely the reward function which depends on the action choice a. Base Policy In the rollout algorithm, a heuristic algorithm is needed to serve as the base policy, which is also designed on the basis of control interval structure.
The second approximation is based on l-step lookahead policy and rollout algorithm.
To enhance performance of the rollout algorithm, we employ constraint programming (CP) to improve the performance of base policy offered by a priority-rule heuristic.
Finally, we design and implement a matheuristic approach, integrating a rollout algorithm and an optimal solution of mixed-integer linear programming models, which is able to solve realistic size problem instances.
In particular, we present a parallel version of the rollout algorithm, an innovative heuristic method for solving NP-Hard combinatorial optimization problems, recently proposed by Bertsekas et al. [J. Heuristics 3 (1997) 245].
Instead of tracing the accurate value, the rollout algorithm can estimate the value function approximately.
By rolling out the base policy, the proposed rollout algorithm can approximately calculate the value function defined in the POMDP framework and reduces the times of channel switch.
Rollout algorithm [15], as an approximate dynamic programming methodology based on policy iteration ideas, has been successfully applied to various domains such as combinatorial optimization [18] and stochastic scheduling [19].
To obtain a suboptimal solution, a problem-dependent heuristics is proposed first as the base policy, and then the reward of the base policy can be used by the rollout algorithm in a one-step lookahead method to approximate the value function.
Rollout algorithms are applied to the heuristics to improve the solution quality.
In this research, we develop effective and efficient approximate dynamic programming (ADP) algorithms based on the rollout policy for this category of stochastic scheduling problems.
