Material Detail
Hierarchical Model-Based Reinforcement Learning
This video was recorded at 25th International Conference on Machine Learning (ICML), Helsinki 2008. Hierarchical decomposition promises to help scale reinforcement learning algorithms naturally to real-world problems by exploiting their underlying structure. Model-based algorithms, which provided the first finite-time convergence guarantees for reinforcement learning, may also play an important role in coping with the relative scarcity of data in large environments. In this paper, we introduce an algorithm that fully integrates modern hierarchical and model-learning methods in the standard reinforcement learning setting. Our algorithm, R-maxq, inherits the efficient model-based exploration of the R-max algorithm and the opportunities for abstraction provided by the MAXQ framework. We analyze the sample complexity of our algorithm, and our experiments in a standard simulation environment illustrate the advantages of combining hierarchies and models.
Quality
- User Rating
- Comments
- Learning Exercises
- Bookmark Collections
- Course ePortfolios
- Accessibility Info
More about this material
- Mobile Compatibility:
- Not specified at this time
- Language:
- English
- Cost Involved:
- No
- Source Code Available:
- No
- Accessibility Information Available:
- Unknown
- Creative Commons:
-
This work is licensed under a Attribution-NonCommercial-NoDerivs 3.0 United States
