Mark Humphrys - Teaching - CA425


Search:   Help on Search

Artificial Intelligence

How to contact me

I prefer not to receive first contact by email. See What I think about email.

See How to contact me.

Course Descriptor

Notes

  1. Introduction - State-space control
    1. Introduction to AI
    2. Survey of AI
    3. AI Links
    4. Robotics Links
    5. History of AI

    6. Continuum of Autonomy
    7. State-space control
    8. RL as Pattern Classification

  2. Reinforcement Learning
    1. Formal introduction to Reinforcement Learning (Chapter 2 of my PhD)
      1. Notation
      2. Appendix A of PhD
      3. Appendix B of PhD
      4. Appendix C - 2-reward reward functions
      5. Appendix D - 3-reward (or more) reward functions

    2. Reinforcement Learning - Accompanying Notes
      1. Exercise
      2. How Q-learning works
      3. Convergence
      4. The control policy
      5. Boltzmann "soft max" distribution

    3. Program code
      1. How to make a decision probabilistically
      2. Coding the state-space as a lookup-table
      3. Sample code for lookup-table Q-learning

  3. Movie demo
    1. Movie demo of W-learning contains within it a demo of basic Q-learning.

  4. BACKGROUND READING - Some extra bits:
    1. Ch.7 - Rewarding on transitions or continuously
    2. Ch.18 - Feudal Q-learning

  5. NOT ON COURSE THIS YEAR - Reinforcement Learning with Neural Networks (Pre-requisite is CA300.)
    1. Neural Networks (Revision from CA300)
    2. Using a Neural Network as a generalisation in RL
    3. Q-learning with a Neural Network
    4. Ch.4 - Using a Neural Network with RL

  6. NOT ON COURSE - Multiple Minds
    1. Ch.3 - Multi-Module Reinforcement Learning
    2. Ch.4 - Multiple Minds in the same body - Test of Hierarchical Q-learning
    3. Ch.18 - The general form of a Society of Mind based on Reinforcement Learning
    4. Open Issues in AI
    5. Architectures of Autonomous Agents
    6. The World-Wide-Mind (my idea)

  7. Reinforcement Learning - Reference

Practical

Practical - Not yet ready to launch.

Deadline Fri 5 Dec 2008.

Recommended Reading

Experiments in Adaptive State-Space Robotics, Clocksin and Moore, 1989. - Online. - A simple introduction to the very idea of state-space robotic or agent control.

How to Make Software Agents Do the Right Thing: An Introduction to Reinforcement Learning, Singh et al, 1996. - Online. - A simple introduction to the idea of RL.

Action Selection methods using Reinforcement Learning, Humphrys, 1997 (my PhD thesis). - Online. - Chapter 2 is the more formal introduction to RL above.

Kaelbling et al (1996), "Reinforcement Learning: A Survey", Journal of Artificial Intelligence Research 4:237-285. - Online.

Reinforcement Learning: An Introduction, Sutton and Barto, 1998. - Bookshop, and Online (also here and here).

Library categories