Introduction -- The dynamic programming algorithm -- Applications in specific areas -- Problems with imperfect state information -- Computational aspects of dynamic programming-suboptimal control -- Minimization of total expected value-discounted cost -- Minimization of total expected value-undiscounted cost -- Minimization of average expected value -- Appendix A. Mathematical review -- Appendix B. On optimization theory -- Appendix C. On probability theory -- Appendix D. On finite state Markov chains.