Stochastic Optimization Notes

A list of my Reinforcement Learning notes (PDF).

• RL 7: value function approximation — PDF (2026-4-13)
• Convergence of Q-Learning via Stochastic Approximation — PDF (2025-11-18)
• RL 6: Learning from a Stream of Data 2 — Temporal Difference Learning (TD Error, SARSA, Q-Learning, SA Convergence) — PDF (2025-11-17)
• RL 5: Learning from a Stream of Data 1 — Reward Learning (Stochastic Approximation, Exploration–Exploitation) — PDF (2025-11-01)
• RL 4: Planning with a Known Model (Policy Evaluation, Value Iteration, Policy Iteration, LP Formulation) — PDF (2025-10-15)
• RL 3: Bellman Operators — Properties and Consequences (Monotonicity, Contraction, Fixed-Point Uniqueness, Error Bounds) — PDF (2025-09-22)
• RL 2: Bellman Equations and Operators (Greedy Policy, Banach Fixed Point, Contraction & Monotonicity) — PDF (2025-09-15)