Blog posts

Learning notes

Optimization

Stochastic Programming

• notes1: L-shaped Method from the Dual View — PDF (2026-4-13)

Machine Learning

Reinforcement Learning

• RL 7: value function approximation — PDF (2026-4-13)
• Convergence of Q-Learning via Stochastic Approximation — PDF (2025-11-18)
• RL 6: Learning from a Stream of Data 2 — Temporal Difference Learning (TD Error, SARSA, Q-Learning, SA Convergence) — PDF (2025-11-17)
• RL 5: Learning from a Stream of Data 1 — Reward Learning (Stochastic Approximation, Exploration–Exploitation) — PDF (2025-11-01)
• RL 4: Planning with a Known Model (Policy Evaluation, Value Iteration, Policy Iteration, LP Formulation) — PDF (2025-10-15)
• RL 3: Bellman Operators — Properties and Consequences (Monotonicity, Contraction, Fixed-Point Uniqueness, Error Bounds) — PDF (2025-09-22)
• RL 2: Bellman Equations and Operators (Greedy Policy, Banach Fixed Point, Contraction & Monotonicity) — PDF (2025-09-15)

Score-Based Generative Modeling

• SDE to Score-Based Generation (1) — Interactive note (Part I)

Paper notes

• Class-Balanced Loss
  Based on Cui et al., CVPR 2019 — PDF
• Graph Laplacian, Fourier Transform, and Spectral Filtering
  Notes on graph signal processing and spectral GCN derivations — PDF