Unit 1: Introduction, Univariate Problems
Feb 3: Lecture 1 — Introduction
Contents:
- Course Organization
- Optimization problems
- Classification
- Constraints
- Critical Points
- Conditions for Local Minima
- Contour Plots
- Asymptotic Notation
- Taylor Expansion
- Convexity
- Norms
- Matrix Calculus
- Positive Definiteness
- Gaussian Distribution
Resources:
- Slides
- Chapter 1 of [KW]
Feb 6: Lecture 2 — Derivatives and Gradients
Contents:
- Derivatives
- Derivatives in Multiple Dimensions
- Numerical Differentiation
- Automatic Differentiation
Resources:
- Slides
- Chapter 2 and Appendix of [KW]
Feb 10: Lecture 3 — Bracketing
Contents:
- Unimodality
- Finding an Initial Bracket
- Fibonacci Search
- Golden Section Search
- Quadratic Fit Search
- Shubert-Piyavskii Method
- Bisection Method
Resources