STAT 6341
Numerical Linear Algebra and Statistical Computing
Term: Fall 2024, Fall 2025, Fall 2026
Instructor: Yun Wei
Course description
This graduate course covers numerical linear algebra and statistical computing, including matrix decomposition, optimization, and machine learning.
Textbooks
- Stephen Boyd and Lieven Vandenberghe, Convex Optimization.
- Amir Beck, First-Order Methods in Optimization .
Lecture notes
The lecture notes evolve during the semester. It only contains material from the lectures that has no slides provided.
Schedule (Fall 2026)
| Date | Topic |
|---|---|
| August 24 | Multi-Armed Bandit and epsilon greedy algorithm, Foster and Rakhlin Section 2.1-2.2 Scribed lecture notes |
| August 26 | epsilon greedy and UCB algorithms, Foster and Rakhlin Section 2.2-2.3 Scribed lecture notes |
| August 31 | UCB algorithm, Foster and Rakhlin Section 2.2-2.3; Convex functions, Boyd and Vandenberghe Sec 3.1 Scribed lecture notes |
| September 2 | Convex functions, Boyd and Vandenberghe Sec 3.1; Bisection and Newton’s method; strong convexity, Boyd and Vandenberghe Sec 9.1.2 (slide) |
| September 9 | Strong convexity, Boyd and Vandenberghe Sec 9.1.2 (slide) |
| September 14 | Ridge Regression (slide) |
| September 16 | Ridge Regression: GD, SGD (slide) |
| September 21 | SGD; and robust regression: MM algorithms; constrained optimization (Boyd and Vandenberghe Sec 5.1-5.2) (slide) |
| September 23 | constrained optimization and KKT condition (Boyd and Vandenberghe Sec 5.1, 5.2, 5.5) (slide) |
| September 28 | equality constrained minimization problems (Boyd and Vandenberghe Sec 10.1) Scribed lecture notes |
| September 30 | equality constrained minimization problems (Boyd and Vandenberghe Sec 10.1) and Newton’s method (sec 9.5) Scribed lecture notes |
| October 5 | Newton’s method (Boyd and Vandenberghe sec 9.5) and Newton’s method with equality constraint ( Sec 10.2) Scribed lecture notes |
| October 7 | PCA Guest lecture, Scribed lecture notes |
| October 12 | Vector space, norms, dual space (Beck Chapter 1) Scribed lecture notes |
| October 14 | dual space (Beck Sec 1.11), closed functions (Beck Section 2.1) and Subgradients (Beck Sec. 3.1) Scribed lecture notes |
| October 19 | Subgradients (slide) |
| October 21 | Subgradients (slide) |
| October 26 | Subgradients (slide); Subgradient method (slide) |
| October 28 | Subgradient method (slide); Proximal gradient method (slide) |
| Nov 2 | Revisit convex functions, L-smooth functions, strongly convex functions (slide) |