STAT 6390
High Dimensional Probability and Statistics
Term: Fall 2026
Instructor: Yun Wei
Course description
Modern statistical applications often involve data sets in which the number of variables is comparable to, or substantially larger than, the number of observations. This course develops the mathematical and statistical foundations needed to analyze high-dimensional data.
Topics include concentration inequalities, random vectors and random matrices, high-dimensional geometry, sparse linear models, regularization methods, the Lasso, model selection, low-rank matrix estimation, principal component analysis, graphical models, minimax lower bounds, and computational considerations.
Particular emphasis is placed on how structural assumptions, including sparsity, low rank, and graphical structure, make statistically meaningful estimation possible in high dimensions. Students will use nonasymptotic techniques to derive finite-sample guarantees and evaluate estimators by consistency, prediction error, estimation error, and variable-selection performance.
Textbooks
- Martin J. Wainwright, High-Dimensional Statistics: A Non-Asymptotic Viewpoint.
- Roman Vershynin, High-Dimensional Probability (optional).
- Richard J. Samworth, Modern Statistical Methods and Theory: An Introduction to Nonparametric and High-Dimensional Statistics (optional)
Other similar courses:
- Clay Scott, Statistical Learning Theory, https://web.eecs.umich.edu/~cscott/past_courses/eecs598w14/index.html
Lecture notes
The lecture notes evolve during the semester.
Schedule
| Date | Topic |
|---|---|
| August 24 | SubGaussian (Sec 2.1.1 and Sec 2.1.2) Scribed lecture notes |
| August 26 | SubGaussian (Sec 2.1.2) and Sub-exponential (Sec 2.1.3) Scribed lecture notes |
| August 31 | Symmetrization (Vershynin’s book Sec 6.3) and Sub-exponential (Sec 2.1.3) Scribed lecture notes |
| September 2 | Sub-exponential, Bernstein Bounds, Jonson-Lindenstrauss Lemma (Sec 2.1.3) Scribed lecture notes |
| September 9 | Sub-exponential (Sec 2.1.3) and Azuma-Hoeffding inequality (Sec 2.2.2) Scribed lecture notes |
| September 14 | Bounded difference inequality (Sec 2.2.2) Scribed lecture notes |
| September 16 | Liptchize function of Gaussian variables (Sec 2.3) and Entropic methods (Sec 3.1) Scribed lecture notes |
| September 21 | Entropic methods (Sec 3.1.2) and Wasserstein distances (Sec 3.3.1) Scribed lecture notes |
| September 23 | Wasserstein distances (Sec 3.3.1, Sec 3.3.2) and Uniform LLN (Sec 4.1) Scribed lecture notes |
| September 28 | Uniform Law of Large Number and empirical risk minimization (Sec 4.1 ) Scribed lecture notes |
| September 30 | Uniform Law of Large Number; empirical processes, and Rademacher Complexity (Sec 4.1 and Sec 4.2) Scribed lecture notes |
| October 5 | Rademacher Complexity and VC dimension (Sec 4.2 and Sec 4.3) Scribed lecture notes |
| October 7 | Covering and packing (Sec 5.1) Guest lecture, Scribed lecture notes |
| October 12 | VC dimension (Sec 4.3) Scribed lecture notes |
| October 14 | VC dimension and Covering and packing (Sec 4.3 and Sec 5.1) Scribed lecture notes |
| October 19 | Covering and packing (Sec 5.1); Gaussian and Rademacher complexity (Sec 5.2) Scribed lecture notes |
| October 21 | Gaussian and Rademacher complexity (Sec 5.2); sub-Gaussian processes (Sec 5.3) Scribed lecture notes |
| October 26 | sub-Gaussian processes and Dudley integral bound (Sec 5.3) Scribed lecture notes |
| October 28 | sub-Gaussian processes and Dudley integral bound (Sec 5.3) |
| Nov 2 | Preliminary on linear algebra (Section 6.1) and Wishart matrices (Sec 6.2) |