Statistical Physics for Optimization and Learning course, Lecture Notes

To access the document, please click [PDF]

Updates

10/22 Corrected version of Chapters 4, 5, 6 & Appendix A

09/22: Corrected version of Chapters 1, 2, 3 & Appendices A, B

100 pages of lecture notes from the original course. Written to explore the topic and shape research interest under the suggestion of mentors from the Visiting Student initiative. The course, the videos and the original lecture notes are available here.

Abstract

Statistical Physics can be seen as a set of theoretical results and methods to describe and tackle the computational hurdles of large inference problems. Building on the great contributions from the ther- modynamics and statistical mechanics worlds, one can show that the same limiting properties apply for models spurring out of the two fields into computer science, physics, and machine learning. Such a for- malism allows to draw similarities of solutions across different questions.
The document is a redaction of lecture notes from the homonymous PhD course offered at École Polytechnique Fédérale de Lausanne by Professors Krzakala Florent and Zdeborová Lenka. While it mostly follows the videos and the lecture notes, it gives a different (less experienced, but self developed) structure, which is the result of autonomous understanding of the concepts explained.

Disclaimer 1 The document is subject to major updates. There are 50 TODO sections with potential expansions or missing proofs!
Disclaimer 2 Chapters are in the order of teaching. Images are entirely taken from the videos, and come from different sources, please refer to the website to find the exact origins. Hopefully, it will be fixed in future versions.