This four part lecture was given at the Machine Learning Summer School held in Tübingen in 2003 organized by Olivier Bousquet, Bernhard Schölkopf and Ulrike von Luxburg. The lecture discusses Stochastic Approximations and in particular Stochastic Gradient Descent applied to online learning algorithms.

• See part 1: Framework (djvu 157KB) (pdf 2.1MB).
• See part 2: Cookbook (djvu 459KB) (pdf 2.7MB).
• See part 3: Convergence and Martingales (djvu 38KB) (pdf 1.3MB).
• See part 4: Optimal online algorithms (djvu 55KB) (pdf 1.4MB).

• See the corresponding book chapter.