http://leon.bottou.org/ 2008-08-27T22:00:56-04:00 http://leon.bottou.org/ http://leon.bottou.org/lib/images/favicon.ico text/html 2008-08-18T15:00:29-04:00 Leon Bottou http://leon.bottou.org/papers?rev=1219086029&do=diff1219086029 Follow each publication link to access papers and supplemental data. Most papers are available in DjVu, PDF, and PS.GZ. Download a DjVu viewer. 2008 Sequence Labelling SVMs Trained in One PassMachine Learning and Knowledge Discovery in Databases: ECML PKDD 2008 text/html 2008-08-18T14:59:41-04:00 Leon Bottou http://leon.bottou.org/papers/bottou-mlss-2004?rev=1219085981&do=diff1219085981 Stochastic Learning This paper summarizes my lecture at the Machine Learning Summer School 2003, Tübingen. Abstract: This contribution presents an overview of the theoretical and practical aspects of the broad family of learning algorithms based on Stochastic Gradient Descent, including Perceptrons, Adalines, K-Means, LVQ, Multi-Layer Networks, and Graph Transformer Networks. text/html 2008-08-18T14:57:25-04:00 Leon Bottou http://leon.bottou.org/papers/bordes-usunier-bottou-2008?rev=1219085845&do=diff1219085845 Sequence Labelling SVMs Trained in One Pass Abstract: This paper proposes an online solver of the dual formulation of support vector machines for structured output spaces. We apply it to sequence labelling using the exact and greedy inference schemes. In both cases, the per-sequence training time is the same as a perceptron based on the same inference procedure, up to a small multiplicative constant. Comparing the two inference schemes, the greedy version is much faster. It is also amenable to… text/html 2008-08-18T10:43:29-04:00 Leon Bottou http://leon.bottou.org/papers/sonnenburg-2007?rev=1219070609&do=diff1219070609 The Need for Open Source Software in Machine Learning Abstract: Open source tools have recently reached a level of maturity which makes them suitable for building large-scale real-world systems. At the same time, the field of machine learning has developed a large body of powerful learning algorithms for diverse applications. However, the true potential of these methods is not used, since existing implementations are not openly shared, resulting in software with low usability, and weak inter… text/html 2008-08-18T10:38:39-04:00 Leon Bottou http://leon.bottou.org/papers/bottou-bousquet-2008b?rev=1219070319&do=diff1219070319 Learning Using Large Datasets Abstract: This contribution develops a theoretical framework that takes into account the effect of approximate optimization on learning algorithms. The analysis shows distinct tradeoffs for the case of small-scale and large-scale learning problems. Small-scale learning problems are subject to the usual approximation–estimation tradeoff. Large-scale learning problems are subject to a qualitatively different tradeoff involving the computational complexity of the… text/html 2008-08-18T10:31:03-04:00 Leon Bottou http://leon.bottou.org/papers/bottou-bousquet-2008?rev=1219069863&do=diff1219069863 The Tradeoffs of Large Scale Learning Abstract: This contribution develops a theoretical framework that takes into account the effect of approximate optimization on learning algorithms. The analysis shows distinct tradeoffs for the case of small-scale and large-scale learning problems. Small-scale learning problems are subject to the usual approximation--estimation tradeoff. Large-scale learning problems are subject to a qualitatively different tradeoff involving the computational complexity o… text/html 2008-04-23T10:53:42-04:00 Leon Bottou http://leon.bottou.org/papers/loosli-canu-bottou-2006?rev=1208962422&do=diff1208962422 Training Invariant Support Vector Machines using Selective Sampling Abstract: Bordes et al (2005) describe the efficient online LASVM algorithm using selective sampling. On the other hand, Loosli et al. (2005) propose a strategy for handling invariance in SVMs, also using selective sampling. This paper combines the two approaches to build a very large SVM. We present state-of-the-art results obtained on a handwritten digit recognition problem with 8 millions examples on a single processor.… text/html 2008-04-07T18:41:40-04:00 Leon Bottou http://leon.bottou.org/projects/sgd?rev=1207608100&do=diff1207608100 Stochastic Gradient Descent (SGD) has been historically associated with back-propagation algorithms in multilayer neural networks. These nonlinear nonconvex problems can be very difficult. Therefore it is useful to see how Stochastic Gradient Descent performs on simple linear and convex problems such as linear Support Vector Machines (SVMs) or Conditional Random Fields (CRFs).