Boosting Resources

This page is under construction. Welcome to propose new papers or works by writing to me cao4@ifp."you-eye-you-see".edu. Your help will be cordially appreciated and acknowledged. Thank you!

[Updated]: Aug 25, 2008: Helmut’s online boosting, Schapire’s NIPS 07 tutorial

Original Papers and Surveys

Y Freund, RE Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, EuroCOLT, 1995
Y Freund, RE Schapire, Experiments with a new boosting algorithm, ICML, 1996
Robert E. Schapire, The boosting approach to machine learning: An overview, in Nonlinear Estimation and Classification. Springer, 2003
Thomas G. Dietterich. Ensemble learning. in The Handbook of Brain Theory and Neural Networks, Second Edition, 2002

Theoretical Understanding

L Mason, J Baxter, P Bartlett, M Frean, Boosting algorithms as gradient descent , NIPS, 2000
J Friedman, T Hastie, R Tibshirani, Additive logistic regression: a statistical view of boosting, The Annals of Statistics, 2000
J. Friedman, Greedy Function Approximation: A Gradient Boosting Machine, The Annals of Statistics, 2001

· J Friedman, Stochastic gradient boosting, Computational Statistics and Data Analysis, 2002

RE Schapire, Y Freund, P Bartlett, WS Lee, Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods, ICML 1997 ( a longer version appears in The Annals of Statistics, 1998)
G Ratsch, S Mika, B Scholkopf, KR Muller, Constructing boosting algorithms from SVMs: an application to one-class classification, PAMI 2002.

one class SVM

Tong Zhang, Bing Yu, Boosting with Early Stopping: Convergence and Consistency, The Annals of Statistics, 2005

How often does such result happen in real applciations?

P Buhlmann, Boosting for high-dimensional linear models, The Annals of Statistics, 2006

L₂Boost is consistent and efficient for computing cross-validation.

Cynthia Rudin, Robert E. Schapire, Ingrid Daubechies, Analysis of Boosting Algorithms using the Smooth Margin Function, Annals of Statistics, Accepted. [COLT 04 version]

based on Rudin's Dynamics of Adaboost work [NIPS03][JMLR04] [Video Talk]

Bagging, Random Forests, and Other Inspired Research

Leo Breiman, Bagging Predictors, Machine Learning, 1996
Leo Breiman, Random Forests, Machine Learning, 2001
JR Quinlan, Bagging, boosting, and C4. 5, AAAI, 1996
E Bauer, R Kohavi , An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants, Machine Learning, 1999
B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression, Annals of Statistics, 2004

Online Learning and Transfer Learning

Helmut Grabner, Horst Bischof, On-line Boosting and Vision, CVPR 2006
Helmut Grabner, Michael Grabner, Horst Bischof, Real-Time Tracking via On-line Boosting, BMVC, 2006
Liu, Xiao-Ming, Yu, Ting, Gradient Feature Selection for Online Boosting, ICCV07(1-8).
Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu, Boosting for Transfer Learning, ICML, 2007

Viola-Jone Detector and Other Applications

P. Viola and M. Jones, Robust real-time object detection, IJCV, 2002
P. Viola, M. Jones and D. Snow. Detecting Pedestrians using Patterns of Motion and Appearance. IJCV, 63(2): 153-161, 2005.
S. Avidan, Ensemble Tracking, CVPR 2005 (a longer version appears in PAMI, 2007)
Q. Zhu, M.C. Yeh, K-T Cheng and S. Avidan. Fast Human Detection Using a Cascade of Histograms of Oriented Gradients. CVPR, 2006

Optimizing Cascades

P. Viola and M. Jones. Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade. NIPS, 2002.
S. Charles Brubaker, Matthew D. Mullin and James M. Rehg. Towards Optimal Training of Cascaded Detectors. ECCV 2006.
Jianxin Wu, Matthew D. Mullin and James M. Rehg. Linear Asymmetric Classifier for Cascade Detectors. ICML 2005.

Rejection-based

Simon Baker and Shree K. Nayar. Pattern Rejection. CVPR, 1996.
François Fleuret and Donald Geman. Coarse-to-fine Face Detection. IJCV, 41: 85-107, 2001.

New Algorithms

Chang Huang, Haizhou Ai, Yuan Li and Shihong Lao. Vector Boosting for Rotation Invariant Multi-View Face Detection. ICCV 2005.
A. Demiriz and K.P. Bennett and J. Shawe-Taylor, Linear Programming Boosting via Column Generation, Machine Learning, 2002
Pierre Geurts, Louis Wehenkel, and Florence dAlcheBuc, Gradient Boosting for Kernelized Output Spaces, ICML 2007.
C. Liu and H. Y. Shum. Kullback-Leibler boosting. CVPR, 2003. (Liu's presentation)
Xun Xu and Thomas S. Huang, Face Recognition with MRC-Boosting, ICCV 2005. (also refer to Xu's SODA-Boosting for Gender Recognition, AMFG07)
A. Vedaldi, P. Favaro, and E. Grisan, Boosting Invariance and Efficiency in Supervised Learning, ICCV, 2007 [code]

· Lijuan Cai, Thomas Hofmann, Text categorization by boosting automatically extracted concepts, SIGIR, 2003

o Adaboost + pLSA

Multi-class, Multi-task, Tree Structure Boosting

Stan Z. Li, ZhenQiu Zhang, Heung-Yeung Shum, and HongJiang Zhang. FloatBoost for Multi-View Face Detetection. NIPS 2002, ECCV 2002, PAMI 2004, D.Hoiem's presentation.
Robert E. Schapire and Yoram Singer, Improved Boosting Algorithms Using Confidence-rated Predictions, Machine Learning, 2000
R.E. Schapire and Y. Singer, Boostexter: a boosting-based system for text categorization, Machine Learning, 2000
Matthew Johnson and Roberto Cipolla, Improved Image Annotation and Labelling through Multi-label Boosting, BMVC, 2005
A Esuli, and T Fagni, TreeBoost. MH: A Boosting Algorithm for Multi-label Hierarchical Text Categorization, String Processing and Information Retrieval, 2006
J. Zhu, S. Rosset, H. Zou, and T. Hastie, "Multi-class AdaBoost", Technical Report, Dept. Statistics University of Michigan, 2005

Ranking

Yoav Freund, Raj Iyer, Robert E. Schapire and Yoram Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933-969, 2003