TY - GEN
T1 - Two new regularized AdaBoost algorithms
AU - Sun, Yijun
AU - Li, Jian
AU - Hager, William
PY - 2004
Y1 - 2004
N2 - AdaBoost rarely suffers from overfitting problems in low noise data cases. However, recent studies with highly noisy patterns clearly showed that overfitting can occur. A natural strategy to alleviate the problem is to penalize the distribution skewness in the learning process to prevent several hardest examples from spoiling decision boundaries. In this paper, we describe in detail how a penalty scheme can be pursued in the mathematical programming setting as well as in the Boosting setting. By using two smooth convex penalty functions, two new soft margin concepts are defined and two new regularized AdaBoost algorithms are proposed. The effectiveness of the proposed algorithms is demonstrated through a large scale experiment. Compared with other regularized AdaBoost algorithms, our methods can achieve at least the same or much better performances.
AB - AdaBoost rarely suffers from overfitting problems in low noise data cases. However, recent studies with highly noisy patterns clearly showed that overfitting can occur. A natural strategy to alleviate the problem is to penalize the distribution skewness in the learning process to prevent several hardest examples from spoiling decision boundaries. In this paper, we describe in detail how a penalty scheme can be pursued in the mathematical programming setting as well as in the Boosting setting. By using two smooth convex penalty functions, two new soft margin concepts are defined and two new regularized AdaBoost algorithms are proposed. The effectiveness of the proposed algorithms is demonstrated through a large scale experiment. Compared with other regularized AdaBoost algorithms, our methods can achieve at least the same or much better performances.
UR - https://www.scopus.com/pages/publications/21244483107
U2 - 10.1109/ICMLA.2004.1383492
DO - 10.1109/ICMLA.2004.1383492
M3 - Conference contribution
AN - SCOPUS:21244483107
SN - 0780388232
SN - 9780780388239
T3 - Proceedings of the 2004 International Conference on Machine Learning and Applications, ICMLA '04
SP - 41
EP - 48
BT - Proceedings of the 2004 International Conference on Machine Learning and Applications, ICMLA '04
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 3rd International Conference on Machine Learning and Applications, ICMLA 2004
Y2 - 16 December 2004 through 18 December 2004
ER -