基于最大间隔的贝叶斯分类器

Tao-cheng HU; Jin-hui YU

doi:10.1631/FITEE.1601078

Your Location：

Home >

Browse articles >

基于最大间隔的贝叶斯分类器

常规文章 | Updated：2022-05-19

- 基于最大间隔的贝叶斯分类器
- Max-margin based Bayesian classifier
- 信息与电子工程前沿（英文） 2016年17卷第10期页码：973-981
- Affiliations：
  
  State Key Lab of CAD & CG, Zhejiang University, Hangzhou 310058, China
- Author bio：
- Funds：
  
  Project supported by the National Natural Science Foundation of China (No. 61379069), the Major Program of the National Social Science Foundation of China (No. 12&ZD231), and the National Key Technology R&D Program of China (No. 2014BAK09B04)
- DOI：10.1631/FITEE.1601078
  中图分类号： TP181
- 收稿：2016-03-10，
  
  录用：2016-9-14，
  
  纸质出版：2016-10
- Accepted：
Scan QR Code
基于最大间隔的贝叶斯分类器[J]. 信息与电子工程前沿（英文）, 2016,17(10):973-981.

Tao-cheng HU, Jin-hui YU. Max-margin based Bayesian classifier[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(10): 973-981.
基于最大间隔的贝叶斯分类器[J]. 信息与电子工程前沿（英文）, 2016,17(10):973-981. DOI： 10.1631/FITEE.1601078.

Tao-cheng HU, Jin-hui YU. Max-margin based Bayesian classifier[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(10): 973-981. DOI： 10.1631/FITEE.1601078.

摘要

多分类学习中经常需要考虑在泛化性能和计算开销间进行权衡。本文提出一个生成式概率多分类器，综合考虑了泛化性和学习/预测速率。我们首先证明了我们的分类器具有最大间隔性质，这意味着对于未来数据的预测精度几乎和训练阶段一样高。此外，我们消除了目标函数中的大量的局部变元，极大地简化了优化问题。通过凸分析和概率语义分析，我们设计了高效的在线算法，与经典情形的最大不同在于这个算法使用聚集而非平均化处理梯度。实验证明了我们的算法具有很好的泛化性能和收敛速度。

Abstract

There is a tradeoff between generalization capability and computational overhead in multi-class learning. We propose a generative probabilistic multi-class classifier

considering both the generalization capability and the learning/prediction rate. We show that the classifier has a max-margin property. Thus

prediction on future unseen data can nearly achieve the same performance as in the training stage. In addition

local variables are eliminated

which greatly simplifies the optimization problem. By convex and probabilistic analysis

an efficient online learning algorithm is developed. The algorithm aggregates rather than averages dualities

which is different from the classical situations. Empirical results indicate that our method has a good generalization capability and coverage rate.

关键词

Keywords

references

A Agarwal , , , S.M Kakade , , , N Karampatziakis , , , 等 . . Least squares revisited: calable approaches for multiclass prediction . . 2014 . . Proc. Int. Conf. on Machine Learning . . 541 - - 549 . . . .

C.M Bishop . . Pattern Recognition and Machine Learning , , 2006 . . : : New York, USA Springer , , . .

D.M Blei , , , A.Y Ng , , , M.I Jordan . . Latent Dirichlet allocation . . J. Mach. Learn. Res. , , 2003 . . 3 ( ( Jan ): ): 993 - - 1022 . . . .

S Boyd , , , L Vandenberghe . . Convex Optimization , , 2004 . . : : Cambridge, UK Cambridge University Press , , . .

Q Cai , , , Y.F Yin , , , H Man . . DSPM: dynamic structure preserving map for action recognition . . 2013 . . IEEE Int. Conf. on Multimedia and Expo . . 1 - - 6 . . DOI: 10.1109/ICME.2013.6607606 http://doi.org/10.1109/ICME.2013.6607606 . .

A Coates , , , H Lee , , , A.Y Ng . . An analysis of singlelayer networks in unsupervised feature learning . . 2011 . . Int. Conf. on Artificial Intelligence and Statistics . . 215 - - 223 . . . .

A Daniely , , , S Shalev-Shwartz . . Optimal learners for multiclass problems . . 2014 . . Proc. Conf. on Learning Theory . . 287 - - 316 . . . .

J Duchi , , , E Hazan , , , Y Singer . . Adaptive subgradient methods for online learning and stochastic optimization . . J. Mach. Learn. Res. , , 2011 . . 12 2121 - - 2159 . . . .

M Galar , , , A Fernández , , , E Barrenechea , , , 等 . . An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes . . Patt. Recogn. , , 2011 . . 44 ( ( 8 ): ): 1761 - - 1776 . . DOI: 10.1016/j.patcog.2011.01.017 http://doi.org/10.1016/j.patcog.2011.01.017 . .

E Hazan , , , A Rakhlin , , , P.L Bartlett . . Adaptive online gradient descent . . 2007 . . In: Platt, J.C., Koller, D., Singer, Y., et al. (Eds.), Advances in Neural Information Processing Systems 20. MIT Press, Canada . . 65 - - 72 . . . .

T.C Hu , , , J.H Yu . . Generalized entropy based semi-supervised learning . . 2015 . . IEEE/ACIS Int. Conf. on Computer and Information Science . . 259 - - 263 . . DOI: 10.1109/ICIS.2015.7166603 http://doi.org/10.1109/ICIS.2015.7166603 . .

T.C Hu , , , J.H Yu . . Incremental max-margin learning for semi-supervised multi-class problem . . Stud. Comput. Intell. , , 2016 . . 612 31 - - 43 . . DOI: 10.1007/978-3-319-23509-7_3 http://doi.org/10.1007/978-3-319-23509-7_3 . .

T Jebara . . Machine learning: discriminative and generative. In: Meila, M. (Ed.), the Kluwer International Series in Engineering and Computer Science , , 2004 . . : : Germany Kluwer Academic , , . .

Y LeCun , , , L Bottou , , , Y Bengio , , , 等 . . Gradientbased learning applied to document recognition . . Proc. IEEE , , 1998 . . 86 ( ( 11 ): ): 2278 - - 2324 . . . .

S.A Nene , , , S.K Nayar , , , H Murase . . Columbia Object Image Library (COIL-20) , , 1996a . . Available from: http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php [Accessed on Feb. 1, 2016] . .

S.A Nene , , , S.K Nayar , , , H Murase . . Columbia Object Image Library (COIL-100) , , 1996b . . Available from: http://www.cs.columbia.edu/CAVE/software/softlib/coil-100.php http://www.cs.columbia.edu/CAVE/software/softlib/coil-100.php [Accessed on Feb. 1, 2016] . .

A Rahimi , , , B Recht . . Random features for large-scale kernel machines . . 2007 . . In: Platt, J.C., Koller, D., Singer, Y., et al. (Eds.), Advances in Neural Information Processing Systems 20. MIT Press, Canada . . 1177 - - 1184 . . . .

H.G Ramaswamy , , , B.S Babu , , , S Agarwal , , , 等 . . On the consistency of output code based learning algorithms for multiclass learning problems . . 2014 . . Proc. Conf. on Learning Theory . . 885 - - 902 . . . .

S Shalev-Shwartz . . Online learning: theory, algorithms and applications , , 2007 . . : : Jerusalem, Israel Hebrew University , , PhD Thesis . .

S Shalev-Shwartz , , , S.M Kakade . . Mind the duality gap: logarithmic regret algorithms for online optimization . . 2009 . . In: Koller, D., Schuurmans, D., Bengio, Y. (Eds.), Advances in Neural Information Processing Systems 21. MIT Press, Canada . . 1457 - - 1464 . . . .

N Srebro , , , K Sridharan , , , A Tewari . . On the universality of online mirror descent . . 2011 . . In: Saul, L.K., Weiss, Y., Bottou, L. (Eds.), Advances in Neural Information Processing Systems 17. MIT Press, Canada . . 2645 - - 2653 . . . .

J Zhu . . Max-margin nonparametric latent feature models for link prediction . . 2012 . . Proc. Int. Conf. on Machine Learning . . 719 - - 726 . . . .

J Zhu , , , E.P Xing . . Maximum entropy discrimination Markov networks . . J. Mach. Learn. Res. , , 2009 . . 10 ( ( Nov ): ): 2531 - - 2569 . . . .

J Zhu , , , N Chen , , , E.P Xing . . Infinite latent SVM for classification and multi-task learning . . 2011 . . In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., et al. (Eds.), Advances in Neural Information Processing Systems 24. MIT Press, Canada . . 1620 - - 1628 . . . .

J Zhu , , , N Chen , , , H Perkins , , , 等 . . Gibbs maxmargin topic models with fast sampling algorithms . . 2013 . . Proc. Int. Conf. on Machine Learning . . 124 - - 132 . . . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据