A new constrained maximum margin approach to discriminative learning of Bayesian classifiers

Ke GUO; Xia-bi LIU; Lun-hao GUO; Zong-jie LI; Zeng-min GENG

doi:10.1631/FITEE.1700007

Your Location：

Home >

Browse articles >

A new constrained maximum margin approach to discriminative learning of Bayesian classifiers

Regular Papers | Updated：2022-05-19

- A new constrained maximum margin approach to discriminative learning of Bayesian classifiers
- 基于带约束最大间隔的贝叶斯分类器判别学习方法
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 5, Pages: 639-650(2018)
- Affiliations：
  
  Beijing Laboratory of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing 100081, China
  Computer Information Center, Beijing Institute of Fashion Technology, Beijing 100029, China
- Author bio：
  
  [ "Ke GUO, guoke@bit.edu.cn" ]
  [ "Xia-bi LIU, liuxiabi@bit.edu.cn" ]
  [ "Lun-hao GUO, guolunhao@bit.edu.cn" ]
  [ "Zong-jie LI, leezongjie@163.com" ]
  Zeng-min GENG, E-mail: jsjgzm@bift.edu.cn
- Funds：
- DOI：10.1631/FITEE.1700007
  CLC： TP391
- Received：04 January 2017，
  
  Revised：2018-;5-10，
  
  Published：2018-05
- Accepted：
Scan QR Code
Ke GUO, Xia-bi LIU, Lun-hao GUO, et al. A new constrained maximum margin approach to discriminative learning of Bayesian classifiers[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(5): 639-650.
DOI：

Ke GUO, Xia-bi LIU, Lun-hao GUO, et al. A new constrained maximum margin approach to discriminative learning of Bayesian classifiers[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(5): 639-650. DOI： 10.1631/FITEE.1700007.

摘要

提出一种新的面向贝叶斯模式分类的判别学习方法，称作"带约束的最大间隔（CMM）方法"。通过计算正样本最小决策值和负样本最大决策值的差异，定义类别之间的类别间隔。基于该类别间隔和正确分类的约束，将间隔函数学习问题转化为最大化类别间隔问题。利用序列无约束最小化技术解决该非线性规划问题。运用CMM方法得到基于高斯混合模型的贝叶斯分类器，并在10个UCI数据集上进行实验。结果表明，利用CMM方法得到的分类器分类性能，明显优于代表性的生成式学习方法期望最大化（EM）和判别式学习方法支持向量机（SVM），并且在多个数据集上取得了相比之前最优结果更好的效果。分类实验和分类器对比实验证明，CMM方法有效，具有一定应用前景。

Abstract

We propose a novel discriminative learning approach for Bayesian pattern classification

called 'constrained maximum margin (CMM)'. We define the margin between two classes as the difference between the minimum decision value for positive samples and the maximum decision value for negative samples. The learning problem is to maximize the margin under the constraint that each training pattern is classified correctly. This nonlinear programming problem is solved using the sequential unconstrained minimization technique. We applied the proposed CMM approach to learn Bayesian classifiers based on Gaussian mixture models

and conducted the experiments on 10 UCI datasets. The performance of our approach was compared with those of the expectation-maximization algorithm

the support vector machine

and other state-of-the-art approaches. The experimental results demonstrated the effectiveness of our approach.

关键词

Keywords

references

J Alcalá-Fdez , , , L Sanchez , , , S Garcia , , , 等 . . KEEL: a software tool to assess evolutionary algorithms for data mining problems . . Soft Comput , , 2009 . . 13 ( ( 3 ): ): 307 - - 318 . . DOI: 10.1007/s00500-008-0323-y http://doi.org/10.1007/s00500-008-0323-y . .

J Alcalá-Fdez , , , A Fernández , , , J Luengo , , , 等 . . KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework . . J Multi-Valued Log Soft Comput , , 2011 . . 17 ( ( 2-3 ): ): 255 - - 287 . . . .

EJ Bredensteiner , , , KP Bennett . . Multicategory classification by support vector machines . . In: Pang JS (Ed.), Computational Optimization. Springer US, New York , , 1999 . . p.53 - - 79 . . DOI: 10.1007/978-1-4615-5197-3_5 http://doi.org/10.1007/978-1-4615-5197-3_5 . .

AP Dempster , , , NM Laird , , , DB Rubin . . Maximum likelihood from incomplete data via the EM algorithm . . J R Stat Soc B , , 1977 . . 39 ( ( 1 ): ): 1 - - 38 . . . .

J Demšar . . Statistical comparisons of classifiers over multiple data sets . . J Mach Learn Res , , 2006 . . 7 ( ( Jan ): ): 1 - - 30 . . . .

W Dong , , , M Zhou . . Gaussian classifier-based evolutionary strategy for multimodal optimization . . IEEE Trans Neur Netw Learn Syst , , 2014 . . 25 ( ( 6 ): ): 1200 - - 1216 . . DOI: 10.1109/TNNLS.2014.2298402 http://doi.org/10.1109/TNNLS.2014.2298402 . .

J Dvořák , , , P Savick . . Softening splits in decision trees using simulated annealing . . Int Conf on Adaptive and Natural Computing Algorithms , , 2007 . . p.721 - - 729 . . DOI: 10.1007/978-3-540-71618-1_80 http://doi.org/10.1007/978-3-540-71618-1_80 . .

AV Fiacco , , , GP McCormick . . Nonlinear Programming: Sequential Unconstrained Minimization Techniques . . SIAM, Philadelphia , , 1990 . . DOI: 10.1137/1.9781611971316 http://doi.org/10.1137/1.9781611971316 . .

GE Forsythe , , , MA Malcolm , , , CB Moler . . Computer Methods for Mathematical Computations (1 st Ed.) . . Prentice Hall, New Jersey , , 1977 . . .

N Friedman , , , D Geiger , , , M Goldszmidt . . Bayesian network classifiers . . Mach Learn , , 1997 . . 29 ( ( 2-3 ): ): 131 - - 163 . . DOI: 10.1023/A:1007465528199 http://doi.org/10.1023/A:1007465528199 . .

RP Gorman , , , TJ Sejnowski . . Analysis of hidden units in a layered network trained to classify sonar targets . . Neur Netw , , 1988 . . 1 ( ( 1 ): ): 75 - - 89 . . DOI: 10.1016/0893-6080(88)90023-8 http://doi.org/10.1016/0893-6080(88)90023-8 . .

M Hall , , , E Frank , , , G Holmes , , , 等 . . The WEKA data mining software: an update . . ACM SIGKDD Explor Newsl , , 2009 . . 11 ( ( 1 ): ): 10 - - 18 . . DOI: 10.1145/1656274.1656278 http://doi.org/10.1145/1656274.1656278 . .

H Jiang . . Discriminative training of HMMs for automatic speech recognition: a survey . . Comput Speech Lang , , 2010 . . 24 ( ( 4 ): ): 589 - - 608 . . DOI: 10.1016/j.csl.2009.08.002 http://doi.org/10.1016/j.csl.2009.08.002 . .

L Jiang , , , H Zhang , , , Z Cai . . A novel Bayes model: hidden nave Bayes . . IEEE Trans Knowl Data Eng , , 2009 . . 21 ( ( 10 ): ): 1361 - - 1371 . . DOI: 10.1109/TKDE.2008.234 http://doi.org/10.1109/TKDE.2008.234 . .

L Jiang , , , H Zhang , , , Z Cai , , , 等 . . Weighted average of one-dependence estimators . . J Exp Theor Artif Intell , , 2012 . . 24 ( ( 2 ): ): 219 - - 230 . . DOI: 10.1080/0952813X.2011.639092 http://doi.org/10.1080/0952813X.2011.639092 . .

Y Jiang , , , ZH Zhou . . Editing training data for k NN classifiers with neural network ensemble . . Advances in Neural Networks—Int Symp on Neural Networks , , 2004 . . p.356 - - 361 . . DOI: 10.1007/978-3-540-28647-9_60 http://doi.org/10.1007/978-3-540-28647-9_60 . .

BH Juang , , , S Katagiri . . Discriminative learning for minimum error classification (pattern recognition) . . IEEE Trans Signal Process , , 1992 . . 40 ( ( 12 ): ): 3043 - - 3054 . . DOI: 10.1109/78.175747 http://doi.org/10.1109/78.175747 . .

M Karabatak . . A new classifier for breast cancer detection based on nave Bayesian . . Measurement , , 2015 . . 72 32 - - 36 . . DOI: 10.1016/j.measurement.2015.04.028 http://doi.org/10.1016/j.measurement.2015.04.028 . .

BH Kim , , , HD Pfister . . An iterative joint linear- programming decoding of LDPC codes and finite-state channels . . IEEE Conf on Communications , , 2011 . . p.1 - - 6 . . DOI: 10.1109/icc.2011.5962814 http://doi.org/10.1109/icc.2011.5962814 . .

JTY Kwok . . Moderating the outputs of support vector machine classifiers . . IEEE Trans Neur Netw , , 1999 . . 10 ( ( 5 ): ): 1018 - - 1031 . . DOI: 10.1109/72.788642 http://doi.org/10.1109/72.788642 . .

P Moerland . . A comparison of mixture models for density estimation . . 9 th Int Conf on Artificial Neural Networks , , 1999 . . p.25 - - 30 . . DOI: 10.1049/cp:19991079 http://doi.org/10.1049/cp:19991079 . .

A Nádas . . A decision theoretic formulation of a training problem in speech recognition and a comparison of training by unconditional versus conditional maximum likelihood . . IEEE Trans Audio Speech Signal Process , , 1983 . . 31 ( ( 4 ): ): 814 - - 817 . . DOI: 10.1109/TASSP.1983.1164173 http://doi.org/10.1109/TASSP.1983.1164173 . .

Team OpenCV . . Open Source Computer Vision Library . . http://opencv.org [Accessed on July 15. 2016] , , 2015 . . .

F Pernkopf , , , M Wohlmayr . . Large margin learning of Bayesian classifiers based on Gaussian mixture models . . Joint European Conf on Machine Learning and Knowledge Discovery in Databases , , 2010 . . p.50 - - 66 . . DOI: 10.1007/978-3-642-15939-8_4 http://doi.org/10.1007/978-3-642-15939-8_4 . .

F Pernkopf , , , M Wohlmayr , , , S Tschiatschek . . Maximum margin Bayesian network classifiers . . IEEE Trans Patt Anal Mach Intell , , 2012 . . 34 ( ( 3 ): ): 521 - - 532 . . DOI: 10.1109/TPAMI.2011.149 http://doi.org/10.1109/TPAMI.2011.149 . .

D Povey , , , PC Woodland . . Minimum phone error and Ⅰ-smoothing for improved discriminative training . . IEEE Int Conf on Acoustics , , 2002 . . p.105 - - 108 . . DOI: 10.1109/ICASSP.2002.5743665 http://doi.org/10.1109/ICASSP.2002.5743665 . .

University of California . . UCI Machine Learning Repository . . http://archive.ics.uci.edu/ml [Accessed on Aug. 10. 2016] , , 2013 . . .

V Vapnik . . The Nature of Statistical Learning Theory (2 nd Ed.) . . Springer-Verlag, New York , , 2013 . . DOI: 10.1007/978-1-4757-3264-1 http://doi.org/10.1007/978-1-4757-3264-1 . .

N Vlassis , , , A Likas . . A kurtosis-based dynamic approach to Gaussian mixture modeling . . IEEE Trans Syst Man Cybern A , , 1999 . . 29 ( ( 4 ): ): 393 - - 399 . . DOI: 10.1109/3468.769758 http://doi.org/10.1109/3468.769758 . .

GI Webb , , , JR Boughton , , , Z Wang . . Not so nave Bayes: aggregating one-dependence estimators . . Mach Learn , , 2005 . . 58 ( ( 1 ): ): 5 - - 24 . . DOI: 10.1007/s10994-005-4258-6 http://doi.org/10.1007/s10994-005-4258-6 . .

PC Woodland , , , D Povey . . Large scale discriminative training of hidden Markov models for speech recognition . . Comput Speech Lang , , 2002 . . 16 ( ( 1 ): ): 25 - - 47 . . DOI: 10.1006/csla.2001.0182 http://doi.org/10.1006/csla.2001.0182 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰