Supervised topic models with weighted words: multi-label document classification

Yue-peng ZOU; Ji-hong OUYANG; Xi-ming LI

doi:10.1631/FITEE.1601668

Your Location：

Home >

Browse articles >

Supervised topic models with weighted words: multi-label document classification

Regular Papers | Updated：2022-05-19

- Supervised topic models with weighted words: multi-label document classification
- 词加权有监督主题模型：多标签文本分类
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 4, Pages: 513-523(2018)
- Affiliations：
  
  College of Computer Science and Technology, Jilin University, Changchun 130012, China
  MOE Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University, Changchun 130012, China
- Author bio：
  
  [ "Ji-hong OUYANG, E-mail: ouyj@jlu.edu.cn" ]
  Xi-ming LI, E-mail: liximing86@gmail.com
- Funds：
- DOI：10.1631/FITEE.1601668
  CLC： TP391
- Received：26 October 2016，
  
  Revised：2018-;4-;3，
  
  Published：2018-04
- Accepted：
Scan QR Code
Yue-peng ZOU, Ji-hong OUYANG, Xi-ming LI. Supervised topic models with weighted words: multi-label document classification[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(4): 513-523.
DOI：

Yue-peng ZOU, Ji-hong OUYANG, Xi-ming LI. Supervised topic models with weighted words: multi-label document classification[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(4): 513-523. DOI： 10.1631/FITEE.1601668.

摘要

有监督主题模型已成功应用于多标签文本分类任务。代表性模型包括有监督隐含狄利克雷分配模型（labeled latent Dirichlet allocation

L-LDA）和判别隐含狄利克雷分配模型（dependency-LDA）。这些已有模型忽略单词类别频率信息，即训练集中单词出现的类别数量，对分类任务的影响。对此引入类别频率信息，提出一个类别频率词权重方法（class frequency weight

CF-weight）。CF-weight方法基于如下假设：具有较高（或较低）类别频率的单词在分类问题中具有较低（或较高）判别力。将CF-weight方法应用于L-LDA和dependency-LDA模型。实验结果表明，相比传统有监督主题模型，基于CF-weight的模型在多标签分类性能上具有优势。

Abstract

Supervised topic modeling algorithms have been successfully applied to multi-label document classification tasks. Representative models include labeled latent Dirichlet allocation (L-LDA) and dependency-LDA. However

these models neglect the class frequency information of words (i.e.

the number of classes where a word has occurred in the training data)

which is significant for classification. To address this

we propose a method

namely the class frequency weight (CF-weight)

to weight words by considering the class frequency knowledge. This CF-weight is based on the intuition that a word with higher (lower) class frequency will be less (more) discriminative. In this study

the CF-weight is used to improve L-LDA and dependency-LDA. A number of experiments have been conducted on real-world multi-label datasets. Experimental results demonstrate that CF-weight based algorithms are competitive with the existing supervised topic models.

关键词

Keywords

references

Machine Learning & Knowledge Discovery Group . . Learning from Multi-label Data . . http://mlkd.csd.auth.gr/multilabel.html http://mlkd.csd.auth.gr/multilabel.html [Accessed on May 12, 2018] , , 2011 . . .

DM Blei , , , JD McAuliffe . . Supervised topic models . . 20 th Int Conf on Neural Information Processing Systems , , 2007 . . p.121 - - 128 . . . .

DM Blei , , , AY Ng , , , MI Jordan . . Latent Dirichlet allocation . . J Mach Learn Res , , 2003 . . 3 993 - - 1022 . . . .

CC Chang , , , CJ Lin . . LIBSVM—a Library for Support Vector Machines . . https://www.csie.ntu.edu.tw/~cjlin/libsvm/ https://www.csie.ntu.edu.tw/~cjlin/libsvm/ [Accessed on May 22, 2018] , , 2016 . . .

F Debole , , , F Sebastiani . . Supervised term weighting for automated text categorization . . In: Sirmakessis S (Ed.), Text Mining and Its Applications. Springer, Berlin , , 2004 . . p.81 - - 97 . . DOI: 10.1007/978-3-540-45219-5_7 http://doi.org/10.1007/978-3-540-45219-5_7 . .

Z Ghahramani . . An introduction to hidden Markov models and Bayesian networks . . Int J Patt Recogn Artif Intell , , 2001 . . 15 ( ( 1 ): ): 9 - - 42 . . DOI: 10.1142/S0218001401000836 http://doi.org/10.1142/S0218001401000836 . .

TL Griffiths , , , M Steyvers . . Finding scientific topics . . Proc Nat Acad Sci USA , , 2004 . . 101 ( ( Suppl 1 ): ): 5228 - - 5235 . . DOI: 10.1073/pnas.0307752101 http://doi.org/10.1073/pnas.0307752101 . .

H Guan , , , JY Zhou , , , MY Guo . . A class-feature-centroid classifier for text categorization . . 18 th Int Conf on World Wide Web , , 2009 . . p.201 - - 210 . . DOI: 10.1145/1526709.1526737 http://doi.org/10.1145/1526709.1526737 . .

D Kim , , , S Kim , , , A Oh . . Dirichlet process with mixed random measures: a nonparametric topic model for labeled data . . 29 th Int Conf on Machine Learning , , 2012 . . p.675 - - 682 . . . .

S Lacoste-Julien , , , F Sha , , , MI Jordan . . DiscLDA: discriminative learning for dimensionality reduction and classification . . 21 st Int Conf on Neural Information Processing Systems , , 2008 . . p.897 - - 904 . . . .

S Lee , , , J Kim , , , SH Myaeng . . An extension of topic models for text classification: a term weighting approach . . Int Conf on Big Data and Smart Computing , , 2015 . . p.217 - - 224 . . DOI: 10.1109/35021BIGCOMP.2015.7072834 http://doi.org/10.1109/35021BIGCOMP.2015.7072834 . .

XM Li , , , JH Ouyang , , , XT Zhou . . Centroid prior topic model for multi-label classification . . Patt Recogn Lett , , 2015a . . 62 8 - - 13 . . DOI: 10.1016/j.patrec.2015.04.012 http://doi.org/10.1016/j.patrec.2015.04.012 . .

XM Li , , , JH Ouyang , , , XT Zhou . . Supervised topic models for multi-label classification . . Neurocomputing , , 2015b . . 149 811 - - 819 . . DOI: 10.1016/j.neucom.2014.07.053 http://doi.org/10.1016/j.neucom.2014.07.053 . .

RE Madsen , , , D Kauchak , , , C Elkan . . Modeling word burstiness using the Dirichlet distribution . . 22 nd Int Conf on Machine Learning , , 2005 . . p.545 - - 552 . . DOI: 10.1145/1102351.1102420 http://doi.org/10.1145/1102351.1102420 . .

J Petterson , , , A Smola , , , T Caetano , , , 等 . . Word features for latent Dirichlet allocation . . 23 rd Int Conf on Neural Information Processing Systems , , 2010 . . p.1921 - - 1929 . . . .

D Ramage , , , D Hall , , , R Nallapati , , , 等 . . Labeled LDA: a supervised topic model for credit attribution in multilabeled corpora . . Conf on Empirical Methods in Natural Language Processing , , 2009 . . p.248 - - 256 . . DOI: 10.3115/1699510.1699543 http://doi.org/10.3115/1699510.1699543 . .

D Ramage , , , CD Manning , , , S Dumais . . Partially labeled topic models for interpretable text mining . . 17 th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining , , 2011 . . p.457 - - 465 . . DOI: 10.1145/2020408.2020481 http://doi.org/10.1145/2020408.2020481 . .

J Reisinger , , , A Waters , , , B Silverthorn , , , 等 . . Spherical topic models . . Proc 27 th Int Conf on Machine Learning , , 2010 . . p.1 - - 8 . . . .

TN Rubin , , , A Chambers , , , P Smyth , , , 等 . . Statistical topic models for multi-label document classification . . Mach Learn , , 2012 . . 88 ( ( 1-2 ): ): 157 - - 208 . . DOI: 10.1007/s10994-011-5272-5 http://doi.org/10.1007/s10994-011-5272-5 . .

G Salton , , , C Buckley . . Term-weighting approaches in automatic text retrieval . . Inform Process Manag , , 1988 . . 24 ( ( 5 ): ): 513 - - 523 . . DOI: 10.1016/0306-4573(88)90021-0 http://doi.org/10.1016/0306-4573(88)90021-0 . .

LF Shang , , , KP Chan , , , GD Pan . . DTTM: a discriminative temporal topic model for facial expression recognition . . 7 th Int Conf on Advances in Visual Computing , , 2011 . . p.596 - - 606 . . DOI: 10.1007/978-3-642-24028-7_55 http://doi.org/10.1007/978-3-642-24028-7_55 . .

G Tsoumakas , , , E Spyromitros-Xioufis , , , J Vilcek , , , 等 . . Mulan: a Java library for multi-label learning . . J Mach Learn Res , , 2011a . . 12 ( ( 7 ): ): 2411 - - 2414 . . . .

G Tsoumakas , , , I Katakis , , , I Vlahavas . . Random k -labelsets for multilabel classification . . IEEE Trans Knowl Data Eng , , 2011b . . 23 ( ( 7 ): ): 1079 - - 1089 . . DOI: 10.1109/TKDE.2010.164 http://doi.org/10.1109/TKDE.2010.164 . .

AT Wilson , , , PA Chew . . Term weighting schemes for latent Dirichlet allocation . . Human Language Technologies: Annual Conf of the North American Chapter of the Association for Computational Linguistics , , 2010 . . p.465 - - 473 . . . .

J Zhu , , , A Ahmed , , , EP Xing . . MedLDA: maximum margin supervised topic models . . 26 th Annual Int Conf on Machine Learning , , 2012 . . p.1257 - - 1264 . . DOI: 10.1145/1553374.1553535 http://doi.org/10.1145/1553374.1553535 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

A graph-based two-stage classification network for mobile screen defect inspection

Related Author

Chaofan ZHOU

Meiqin LIU

Senlin ZHANG

Ping WEI

Badong CHEN

Related Institution

State Key Laboratory of Industrial Control Technology, Zhejiang University

College of Electrical Engineering, Zhejiang University

Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰