词加权有监督主题模型：多标签文本分类

邹跃鹏; 欧阳继红; 李熙铭

doi:10.1631/FITEE.1601668

Your Location：

Home >

Browse articles >

词加权有监督主题模型：多标签文本分类

常规文章 | Updated：2022-05-19

- 词加权有监督主题模型：多标签文本分类
- Supervised topic models with weighted words: multi-label document classification
- 信息与电子工程前沿（英文） 2018年19卷第4期页码：513-523
- Affiliations：
  
  College of Computer Science and Technology, Jilin University, Changchun 130012, China
  MOE Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University, Changchun 130012, China
- Author bio：
  
  [ "Ji-hong OUYANG, E-mail: ouyj@jlu.edu.cn" ]
  Xi-ming LI, E-mail: liximing86@gmail.com
- Funds：
  
  Project supported by the National Natural Science Foundation of China (No. 61602204)
- DOI：10.1631/FITEE.1601668
  中图分类号： TP391
- 收稿：2016-10-26，
  
  修回：2018-;4-;3，
  
  纸质出版：2018-04
- Accepted：
Scan QR Code
邹跃鹏, 欧阳继红, 李熙铭. 词加权有监督主题模型：多标签文本分类[J]. 信息与电子工程前沿（英文）, 2018,19(4):513-523.

Yue-peng ZOU, Ji-hong OUYANG, Xi-ming LI. Supervised topic models with weighted words: multi-label document classification[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(4): 513-523.
邹跃鹏, 欧阳继红, 李熙铭. 词加权有监督主题模型：多标签文本分类[J]. 信息与电子工程前沿（英文）, 2018,19(4):513-523. DOI： 10.1631/FITEE.1601668.

Yue-peng ZOU, Ji-hong OUYANG, Xi-ming LI. Supervised topic models with weighted words: multi-label document classification[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(4): 513-523. DOI： 10.1631/FITEE.1601668.

摘要

有监督主题模型已成功应用于多标签文本分类任务。代表性模型包括有监督隐含狄利克雷分配模型（labeled latent Dirichlet allocation

L-LDA）和判别隐含狄利克雷分配模型（dependency-LDA）。这些已有模型忽略单词类别频率信息，即训练集中单词出现的类别数量，对分类任务的影响。对此引入类别频率信息，提出一个类别频率词权重方法（class frequency weight

CF-weight）。CF-weight方法基于如下假设：具有较高（或较低）类别频率的单词在分类问题中具有较低（或较高）判别力。将CF-weight方法应用于L-LDA和dependency-LDA模型。实验结果表明，相比传统有监督主题模型，基于CF-weight的模型在多标签分类性能上具有优势。

Abstract

Supervised topic modeling algorithms have been successfully applied to multi-label document classification tasks. Representative models include labeled latent Dirichlet allocation (L-LDA) and dependency-LDA. However

these models neglect the class frequency information of words (i.e.

the number of classes where a word has occurred in the training data)

which is significant for classification. To address this

we propose a method

namely the class frequency weight (CF-weight)

to weight words by considering the class frequency knowledge. This CF-weight is based on the intuition that a word with higher (lower) class frequency will be less (more) discriminative. In this study

the CF-weight is used to improve L-LDA and dependency-LDA. A number of experiments have been conducted on real-world multi-label datasets. Experimental results demonstrate that CF-weight based algorithms are competitive with the existing supervised topic models.

关键词

Keywords

references

Machine Learning & Knowledge Discovery Group . . Learning from Multi-label Data . . http://mlkd.csd.auth.gr/multilabel.html http://mlkd.csd.auth.gr/multilabel.html [Accessed on May 12, 2018] , , 2011 . . .

DM Blei , , , JD McAuliffe . . Supervised topic models . . 20 th Int Conf on Neural Information Processing Systems , , 2007 . . p.121 - - 128 . . . .

DM Blei , , , AY Ng , , , MI Jordan . . Latent Dirichlet allocation . . J Mach Learn Res , , 2003 . . 3 993 - - 1022 . . . .

CC Chang , , , CJ Lin . . LIBSVM—a Library for Support Vector Machines . . https://www.csie.ntu.edu.tw/~cjlin/libsvm/ https://www.csie.ntu.edu.tw/~cjlin/libsvm/ [Accessed on May 22, 2018] , , 2016 . . .

F Debole , , , F Sebastiani . . Supervised term weighting for automated text categorization . . In: Sirmakessis S (Ed.), Text Mining and Its Applications. Springer, Berlin , , 2004 . . p.81 - - 97 . . DOI: 10.1007/978-3-540-45219-5_7 http://doi.org/10.1007/978-3-540-45219-5_7 . .

Z Ghahramani . . An introduction to hidden Markov models and Bayesian networks . . Int J Patt Recogn Artif Intell , , 2001 . . 15 ( ( 1 ): ): 9 - - 42 . . DOI: 10.1142/S0218001401000836 http://doi.org/10.1142/S0218001401000836 . .

TL Griffiths , , , M Steyvers . . Finding scientific topics . . Proc Nat Acad Sci USA , , 2004 . . 101 ( ( Suppl 1 ): ): 5228 - - 5235 . . DOI: 10.1073/pnas.0307752101 http://doi.org/10.1073/pnas.0307752101 . .

H Guan , , , JY Zhou , , , MY Guo . . A class-feature-centroid classifier for text categorization . . 18 th Int Conf on World Wide Web , , 2009 . . p.201 - - 210 . . DOI: 10.1145/1526709.1526737 http://doi.org/10.1145/1526709.1526737 . .

D Kim , , , S Kim , , , A Oh . . Dirichlet process with mixed random measures: a nonparametric topic model for labeled data . . 29 th Int Conf on Machine Learning , , 2012 . . p.675 - - 682 . . . .

S Lacoste-Julien , , , F Sha , , , MI Jordan . . DiscLDA: discriminative learning for dimensionality reduction and classification . . 21 st Int Conf on Neural Information Processing Systems , , 2008 . . p.897 - - 904 . . . .

S Lee , , , J Kim , , , SH Myaeng . . An extension of topic models for text classification: a term weighting approach . . Int Conf on Big Data and Smart Computing , , 2015 . . p.217 - - 224 . . DOI: 10.1109/35021BIGCOMP.2015.7072834 http://doi.org/10.1109/35021BIGCOMP.2015.7072834 . .

XM Li , , , JH Ouyang , , , XT Zhou . . Centroid prior topic model for multi-label classification . . Patt Recogn Lett , , 2015a . . 62 8 - - 13 . . DOI: 10.1016/j.patrec.2015.04.012 http://doi.org/10.1016/j.patrec.2015.04.012 . .

XM Li , , , JH Ouyang , , , XT Zhou . . Supervised topic models for multi-label classification . . Neurocomputing , , 2015b . . 149 811 - - 819 . . DOI: 10.1016/j.neucom.2014.07.053 http://doi.org/10.1016/j.neucom.2014.07.053 . .

RE Madsen , , , D Kauchak , , , C Elkan . . Modeling word burstiness using the Dirichlet distribution . . 22 nd Int Conf on Machine Learning , , 2005 . . p.545 - - 552 . . DOI: 10.1145/1102351.1102420 http://doi.org/10.1145/1102351.1102420 . .

J Petterson , , , A Smola , , , T Caetano , , , 等 . . Word features for latent Dirichlet allocation . . 23 rd Int Conf on Neural Information Processing Systems , , 2010 . . p.1921 - - 1929 . . . .

D Ramage , , , D Hall , , , R Nallapati , , , 等 . . Labeled LDA: a supervised topic model for credit attribution in multilabeled corpora . . Conf on Empirical Methods in Natural Language Processing , , 2009 . . p.248 - - 256 . . DOI: 10.3115/1699510.1699543 http://doi.org/10.3115/1699510.1699543 . .

D Ramage , , , CD Manning , , , S Dumais . . Partially labeled topic models for interpretable text mining . . 17 th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining , , 2011 . . p.457 - - 465 . . DOI: 10.1145/2020408.2020481 http://doi.org/10.1145/2020408.2020481 . .

J Reisinger , , , A Waters , , , B Silverthorn , , , 等 . . Spherical topic models . . Proc 27 th Int Conf on Machine Learning , , 2010 . . p.1 - - 8 . . . .

TN Rubin , , , A Chambers , , , P Smyth , , , 等 . . Statistical topic models for multi-label document classification . . Mach Learn , , 2012 . . 88 ( ( 1-2 ): ): 157 - - 208 . . DOI: 10.1007/s10994-011-5272-5 http://doi.org/10.1007/s10994-011-5272-5 . .

G Salton , , , C Buckley . . Term-weighting approaches in automatic text retrieval . . Inform Process Manag , , 1988 . . 24 ( ( 5 ): ): 513 - - 523 . . DOI: 10.1016/0306-4573(88)90021-0 http://doi.org/10.1016/0306-4573(88)90021-0 . .

LF Shang , , , KP Chan , , , GD Pan . . DTTM: a discriminative temporal topic model for facial expression recognition . . 7 th Int Conf on Advances in Visual Computing , , 2011 . . p.596 - - 606 . . DOI: 10.1007/978-3-642-24028-7_55 http://doi.org/10.1007/978-3-642-24028-7_55 . .

G Tsoumakas , , , E Spyromitros-Xioufis , , , J Vilcek , , , 等 . . Mulan: a Java library for multi-label learning . . J Mach Learn Res , , 2011a . . 12 ( ( 7 ): ): 2411 - - 2414 . . . .

G Tsoumakas , , , I Katakis , , , I Vlahavas . . Random k -labelsets for multilabel classification . . IEEE Trans Knowl Data Eng , , 2011b . . 23 ( ( 7 ): ): 1079 - - 1089 . . DOI: 10.1109/TKDE.2010.164 http://doi.org/10.1109/TKDE.2010.164 . .

AT Wilson , , , PA Chew . . Term weighting schemes for latent Dirichlet allocation . . Human Language Technologies: Annual Conf of the North American Chapter of the Association for Computational Linguistics , , 2010 . . p.465 - - 473 . . . .

J Zhu , , , A Ahmed , , , EP Xing . . MedLDA: maximum margin supervised topic models . . 26 th Annual Int Conf on Machine Learning , , 2012 . . p.1257 - - 1264 . . DOI: 10.1145/1553374.1553535 http://doi.org/10.1145/1553374.1553535 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

A graph-based two-stage classification network for mobile screen defect inspection