HAM: a deep collaborative ranking method incorporating textual information

Cheng-wei WANG; Teng-fei ZHOU; Chen CHEN; Tian-lei HU; Gang CHEN

doi:10.1631/FITEE.1900382

Your Location：

Home >

Browse articles >

HAM: a deep collaborative ranking method incorporating textual information

Regular Papers | Updated：2022-05-19

- HAM: a deep collaborative ranking method incorporating textual information
- HAM：一种融入文本信息的深度协同排序方法
- Frontiers of Information Technology & Electronic Engineering Vol. 21, Issue 8, Pages: 1206-1216(2020)
- Affiliations：
  
  The Key Laboratory of Big Data Intelligent Computing of Zhejiang Province, Hangzhou 310027, China
  State Key Lab of CAD & CG, Zhejiang University, Hangzhou 310027, China
  College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  [ "Cheng-wei WANG, E-mail: rr@zju.edu.cn" ]
  [ "Teng-fei ZHOU, E-mail: zhoutengfei@zju.edu.cn" ]
  [ "Chen CHEN, E-mail:cc33@zju.edu.cn" ]
  [ "Tian-lei HU, E-mail:htl@zju.edu.cn" ]
  [ "Gang CHEN, E-mail:cg@zju.edu.cn" ]
- Funds：
- DOI：10.1631/FITEE.1900382
  CLC： TP181
- Received：28 July 2019，
  
  Revised：2020-;7-13，
  
  Published：2020-08
- Accepted：
Scan QR Code
Cheng-wei WANG, Teng-fei ZHOU, Chen CHEN, et al. HAM: a deep collaborative ranking method incorporating textual information[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1206-1216.
DOI：

Cheng-wei WANG, Teng-fei ZHOU, Chen CHEN, et al. HAM: a deep collaborative ranking method incorporating textual information[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1206-1216. DOI： 10.1631/FITEE.1900382.

摘要

基于文本语料库的推荐任务旨在通过挖掘用户反馈数据以及物品文本描述数据对用户偏好建模。当前研究人员亟需探索使用深度神经网络捕获复杂的非线性偏好。然而，训练网络结构更深的推荐器并不能通过简单添加网络层数实现。一个网络结构更深的推荐器会面临梯度消失/爆炸问题，导致其无法通过基于梯度的方法进行模型训练。此外，物品文字描述数据可能包含嘈杂的单词序列；直接从这类特征向量中提取特征可能会影响推荐器性能。为解决上述问题，本文提出一种新的基于极深神经网络的排序推荐方法：高速网络推荐器(HighwAy recoMmender

HAM)。首先基于高速公路网络设计一个全新神经网络推荐框架，该框架能有效地稳定深度推荐器的梯度流。其次，使用一种多头注意力编码器，自动对文本信息降噪。最后，提出一种全新的基于块坐标下降的方法，可更加有效地训练具有更深网络结构的推荐器。实验结果表明，与当前先进方法相比，HAM具有更好推荐性能。

Abstract

The recommendation task with a textual corpus aims to model customer preferences from both user feedback and item textual descriptions. It is highly desirable to explore a very deep neural network to capture the complicated nonlinear preferences. However

training a deeper recommender is not as effortless as simply adding layers. A deeper recommender suffers from the gradient vanishing/exploding issue and cannot be easily trained by gradient-based methods. Moreover

textual descriptions probably contain noisy word sequences. Directly extracting feature vectors from them can harm the recommender's performance. To overcome these difficulties

we propose a new recommendation method named the HighwAy recoMmender (HAM). HAM explores a highway mechanism to make gradient-based training methods stable. A multi-head attention mechanism is devised to automatically denoise textual information. Moreover

a block coordinate descent method is devised to train a deep neural recommender. Empirical studies show that the proposed method outperforms state-of-the-art methods significantly in terms of accuracy.

关键词

Keywords

references

G Adomavicius , , , A Tuzhilin . . Toward the next generation of recommender systems: a survey of the state-ofthe-art and possible extensions . . IEEE Trans Knowl Data Eng , , 2005 . . 17 ( ( 6 ): ): 734 - - 749 . . DOI: 10.1109/TKDE.2005.99 http://doi.org/10.1109/TKDE.2005.99 . .

T Bansal , , , D Belanger , , , A McCallum . . Ask the GRU: multi-task learning for deep text recommendations . . Proc 10 th ACM Conf on Recommender Systems , , 2016 . . p.107 - - 114 . . DOI: 10.1145/2959100.2959180 http://doi.org/10.1145/2959100.2959180 . .

J Bennett , , , S Lanning . . The Netflix prize . . Proc KDD Cup and Workshop , , 2007 . . p.35 . .

XY Cai , , , J Han , , , L Yang . . Generative adversarial network based heterogeneous bibliographic network representation for personalized citation recommendation . . 32 nd AAAI Conf on Artificial Intelligence , , 2018 . . p.5747 - - 5754 . . . .

JK Chorowski , , , D Bahdanau , , , D Serdyuk , , , 等 . . Attention-based models for speech recognition . . Proc 30 th Int Conf on Neural Information Processing Systems , , 2015 . . p.577 - - 585 . . . .

R Devooght , , , H Bersini . . Collaborative filtering with recurrent neural networks . . 2016 . . .

I Goodfellow , , , Y Bengio , , , A Courville . . Deep Learning , , : : Cambridge, MA MIT Press , , 2016 . . .

PK Gopalan , , , L Charlin , , , D Blei . . Content-based recommendations with Poisson factorization . . Proc Advances in Neural Information Processing Systems , , 2014 . . p.3176 - - 3184 . . . .

M Grčar , , , D Mladenič , , , B Fortuna , , , 等 . . Data sparsity issues in the collaborative filtering framework . . Proc 7 th Int Workshop on Knowledge Discovery on the Web , , 2005 . . p.58 - - 76 . . . .

XN He , , , LZ Liao , , , HW Zhang , , , 等 . . Neural collaborative filtering . . Proc 26 th Int Conf on World Wide Web , , 2017 . . p.173 - - 182 . . DOI: 10.1145/3038912.3052569 http://doi.org/10.1145/3038912.3052569 . .

CK Hsieh , , , L Yang , , , Y Cui , , , 等 . . Collaborative metric learning . . Proc 26 th Int Conf on World Wide Web , , 2017 . . p.193 - - 201 . . . .

M Jin , , , X Luo , , , H Zhu , , , 等 . . Combining deep learning and topic modeling for review understanding in contextaware recommendation . . Proc Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , , 2018 . . p.1605 - - 1614 . . . .

D Kim , , , C Park , , , J Oh , , , 等 . . Convolutional matrix factorization for document context-aware recommendation . . Proc 10 th ACM Conf on Recommender Systems , , 2016 . . p.233 - - 240 . . DOI: 10.1145/2959100.2959165 http://doi.org/10.1145/2959100.2959165 . .

R Kiros , , , Y Zhu , , , RR Salakhutdinov , , , 等 . . Skip-thought vectors . . Advances in Neural Information Processing Systems , , 2015 . . p.3294 - - 3302 . . . .

Y Koren . . Factorization meets the neighborhood: a multifaceted collaborative filtering model . . Proc 14 th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining , , 2008 . . p.426 - - 434 . . DOI: 10.1145/1401890.1401944 http://doi.org/10.1145/1401890.1401944 . .

Y Koren , , , R Bell , , , C Volinsky . . Matrix factorization techniques for recommender systems . . Computer , , 2009 . . 42 ( ( 8 ): ): 30 - - 37 . . DOI: 10.1109/MC.2009.263 http://doi.org/10.1109/MC.2009.263 . .

O Levy , , , Y Goldberg . . Neural word embedding as implicit matrix factorization . . Proc 27 th Int Conf on Neural Information Processing Systems , , 2014 . . p.2177 - - 2185 . . . .

G Linden , , , B Smith , , , J York . . Amazon . . com recommendations: item-to-item collaborative filtering. IEEE Int Comput , , 2003 . . 7 ( ( 1 ): ): 76 - - 80 . . DOI: 10.1109/MIC.2003.1167344 http://doi.org/10.1109/MIC.2003.1167344 . .

CH Liu , , , T Jin , , , SCH Hoi , , , 等 . . Collaborative topic regression for online recommender systems: an online and Bayesian approach . . Mach Learn , , 2017 . . 106 ( ( 5 ): ): 651 - - 670 . . DOI: 10.1007/s10994-016-5599-z http://doi.org/10.1007/s10994-016-5599-z . .

MR McLaughlin , , , JL Herlocker . . A collaborative filtering algorithm and evaluation metric that accurately model the user experience . . Proc 27 th Annual Int ACM SIGIR Conf on Research and Development in Information Retrieval , , 2004 . . p.329 - - 336 . . DOI: 10.1145/1008992.1009050 http://doi.org/10.1145/1008992.1009050 . .

H Mhaskar , , , Q Liao , , , T Poggio . . When and why are deep networks better than shallow ones . . Proc 31 st AAAI Conf on Artificial Intelligence , , 2017 . . p.2343 - - 2349 . . . .

B Neyshabur , , , S Bhojanapalli , , , D McAllester , , , 等 . . Exploring generalization in deep learning . . Proc 30 th Conf on Advances in Neural Information Processing Systems , , 2017 . . p.5947 - - 5956 . . . .

A Paterek . . Improving regularized singular value decomposition for collaborative filtering . . Proc KDD Cup and Workshop , , 2007 . . p.5 - - 8 . . . .

M Raghu , , , B Poole , , , J Kleinberg , , , 等 . . On the expressive power of deep neural networks . . Proc 34 th Int Conf on Machine Learning , , 2017 . . p.2847 - - 2854 . . . .

S Rendle , , , C Freudenthaler , , , Z Gantner , , , 等 . . BPR: Bayesian personalized ranking from implicit feedback . . Proc 25 th Conf on Uncertainty in Artificial Intelligence , , 2009 . . p.452 - - 461 . . . .

S Ruder . . An overview of multi-task learning in deep neural networks . . 2017 . . https://arxiv.org/abs/1706.05098 https://arxiv.org/abs/1706.05098 , , . .

R Salakhutdinov , , , A Mnih . . Probabilistic matrix factorization . . Proc 20 th Int Conf on Neural Information Processing Systems , , 2007 . . p.1257 - - 1264 . . . .

R Salakhutdinov , , , A Mnih , , , G Hinton . . Restricted Boltzmann machines for collaborative filtering . . Proc 24 th Int Conf on Machine Learning , , 2007 . . p.791 - - 798 . . DOI: 10.1145/1273496.1273596 http://doi.org/10.1145/1273496.1273596 . .

BM Shoja , , , N Tabrizi . . Customer reviews analysis with deep neural networks for e-commerce recommender systems . . IEEE Access , , 2019 . . 7 119121 - - 119130 . . DOI: 10.1109/ACCESS.2019.2937518 http://doi.org/10.1109/ACCESS.2019.2937518 . .

N Srebro , , , J Rennie , , , TS Jaakkola . . Maximum-margin matrix factorization . . Conf on Neural Information Processing Systems , , 2004 . . p.1329 - - 1336 . . . .

RK Srivastava , , , K Greff , , , J Schmidhuber . . Training very deep networks . . Advances in Neural Information Processing Systems , , 2015 . . p.2377 - - 2385 . . . .

F Strub , , , J Mary . . Collaborative filtering with stacked denoising autoencoders and sparse inputs . . NIPS Workshop on Machine Learning for e-Commerce , , 2015 . . p.1 - - 8 . . . .

A Vaswani , , , N Shazeer , , , N Parmar , , , 等 . . Attention is all you need . . 31 st Conf on Neural Information Processing Systems , , 2017 . . p.5998 - - 6008 . . . .

C Wang , , , DM Blei . . Collaborative topic modeling for recommending scientific articles . . Proc 17 th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining , , 2011 . . p.448 - - 456 . . DOI: 10.1145/2020408.2020480 http://doi.org/10.1145/2020408.2020480 . .

H Wang , , , NY Wang , , , DY Yeung . . Collaborative deep learning for recommender systems . . Proc 21 st ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining , , 2015 . . p.1235 - - 1244 . . DOI: 10.1145/2783258.2783273 http://doi.org/10.1145/2783258.2783273 . .

H Wang , , , XJ Shi , , , DY Yeung . . Collaborative recurrent autoencoder: recommend while learning to fill in the blanks . . Proc 30 th Int Conf on Neural Information Processing Systems , , 2016 . . p.415 - - 423 . . . .

Y Wu , , , C DuBois , , , AX Zheng , , , 等 . . Collaborative denoising auto-encoders for top- N recommender systems . . Proc 9 th ACM Int Conf on Web Search and Data Mining , , 2016 . . p.153 - - 162 . . DOI: 10.1145/2835776.2835837 http://doi.org/10.1145/2835776.2835837 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

MltAuxTSPP: a unified benchmark for deep learning-based traffic state prediction with multi-source auxiliary data

Q-space-coordinate-guided neural networks for high-fidelity diffusion tensor estimation from minimal diffusion-weighted images

CRGT-SA: an interlaced and spatiotemporal deep learning model for network intrusion detection

Federated model with contrastive learning and adaptive control variates for human activity recognition

A comprehensive survey of physical adversarial vulnerabilities in autonomous driving systems

Related Author

Yusong ZHOU

Xiaoyu JIANG

Shu SUN

Xinmin ZHANG

Yuanqiu MO

Zhihuan SONG

Maokun ZHENG

Zhi LI

Related Institution

School of Mathematics, Southeast University

State Key Laboratory of Industrial Control Technology, Zhejiang University

College of Control Science and Engineering, Zhejiang University

State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University

School of Electronic and Electrical Engineering, Shanghai University of Engineering Science

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰