Beyond bag of latent topics: spatial pyramid matching for scene category recognition

LU Fu-xiang; HUANG Jun

doi:10.1631/FITEE.1500070

Your Location：

Home >

Browse articles >

Beyond bag of latent topics: spatial pyramid matching for scene category recognition

Computer & Automation | Updated：2022-06-06

- Beyond bag of latent topics: spatial pyramid matching for scene category recognition
- 超越隐主题包模型：针对场景类别识别的空间金字塔匹配
- Frontiers of Information Technology & Electronic Engineering Vol. 16, Issue 10, Pages: 817-828(2015)
- Affiliations：
  
  1.School of Information Science & Engineering, Lanzhou University, Lanzhou 730000, China
  2.Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China
- Author bio：
  
  Fu-xiang LU, E-mail: lufux@lzu.edu.cn
  [ "Jun HUANG, E-mail:huangj@sari.ac.cn" ]
- Funds：
- DOI：10.1631/FITEE.1500070
  CLC： TP391.4
- Received：07 March 2015，
  
  Revised：2015-09-21，
  
  Published：2015-10
- Accepted：
Scan QR Code
LU Fu-xiang, HUANG Jun. Beyond bag of latent topics: spatial pyramid matching for scene category recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(10): 817-828.
DOI：

LU Fu-xiang, HUANG Jun. Beyond bag of latent topics: spatial pyramid matching for scene category recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(10): 817-828. DOI： 10.1631/FITEE.1500070.

摘要

目的

随着智能手机、数码相机的普及和互联网的高速发展，基于内容的场景类别识别对于图像数据库标注和检索具有重要意义。在场景类别数目比较多的情况下，本文基于概率隐语义分析（pLSA）和自适应提升（AdaBoost）算法，实现一种鲁棒的场景类别识别算法。

创新点

记录pLSA学习得到的主题的位置关系，提出金字塔主题直方图；在词包（bag-of-words）模型中采用不同的兴趣点提取算子和不同的局部区域描述符，实现异质金字塔单词直方图，显著提升场景识别准确率；提出一种两级多分类算法。

方法

利用期望最大化（EM）算法计算图像或图像块的pLSA主题分布，通过空间金字塔（SP）记录主题之间的大致位置关系；通过对兴趣点提取算子和区域描述符的比较研究，在词包模型中选用稠密兴趣点提取算子和六种区域描述符，从而得到六个金字塔主题直方图用来表示图像；为充分利用各异质金字塔主题直方图信息，先用AdaBoost选取分辨能力强的主题并计算测试图像的后验概率，再由prod-max融合规则确定测试图像的类别。

结论

对于特定的兴趣点提取算子和特定的区域描述符，金字塔主题直方图对所有基准图像库的场景识别率均高于标准pLSA主题直方图；融合各异质金字塔主题直方图显著提高了场景类别识别率。

Abstract

We propose a heterogeneous

mid-level feature based method for recognizing natural scene categories. The proposed feature introduces spatial information among the latent topics by means of spatial pyramid

while the latent topics are obtained by using probabilistic latent semantic analysis (pLSA) based on the bag-of-words representation. The proposed feature always performs better than standard pLSA because the performance of pLSA is adversely affected in many cases due to the loss of spatial information. By combining various interest point detectors and local region descriptors used in the bag-of-words model

the proposed feature can make further improvement for diverse scene category recognition tasks. We also propose a two-stage framework for multi-class classification. In the first stage

for each of possible detector/descriptor pairs

adaptive boosting classifiers are employed to select the most discriminative topics and further compute posterior probabilities of an unknown image from those selected topics. The second stage uses the prod-max rule to combine information coming from multiple sources and assigns the unknown image to the scene category with the highest 'final' posterior probability. Experimental results on three benchmark scene datasets show that the proposed method exceeds most state-of-the-art methods.

关键词

Keywords

references

Y. Freund , R.E. Schapire . A decision-theoretic generalization of on-line learning and an application to boosting . J. Comput. Syst. Sci. , 1997 . 55 ( 1 ): 119 - 139 . DOI: 10.1006/jcss.1997.1504 http://doi.org/10.1006/jcss.1997.1504 .

C. Harris , M. Stephens . A combined corner and edge detector . Alvey Vision Conf. , 1988 . p.147 - 151 . DOI: 10.5244/C.2.23 http://doi.org/10.5244/C.2.23 .

T. Hofmann . Probabilistic latent semantic indexing. Proc. 22nd Annual Int . ACM SIGIR Conf. on Research and Development in Information Retrieval , 1999 . p.50 - 57 . DOI: 10.1145/312624.312649 http://doi.org/10.1145/312624.312649 .

Z.H. Hu , Y.Z. Cai , Y.G. Li , 等 . Data fusion for fault diagnosis using multi-class support vector machines . J. Zhejiang Univ.-Sci. , 2005 . 6A ( 10 ): 1030 - 1039 . DOI: 10.1631/jzus.2005.A1030 http://doi.org/10.1631/jzus.2005.A1030 .

T. Kadir , M. Brady . Saliency, scale and image description . Int. J. Comput. Vis. , 2001 . 45 ( 2 ): 83 - 105 . DOI: 10.1023/A:1012460413855 http://doi.org/10.1023/A:1012460413855 .

R. Kwitt , N. Vasconcelos , N. Rasiwasia . Scene recognition on the semantic manifold . European Conf. on Computer Vision , 2012 . p.359 - 372 . DOI: 10.1007/978-3-642-33765-9_26 http://doi.org/10.1007/978-3-642-33765-9_26 .

S. Lazebnik , C. Schmid , J. Ponce . Beyond bags of features: spatial pyramid matching for recognizing natural scene categories . IEEE Computer Society Conf. on Computer Vision and Pattern Recognition , 2006 . p.2169 - 2178 . DOI: 10.1109/CVPR.2006.68 http://doi.org/10.1109/CVPR.2006.68 .

F.F. Li , P. Perona . A Bayesian hierarchical model for learning natural scene categories . IEEE Computer Society Conf. on Computer Vision and Pattern Recognition , 2005 . p.524 - 531 . DOI: 10.1109/CVPR.2005.16 http://doi.org/10.1109/CVPR.2005.16 .

J.G. Liu , M. Shah . Scene modeling using coclustering . IEEE Int. Conf. on Computer Vision , 2007 . p.1 - 7 . DOI: 10.1109/ICCV.2007.4408866 http://doi.org/10.1109/ICCV.2007.4408866 .

D.G. Lowe . Distinctive image features from scaleinvariant keypoints . Int. J. Comput. Vis. , 2004 . 60 ( 2 ): 91 - 110 . DOI: 10.1023/B:VISI.0000029664.99615.94 http://doi.org/10.1023/B:VISI.0000029664.99615.94 .

F.X. Lu , X.K. Yang , R. Zhang , 等 . Image classification based on pyramid histogram of topics . IEEE Int. Conf. on Multimedia and Expo , 2009 . p.398 - 401 . DOI: 10.1109/ICME.2009.5202518 http://doi.org/10.1109/ICME.2009.5202518 .

F.X. Lu , X.K. Yang , W.Y. Lin , 等 . Image classification with multiple feature channels . Opt. Eng. , 2011 . 50 ( 5 ): 057210.1 - 057210.9 . DOI: 10.1117/1.3582852 http://doi.org/10.1117/1.3582852 .

J. Matas , O. Chum , M. Urban , 等 . Robust wide-baseline stereo from maximally stable extremal regions . Image Vis. Comput. , 2004 . 22 ( 10 ): 761 - 767 . DOI: 10.1016/j.imavis.2004.02.006 http://doi.org/10.1016/j.imavis.2004.02.006 .

K. Mikolajczyk , C. Schmid . Scale & affine invariant interest point detectors . Int. J. Comput. Vis. , 2004 . 60 ( 1 ): 63 - 86 . DOI: 10.1023/B:VISI.0000027790.02288.f2 http://doi.org/10.1023/B:VISI.0000027790.02288.f2 .

A. Oliva , A. Torralba . Modeling the shape of the scene: a holistic representation of the spatial envelope . Int. J. Comput. Vis. , 2001 . 42 ( 3 ): 145 - 175 . DOI: 10.1023/A:1011139631724 http://doi.org/10.1023/A:1011139631724 .

X.B. Qi , R. Xiao , C.G. Li , 等 . Pairwise rotation invariant co-occurrence local binary pattern . IEEE Trans. Patt. Anal. Mach. Intell. , 2014 . 36 ( 11 ): 2199 - 2213 . DOI: 10.1109/TPAMI.2014.2316826 http://doi.org/10.1109/TPAMI.2014.2316826 .

P. Quelhas , F. Monay , J. Odobez , 等 . A thousand words in a scene . IEEE Trans. Patt. Anal. Mach. Intell. , 2007 . 29 ( 9 ): 1575 - 1589 . DOI: 10.1109/TPAMI.2007.1155 http://doi.org/10.1109/TPAMI.2007.1155 .

E. Shechtman , M. Irani . Matching local selfsimilarities across images and videos . IEEE Conf. on Computer Vision and Pattern Recognition , 2007 . p.1 - 8 . DOI: 10.1109/CVPR.2007.383198 http://doi.org/10.1109/CVPR.2007.383198 .

Z.L. Wang , J.S. Feng , S.C. Yan , 等 . Linear distance coding for image classification . IEEE Trans. Image Process. , 2013 . 22 ( 2 ): 537 - 548 . DOI: 10.1109/TIP.2012.2218826 http://doi.org/10.1109/TIP.2012.2218826 .

J.X. Wu . Efficient HIK SVM learning for image classification . IEEE Trans. Image Process. , 2012 . 21 ( 10 ): 4442 - 4453 . DOI: 10.1109/TIP.2012.2207392 http://doi.org/10.1109/TIP.2012.2207392 .

J.X. Wu , J.M. Rehg . CENTRIST: a visual descriptor for scene categorization . IEEE Trans. Patt. Anal. Mach. Intell. , 2011 . 33 ( 8 ): 1489 - 1501 . DOI: 10.1109/TPAMI.2010.224 http://doi.org/10.1109/TPAMI.2010.224 .

J.G. Zhang , M. Marszałek , S. Lazebnik , 等 . Local features and kernels for classification of texture and object categories: a comprehensive study . Int. J. Comput. Vis. , 2006 . 73 ( 2 ): 213 - 238 . DOI: 10.1007/s11263-006-9794-4 http://doi.org/10.1007/s11263-006-9794-4 .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰