Learning a hierarchical image manifold for Web image classification

Rong ZHU; Min YAO; Li-hua YE; Jun-ying XUAN

doi:10.1631/jzus.C1200032

Your Location：

Home >

Browse articles >

Learning a hierarchical image manifold for Web image classification

计算机与自动化 | Updated：2022-12-09

- Learning a hierarchical image manifold for Web image classification
- Learning a hierarchical image manifold for Web image classification
- 信息与电子工程前沿（英文） 2012年13卷第10期页码：719-735
- Affiliations：
  
  1.School of Information Engineering, Jiaxing University, Jiaxing 314001, China
  2.School of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  [ "Rong ZHU, E-mail: sikexing@163.com" ]
  [ "Min YAO, E-mail: myao@zju.edu.cn" ]
- Funds：
  
  Project supported by the National High-Tech R & D Program (863) of China (No. 2009AA011900), the Zhejiang Provincial Natural Science Foundation of China (No. 2011Y1110960), and the Zhejiang Provincial Nonprofit Technology and Application Research Program of China (Nos. 2011C31045 and 2012C21020)
- DOI：10.1631/jzus.C1200032
  中图分类号： TP391
- 收稿：2012-02-12，
  
  修回：2012-;9-11，
  
  纸质出版：2012-10
- Accepted：
Scan QR Code
Learning a hierarchical image manifold for Web image classification[J]. 信息与电子工程前沿（英文）, 2012,13(10):719-735.

Rong ZHU, Min YAO, Li-hua YE, et al. Learning a hierarchical image manifold for Web image classification[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(10): 719-735.
Learning a hierarchical image manifold for Web image classification[J]. 信息与电子工程前沿（英文）, 2012,13(10):719-735. DOI： 10.1631/jzus.C1200032.

Rong ZHU, Min YAO, Li-hua YE, et al. Learning a hierarchical image manifold for Web image classification[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(10): 719-735. DOI： 10.1631/jzus.C1200032.

摘要

Abstract

Image classification is an essential task in content-based image retrieval. However

due to the semantic gap between low-level visual features and high-level semantic concepts

and the diversification of Web images

the performance of traditional classification approaches is far from users' expectations. In an attempt to reduce the semantic gap and satisfy the urgent requirements for dimensionality reduction

high-quality retrieval results

and batch-based processing

we propose a hierarchical image manifold with novel distance measures for calculation. Assuming that the images in an image set describe the same or similar object but have various scenes

we formulate two kinds of manifolds

object manifold and scene manifold

at different levels of semantic granularity. Object manifold is developed for object-level classification using an algorithm named extended locally linear embedding (ELLE) based on intra- and inter-object difference measures. Scene manifold is built for scene-level classification using an algorithm named locally linear submanifold extraction (LLSE) by combining linear perturbation and region growing. Experimental results show that our method is effective in improving the performance of classifying Web images.

关键词

Keywords

references

M. Ames , , , M. Naaman . . Why We Tag:Motivations for Annotation in Mobile and Online Media . . SIGCHI Conf. on Human Factors in Computing , , 2007 . . p.971 - - 980 . . . .

M. Belkin , , , P. Niyogi . . Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering . . Advances in Neural Information Processing Systems 14. MIT Press , , 2001 . . p.585 - - 591 . . . .

R.E. Bellman . . Adaptive Control Processes:a Guided Tour , , : : New Jersey Princeton University Press , , 1961 . . .

F. Briggs , , , R. Raich , , , X.Z. Fern . . Audio Classification of Bird Species:a Statistical Manifold Approach . . Ninth IEEE Int. Conf. on Data Mining , , 2009 . . p.51 - - 60 . . DOI: 10.1109/ICDM.2009.65 http://doi.org/10.1109/ICDM.2009.65 . .

G. Carlsson , , , T. Ishkhanov , , , V. de Silva , , , A. Zomorodian . . On the local behavior of spaces of natural images . . Int. J. Comput. Vis. , , 2008 . . 76 ( ( 1 ): ): 1 - - 12 . . DOI: 10.1007/s11263-007-0056-x http://doi.org/10.1007/s11263-007-0056-x . .

Y.M. Chai , , , X.Y. Zhu , , , S. Zhou , , , Y.T. Bian , , , F. Bu , , , W. Li , , , J. Zhu . . Ontology-Based Digital Photo Annotation Using Multi-source Information . . IEEE Int. Conf. on Computational Intelligence for Measurement Systems and Applications , , 2009 . . p.38 - - 41 . . DOI: 10.1109/CIMSA.2009.5069914 http://doi.org/10.1109/CIMSA.2009.5069914 . .

E. Chang , , , K. Goh , , , G. Sychay , , , G. Wu . . CBSA:content-based soft annotation for multimodal image retrieval using Bayes point machines . . IEEE Trans. Circ. Syst. Video Technol. , , 2003 . . 13 ( ( 1 ): ): 26 - - 38 . . DOI: 10.1109/TCSVT.2002.808079 http://doi.org/10.1109/TCSVT.2002.808079 . .

E. Cheng , , , F. Jing , , , L. Zhang . . A unified relevance feedback framework for Web image retrieval . . IEEE Trans. Image Process. , , 2009 . . 18 ( ( 6 ): ): 1350 - - 1357 . . DOI: 10.1109/TIP.2009.2017128 http://doi.org/10.1109/TIP.2009.2017128 . .

R. Datta , , , D. Joshi , , , J. Li , , , J.Z. Wang . . Image retrieval:ideas, influences, and trends of the new age . . ACM Comput. Surv. , , 2008 . . 40 ( ( 2 ): ): 1 - - 60 . . DOI: 10.1145/1348246.1348248 http://doi.org/10.1145/1348246.1348248 . .

C. de Juan , , , B. Bodenheimer . . Cartoon Textures . . Proc. ACM SIGGRAPH/Eurographics Symp. on Computer Animation , , 2004 . . p.267 - - 276 . . DOI: 10.1145/1028523.1028559 http://doi.org/10.1145/1028523.1028559 . .

D. de Ridder , , , O. Kouropteva , , , O. Okun , , , M. Pietikainen , , , R.P.W. Duin . . Supervised locally linear embedding . . LNCS , , 2003 . . 2714 175 DOI: 10.1007/3-540-44989-2_40 http://doi.org/10.1007/3-540-44989-2_40 . .

J.A. dos Santos , , , C.D. Ferreira , , , R.S. Torres , , , M.A. Goncalves , , , R.A.C. Lamparelli . . A relevance feedback method based on genetic programming for classification of remote sensing images . . Inform. Sci. , , 2011 . . 181 ( ( 13 ): ): 2671 - - 2684 . . DOI: 10.1016/j.ins.2010.02.003 http://doi.org/10.1016/j.ins.2010.02.003 . .

I. El Sayad , , , J. Martinet , , , T. Urruty , , , S. Amir , , , C. Dieraba . . Effective Object-Based Image Retrieval Using Higher-Level Visual Representation . . Int. Conf. on Machine and Web Intelligence , , 2010 . . p.218 - - 224 . . DOI: 10.1109/ICMWI.2010.5648110 http://doi.org/10.1109/ICMWI.2010.5648110 . .

P. Enser , , , C. Sandom . . Towards a Comprehensive Survey of the Semantic Gap in Visual Image Retrieval . . Int. Conf. on Image and Video Retrieval , , 2003 . . p.291 - - 299 . . DOI: 10.1007/3-540-45113-7_29 http://doi.org/10.1007/3-540-45113-7_29 . .

J.P. Fan , , , Y.L. Gao , , , H.Z. Luo , , , R. Jain . . Mining multilevel image semantic via hierarchical classification . . IEEE Trans. Multimedia , , 2008 . . 10 ( ( 2 ): ): 167 - - 187 . . DOI: 10.1109/TMM.2007.911775 http://doi.org/10.1109/TMM.2007.911775 . .

W. Fan , , , D.Y. Yeung . . Locally Linear Models on Faces Appearance Manifolds with Application to Dual-Subspace Based Classification . . IEEE Computer Society Conf. on Computer Vision and Pattern Recognition , , 2006 . . 2 1384 - - 1390 . . DOI: 10.1109/CVPR.2006.178 http://doi.org/10.1109/CVPR.2006.178 . .

M. Farajtabar , , , H.R. Rabbiee , , , A. Shaban , , , A. Soltani-Farani . . Efficient Iterative Semi-supervised Classification on Manifold . . IEEE 11th Int. Conf. on Data Mining Workshops , , 2011 . . p.228 - - 235 . . DOI: 10.1109/ICDMW.2011.181 http://doi.org/10.1109/ICDMW.2011.181 . .

B. Fischer , , , J.M. Buhmann . . Path-based clustering for grouping of smooth curves and texture segmentation . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2003 . . 25 ( ( 4 ): ): 513 - - 518 . . DOI: 10.1109/TPAMI.2003.1190577 http://doi.org/10.1109/TPAMI.2003.1190577 . .

Y. Gao , , , J.P. Fan . . Semantic Image Classification with Hierarchical Feature Subset Selection . . Proc. 7th ACM SIGMM Int. Workshop on Multimedia Information Retrieval , , 2005 . . p.135 - - 142 . . DOI: 10.1145/1101826.1101850 http://doi.org/10.1145/1101826.1101850 . .

G.D. Guo , , , A.K. Jain , , , W.Y. Ma , , , H.J. Zhang . . Learning similarity measure for natural image retrieval with relevance feedback . . IEEE Trans. Neur. Networks , , 2002 . . 13 ( ( 4 ): ): 811 - - 820 . . DOI: 10.1109/TNN.2002.1021882 http://doi.org/10.1109/TNN.2002.1021882 . .

J. Huang , , , S.R. Kumar , , , R. Zabih . . Automatic hierarchical color image classification . . EURASIP J. Appl. Signal Process. , , 2003 . . ( ( 2 ): ): 151 - - 159 . . DOI: 10.1155/S1110865703211161 http://doi.org/10.1155/S1110865703211161 . .

M.J. Huiskes , , , M.S. Lew . . The MIR Flickr Retrieval Evaluation . . Proc. 1st ACM Int. Conf. on Multimedia Information Retrieval , , 2008 . . p.39 - - 43 . . DOI: 10.1145/1460096.1460104 http://doi.org/10.1145/1460096.1460104 . .

A. Jaimes , , , J.R. Smith . . Semi-automatic, Data-Driven Construction of Multimedia Ontologies . . Proc. Int. Conf. on Multimedia and Expo , , 2003 . . 1 781 - - 784 . . DOI: 10.1109/ICME.2003.1221034 http://doi.org/10.1109/ICME.2003.1221034 . .

A. Jaimes , , , R. Jaimes , , , S.F. Chang . . Model-Based Classification of Visual Information for Content-Based Retrieval . . Conf. on Storage and Retrieval for Image and Video Databases , , 1999 . . p.402 - - 414 . . . .

A.J. Joshi , , , F. Porikli , , , N. Papanikolopoulos . . Multi-class Active Learning for Image Classification . . IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Poster Session 5 , , 2009 . . .

G. Jun , , , J. Ghosh . . Nearest-Manifold Classification with Gaussian Processes . . 20th Int. Conf. on Pattern Recognition , , 2010 . . p.914 - - 917 . . DOI: 10.1109/ICPR.2010.230 http://doi.org/10.1109/ICPR.2010.230 . .

S.D. Kang , , , S.S. Park , , , H.W. Yoo , , , Y.G. Shin , , , D.S. Jang . . Development of expert system for extraction of the objects of interest . . Exp. Syst. Appl. , , 2009 . . 36 ( ( 3 ): ): 7210 - - 7218 . . DOI: 10.1016/j.eswa.2008.09.062 http://doi.org/10.1016/j.eswa.2008.09.062 . .

B.S. Kim , , , J.Y. Park , , , A. Mohan , , , A. Gilbert , , , S. Savarese . . Hierarchical Classification of Images by Sparse Approximation . . Proc. British Machine Vision Conf. , , 2010 . . p.106.1 - - 106.11 . . DOI: 10.5244/C.25.106 http://doi.org/10.5244/C.25.106 . .

D.W. Kim , , , J.H. Song , , , J.H. Lee , , , B.G. Choi . . Support vector machine learning for region-based image retrieval with relevance feedback . . ETRI J. , , 2007 . . 29 ( ( 5 ): ): 700 - - 702 . . DOI: 10.4218/etrij.07.0207.0037 http://doi.org/10.4218/etrij.07.0207.0037 . .

T.K. Kim , , , J. Kittle , , , R. Cipolla . . Discriminative learning and recognition of image set classes using canonical correlations . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2007 . . 29 ( ( 6 ): ): 1005 - - 1008 . . DOI: 10.1109/TPAMI.2007.1037 http://doi.org/10.1109/TPAMI.2007.1037 . .

K. Klaydios . . Relevance Feedback Methods for Web Image . . PhD Thesis, Technical University of Crete, Chania, Greece , , 2004 . . .

L.J. Li , , , C. Wang , , , Y.W. Lim , , , D.M. Blei , , , F.F. Li . . Building and Using a Semantivisual Image Hierarchy . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2010 . . p.336 - - 3343 . . DOI: 10.1109/CVPR.2010.5540027 http://doi.org/10.1109/CVPR.2010.5540027 . .

X.R. Li , , , C.G.M. Snoek , , , M. Worring . . Unsupervised Multi-feature Tag Relevance Learning for Social Image Retrieval . . Proc. ACM Int. Conf. on Image and Video Retrieval , , 2010 . . p.10 - - 17 . . DOI: 10.1145/1816041.1816044 http://doi.org/10.1145/1816041.1816044 . .

Y.Q. Lin , , , F.J. Lv , , , S.H. Zhu , , , M. Yang , , , T. Cour , , , K. Yu , , , L.L. Cao , , , T. Huang . . Large-Scale Image Classification:Fast Feature Extraction and SVM Training . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2011 . . p.1689 - - 1696 . . DOI: 10.1109/CVPR.2011.5995477 http://doi.org/10.1109/CVPR.2011.5995477 . .

D. Liu , , , S.C. Yang , , , Y.D. Mu , , , X.S. Hua , , , H.J. Zhang . . Towards Optimal Discriminating Order for Multiclass Classification . . IEEE 11th Int. Conf. on Data Mining , , 2011 . . p.388 - - 397 . . DOI: 10.1109/ICDM.2011.147 http://doi.org/10.1109/ICDM.2011.147 . .

D. Lu , , , Q. Weng . . A survey of image classification methods and techniques for improving classification performance . . Int. J. Remote Sens. , , 2007 . . 28 ( ( 5 ): ): 823 - - 870 . . DOI: 10.1080/01431160600746456 http://doi.org/10.1080/01431160600746456 . .

D.J. Luo , , , H. Huang , , , C. Ding . . Discriminative High Order SVD:Adaptive Tensor Subspace Selection for Image Classification, Clustering, and Retrieval . . IEEE Int. Conf. on Computer Vision , , 2010 . . p.1443 - - 1448 . . DOI: 10.1109/ICCV.2011.6126400 http://doi.org/10.1109/ICCV.2011.6126400 . .

J.B. Luo , , , A. Singhal , , , S.P. Etz , , , R.T. Gray . . A computational approach to determination of main subject regions in photographic images . . Image Vis. Comput. , , 2004 . . 22 ( ( 3 ): ): 227 - - 241 . . DOI: 10.1016/j.imavis.2003.09.012 http://doi.org/10.1016/j.imavis.2003.09.012 . .

D. Parikh . . Recognizing Jumbled Images:the Role of Local and Global Information in Image Classification . . IEEE Int. Conf. on Computer Vision , , 2011 . . p.519 - - 526 . . DOI: 10.1109/ICCV.2011.6126283 http://doi.org/10.1109/ICCV.2011.6126283 . .

F. Patterson . . Photography and the Art of Seeing . . Baker & Taylor Books, Charlotte, North Carolina , , 1986 . . .

M. Pillati , , , C. Viroli . . Supervised Locally Linear Embedding for Classification:an Application to Gene Expression Data Analysis . . Annual Conf. of the German Classification Society , , 2005 . . p.15 - - 18 . . . .

M. Rizon , , , H. Yazid , , , P. Saad , , , A.Y.M. Shakaff , , , A.R. Saad , , , M.R. Mamat , , , S. Yaacob , , , H. Desa , , , M. Karthigayan . . Object detection using geometric invariant moment . . Am. J. Appl. Sci. , , 2006 . . 3 ( ( 6 ): ): 1876 - - 1878 . . DOI: 10.3844/ajassp.2006.1876.1878 http://doi.org/10.3844/ajassp.2006.1876.1878 . .

S.T. Roweis , , , L.K. Saul . . Nonlinear dimensional reduction by locally linear embedding . . Science , , 2000 . . 290 ( ( 5500 ): ): 2323 - - 2326 . . DOI: 10.1126/science.290.5500.2323 http://doi.org/10.1126/science.290.5500.2323 . .

Y. Rui , , , T.S. Huang , , , M. Ortega , , , S. Mehrotra . . Relevance feedback:a power tool for interactive content-based image retrieval . . IEEE Trans. Circ. Video Technol. , , 1998 . . 8 ( ( 5 ): ): 644 - - 655 . . DOI: 10.1109/76.718510 http://doi.org/10.1109/76.718510 . .

L.K. Saul , , , S.T. Roweis . . Think globally, fit locally:unsupervised learning of low dimensional manifolds . . J. Mach. Learn. Res. , , 2003 . . 4 119 - - 155 . . DOI: 10.1162/153244304322972667 http://doi.org/10.1162/153244304322972667 . .

H.S. Seung , , , D.D. Lee . . The manifold ways of perception . . Science , , 2000 . . 290 ( ( 5500 ): ): 2268 - - 2269 . . DOI: 10.1126/science.290.5500.2268 http://doi.org/10.1126/science.290.5500.2268 . .

L. Shao , , , M. Brady . . Specific object retrieval based on salient regions . . Pattern Recogn. , , 2006 . . 39 ( ( 10 ): ): 1932 - - 1948 . . DOI: 10.1016/j.patcog.2006.04.010 http://doi.org/10.1016/j.patcog.2006.04.010 . .

R. Souvenir , , , R. Pless . . Manifold Clustering . . Int. Conf. on Computer Vision , , 2005 . . p.648 - - 653 . . . .

A. Sun , , , S.S. Bhowmick , , , K.T.N. Nguyen , , , G. Bai . . Tag-based social image retrieval:an empirical evaluation . . J. Am. Soc. Inform. Sci. Technol. , , 2011 . . 62 ( ( 12 ): ): 2364 - - 2381 . . DOI: 10.1002/asi.21659 http://doi.org/10.1002/asi.21659 . .

D. Tao , , , X. Tang , , , X. Li , , , Y. Rui . . Direct kernel biased discriminant analysis:a new content-based image retrieval relevance feedback algorithm . . IEEE Trans. Multimedia , , 2006 . . 8 ( ( 4 ): ): 716 - - 727 . . DOI: 10.1109/TMM.2005.861375 http://doi.org/10.1109/TMM.2005.861375 . .

J.B. Tenenbaum , , , V.D. Silva , , , J.C. Langford . . A global geometric framework for nonlinear dimensionality reduction . . Science , , 2000 . . 290 ( ( 5500 ): ): 2319 - - 2323 . . DOI: 10.1126/science.290.5500.2319 http://doi.org/10.1126/science.290.5500.2319 . .

A. Vailaya , , , A. Jain , , , H.J. Zhang . . On image classification:city images vs . . landscapes. Pattern Recogn. , , 1998 . . 31 ( ( 12 ): ): 1921 - - 1935 . . DOI: 10.1016/S0031-3203(98)00079-X http://doi.org/10.1016/S0031-3203(98)00079-X . .

R. Vieux , , , J.P. Domenger , , , J. Benois-Pineau , , , A. Braquelaire . . Image Classification with User Defined Ontology . . 15th European Signal Processing Conf. , , 2007 . . p.723 - - 727 . . . .

P. Viola , , , M. Jones . . Robust real-time face detection . . Int. J. Comput. Vis. , , 2004 . . 57 ( ( 2 ): ): 137 - - 154 . . DOI: 10.1023/B:VISI.0000013087.49260.fb http://doi.org/10.1023/B:VISI.0000013087.49260.fb . .

C.H. Wang , , , L. Zhang , , , H.J. Zhang . . Learning to Reduce the Semantic Gap in Web Image Retrieval and Annotation . . Proc. 31st Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval , , 2008 . . p.355 - - 362 . . DOI: 10.1145/1390334.1390396 http://doi.org/10.1145/1390334.1390396 . .

L. Wang , , , X. Wang , , , J. Feng . . Subspace distance analysis with application to adaptive Bayesian algorithm for face recognition . . Pattern Recogn. , , 2006 . . 39 ( ( 3 ): ): 456 - - 464 . . DOI: 10.1016/j.patcog.2005.08.015 http://doi.org/10.1016/j.patcog.2005.08.015 . .

R.P. Wang , , , S.G. Shan , , , X.L. Chen , , , W. Gao . . Manifold-Manifold Distance with Application to Face Recognition Based on Image Set . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2008 . . p.1 - - 8 . . DOI: 10.1109/CVPR.2008.4587719 http://doi.org/10.1109/CVPR.2008.4587719 . .

Y. Wu , , , K.L. Chan . . An Extended Isomap Algorithm for Learning Multi-class Manifold . . Int. Conf. on Machine Learning and Cybernetics , , 2004 . . 6 3429 - - 3433 . . . .

M.H. Yang . . Extended Isomap for Pattern Classification . . Proc. AAAI/AAI , , 2002 . . p.224 - - 229 . . . .

Z.Y. Zeng , , , Z.Q. Yao , , , S.G. Liu . . An Efficient and Effective Image Representation for Region-Based Image Retrieval . . Proc. 2nd Int. Conf. on Interaction Sciences:Information Technology, Culture and Human , , 2009 . . p.429 - - 434 . . DOI: 10.1145/1655925.1656004 http://doi.org/10.1145/1655925.1656004 . .

S.D. Zhai , , , B. Luo , , , C.Y. Zhang . . Video abstraction based on manifold learning and mixture model . . J. Image Graph. , , 2008 . . 13 ( ( 4 ): ): 735 - - 740 . . . .

Y.J. Zhang . . Image Classification and Retrieval with Mining Technologies . . In :Song M., Wu, Y.F.B. (Eds.), Handbook of Research on Text and Web Mining Technologies, Chapter Ⅵ , , 2008 . . p.96 - - 110 . . DOI: 10.4018/978-1-59904-990-8.ch006 http://doi.org/10.4018/978-1-59904-990-8.ch006 . .

X. Zhou , , , N. Cui , , , Z. Li , , , F. Liang , , , T.S. Huang . . Hierarchical Gaussianization for Image Classification . . IEEE 12th Int. Conf. on Computer Vision , , 2009 . . p.1971 - - 1977 . . DOI: 10.1109/ICCV.2009.5459435 http://doi.org/10.1109/ICCV.2009.5459435 . .

R. Zhu , , , M. Yao . . Image feature optimization based on nonlinear dimensionality reduction . . J. Zhejiang Univ.-Sci. A , , 2009 . . 10 ( ( 12 ): ): 1720 - - 1737 . . DOI: 10.1631/jzus.A0920310 http://doi.org/10.1631/jzus.A0920310 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据