Image-based 3D model retrieval using manifold learning

Pan-pan MU; San-yuan ZHANG; Yin ZHANG; Xiu-zi YE; Xiang PAN

doi:10.1631/FITEE.1601764

Your Location：

Home >

Browse articles >

Image-based 3D model retrieval using manifold learning

Regular Papers | Updated：2022-05-19

- Image-based 3D model retrieval using manifold learning
- 利用流形学习进行基于图像的三维模型检索
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 11, Pages: 1397-1408(2018)
- Affiliations：
  
  College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
  College of Mathematics and Information Science, Wenzhou University, Wenzhou 325003, China
  College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China
- Author bio：
  
  [ "Pan-pan MU, E-mail: mupanpan0927@zju.edu.cn" ]
- Funds：
- DOI：10.1631/FITEE.1601764
  CLC： TP391
- Received：01 December 2016，
  
  Revised：2018-11-27，
  
  Published：2018-11
- Accepted：
Scan QR Code
Pan-pan MU, San-yuan ZHANG, Yin ZHANG, et al. Image-based 3D model retrieval using manifold learning[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(11): 1397-1408.
DOI：

Pan-pan MU, San-yuan ZHANG, Yin ZHANG, et al. Image-based 3D model retrieval using manifold learning[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(11): 1397-1408. DOI： 10.1631/FITEE.1601764.

摘要

提出一种基于图像的三维模型检索新框架。把查询的图像建模为欧氏空间的一个点，把三维模型所有投影视图建模为一个对称正定矩阵，并视该对称正定矩阵为黎曼流形上的一个点。于是，基于图像的三维模型检索简化为欧氏空间到黎曼流形上的度量学习。为解决异质匹配问题，把欧式空间和对称正定矩阵黎曼空间映射到同一个高维希尔伯特空间，极大缩小彼此之间语义鸿沟。最后，使用核方法设计一个优化算法学习映射。任何新图像描述符，比如深度学习特征描述符，可以很容易嵌入该框架。实验结果表明，该方法相较目前最新基于图像的三维模型检索方法有一定优势。

Abstract

We propose a new framework for image-based three-dimensional (3D) model retrieval. We first model the query image as a Euclidean point. Then we model all projected views of a 3D model as a symmetric positive definite (SPD) matrix

which is a point on a Riemannian manifold. Thus

the image-based 3D model retrieval is reduced to a problem of Euclid-to-Riemann metric learning. To solve this heterogeneous matching problem

we map the Euclidean space and SPD Riemannian manifold to the same high-dimensional Hilbert space

thus shrinking the great gap between them. Finally

we design an optimization algorithm to learn a metric in this Hilbert space using a kernel trick. Any new image descriptors

such as the features from deep learning

can be easily embedded in our framework. Experimental results show the advantages of our approach over the state-of-the-art methods for image-based 3D model retrieval.

关键词

Keywords

references

S Bai , , , X Bai , , , Z Zhou , , , 等 . . GIFT: a real-time and scalable 3D shape search engine . . 16 th IEEE Conf on Computer Vision and Pattern Recognition , , 2016 . . p.5023 - - 5032 . . DOI: 10.1109/CVPR.2016.543 http://doi.org/10.1109/CVPR.2016.543 . .

X Bai , , , S Bai , , , Z Zhu , , , 等 . . 3D shape matching via two layer coding . . IEEE Trans Patt Anal Mach Intell , , 2015 . . 37 ( ( 12 ): ): 2361 - - 2373 . . DOI: 10.1109/TPAMI.2015.2424863 http://doi.org/10.1109/TPAMI.2015.2424863 . .

H Cevikalp , , , B Triggs . . Face recognition based on image sets . . IEEE Society Conf on Computer Vision and Pattern Recognition , , 2010 . . p.2567 - - 2573 . . DOI: 10.1109/CVPR.2010.5539965 http://doi.org/10.1109/CVPR.2010.5539965 . .

K Chatfield , , , K Simonyan , , , A Vedaldi , , , 等 . . Return of the devil in the details: delving deep into convolutional nets , , 2014 . . p.1 - - 11 . . https://arxiv.org/abs/1405.3531 https://arxiv.org/abs/1405.3531 , , . .

DY Chen , , , XP Tian , , , YT Shen , , , 等 . . On visual similarity based 3D model retrieval . . Comput Graph Forum , , 2003 . . 22 ( ( 3 ): ): 223 - - 232 . . DOI: 10.1111/1467-8659.00669 http://doi.org/10.1111/1467-8659.00669 . .

JT Chien , , , CC Wu . . Discriminant waveletfaces and nearest feature classifiers for face recognition . . IEEE Trans Patt Anal Mach Intell , , 2002 . . 24 ( ( 12 ): ): 1644 - - 1649 . . DOI: 10.1109/TPAMI.2002.1114855 http://doi.org/10.1109/TPAMI.2002.1114855 . .

M Eitz , , , R Richter , , , T Boubekeur , , , 等 . . Sketch-based shape retrieval . . ACM Trans Graph , , 2012 . . 31 ( ( 4 ): ): 31 - - 40 . . DOI: 10.1145/2185520.2185527 http://doi.org/10.1145/2185520.2185527 . .

T Furuya , , , R Ohbuchi . . Ranking on cross-domain manifold for sketch-based 3D model retrieval . . Int Conf on Cyberworlds , , 2013 . . p.274 - - 281 . . DOI: 10.1109/CW.2013.60 http://doi.org/10.1109/CW.2013.60 . .

J Hamm , , , DD Lee . . Grassmann discriminant analysis: a unifying view on subspace-based learning . . Proc 25 th Int Conf on Machine Learning , , 2008 . . p.376 - - 383 . . DOI: 10.1145/1390156.1390204 http://doi.org/10.1145/1390156.1390204 . .

J Hamm , , , DD Lee . . Extended Grassmann kernels for subspace-based learning . . Advances in Neural Information Processing Systems , , 2009 . . p.601 - - 608 . . . .

Z Huang , , , R Wang , , , S Shan , , , 等 . . Learning Euclideanto-Riemannian metric for point-to-set classification . . IEEE Conf on Computer Vision and Pattern Recognition , , 2014 . . p.1677 - - 1684 . . DOI: 10.1109/CVPR.2014.217 http://doi.org/10.1109/CVPR.2014.217 . .

S Jayasumana , , , R Hartley , , , M Salzmann , , , 等 . . Kernel methods on the Riemannian manifold of symmetric positive definite matrices . . IEEE Conf on Computer Vision and Pattern Recognition , , 2013 . . p.73 - - 80 . . DOI: 10.1109/CVPR.2013.17 http://doi.org/10.1109/CVPR.2013.17 . .

M Kazhdan , , , T Funkhouser , , , S Rusinkiewicz . . Rotation invariant spherical harmonic representation of 3D shape descriptors . . Proc Eurographics/ACM SIGGRAPH Symp on Geometry Processing , , 2003 . . p.156 - - 164 . . . .

T Kim , , , J Kittler , , , R Cipolla . . Discriminative learning and recognition of image set classes using canonical correlations . . IEEE Trans Patt Anal Mach Intell , , 2007 . . 29 ( ( 6 ): ): 1005 - - 1018 . . DOI: 10.1109/TPAMI.2007.1037 http://doi.org/10.1109/TPAMI.2007.1037 . .

B Li , , , Y Lu , , , A Godil , , , 等 . . A comparison of methods for sketch-based 3D shape retrieval . . Comput Vis Image Underst , , 2014 . . 119 57 - - 80 . . DOI: 10.1016/j.cviu.2013.11.008 http://doi.org/10.1016/j.cviu.2013.11.008 . .

Z Lian , , , A Godil , , , X Sun , , , 等 . . CM-BOF: visual similarity-based 3D shape retrieval using clock matching and bag-of-features . . Mach Vis Appl , , 2013 . . 24 ( ( 8 ): ): 1685 - - 1704 . . DOI: 10.1007/s00138-013-0501-5 http://doi.org/10.1007/s00138-013-0501-5 . .

P Mu , , , S Zhang , , , X Ye . . A metric learning method for image-based 3D shape retrieval . . Proc Int Conf on Data Mining, Communications and Information Technology, Article 17 , , 2017 . . DOI: 10.1145/3089871.3089876 http://doi.org/10.1145/3089871.3089876 . .

R Ohbuchi , , , K Osada , , , T Furuya , , , 等 . . Salient local visual features for shape-based 3D model retrieval . . IEEE Int Conf on Shape Modeling and Applications , , 2008 . . p.93 - - 102 . . DOI: 10.1109/SMI.2008.4547955 http://doi.org/10.1109/SMI.2008.4547955 . .

P Papadakis , , , I Pratikakis , , , T Theoharis , , , 等 . . Panorama: a 3D shape descriptor based on panoramic views for unsupervised 3D object retrieval . . Int J Comput Vis , , 2010 . . 89 ( ( 2-3 ): ): 177 - - 192 . . DOI: 10.1007/s11263-009-0281-6 http://doi.org/10.1007/s11263-009-0281-6 . .

JM Saavedra , , , B Bustos , , , T Schreck , , , 等 . . Sketch-based 3D model retrieval using keyshapes for global and local representation . . Proc 5 th Eurographics Conf on 3D Object Retrieval , , 2012 . . p.47 - - 50 . . DOI: 10.2312/3DOR/3DOR12/047-050 http://doi.org/10.2312/3DOR/3DOR12/047-050 . .

P Shilane , , , P Min , , , M Kazhdan , , , 等 . . The Princeton Shape Benchmark . . Proc Shape Modeling Applications , , 2004 . . p.167 - - 178 . . DOI: 10.1109/SMI.2004.1314504 http://doi.org/10.1109/SMI.2004.1314504 . .

P Sousa , , , MJ Fonseca . . Sketch-based retrieval of drawings using spatial proximity . . J Vis Lang Comput , , 2010 . . 21 ( ( 2 ): ): 69 - - 80 . . DOI: 10.1016/j.jvlc.2009.12.001 http://doi.org/10.1016/j.jvlc.2009.12.001 . .

H Su , , , S Maji , , , E Kalogerakis , , , 等 . . Multi-view convolutional neural networks for 3D shape recognition . . IEEE Int Conf on Computer Vision , , 2015 . . p.945 - - 953 . . DOI: 10.1109/ICCV.2015.114 http://doi.org/10.1109/ICCV.2015.114 . .

H Tabia , , , H Laga , , , D Picard , , , 等 . . Covariance descriptors for 3D shape matching and retrieval . . IEEE Conf on Computer Vision and Pattern Recognition , , 2014 . . p.4185 - - 4192 . . DOI: 10.1109/CVPR.2014.533 http://doi.org/10.1109/CVPR.2014.533 . .

R Vemulapalli , , , JK Pillai , , , R Chellappa . . Kernel learning for extrinsic classification of manifold features . . IEEE Conf on Computer Vision and Pattern Recognition , , 2013 . . p.1782 - - 1789 . . DOI: 10.1109/CVPR.2013.233 http://doi.org/10.1109/CVPR.2013.233 . .

P Vincent , , , Y Bengio . . K-local hyperplane and convex distance nearest neighbor algorithms . . Proc 14 th Int Conf on Neural Information Processing Systems: Natural and Synthetic , , 2001 . . p.985 - - 992 . . . .

F Wang , , , L Kang , , , Y Li . . Sketch-based 3D shape retrieval using convolutional neural networks . . IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.1875 - - 1883 . . DOI: 10.1109/CVPR.2015.7298797 http://doi.org/10.1109/CVPR.2015.7298797 . .

R Wang , , , H Guo , , , LS Davis , , , 等 . . Covariance discriminative learning: a natural and efficient approach to image set classification . . IEEE Conf on Computer Vision and Pattern Recognition , , 2012 . . p.2496 - - 2503 . . DOI: 10.1109/CVPR.2012.6247965 http://doi.org/10.1109/CVPR.2012.6247965 . .

Y Wen , , , K Zhang , , , Z Li , , , 等 . . A discriminative feature learning approach for deep face recognition . . European Conf on Computer Vision , , 2016 . . p.499 - - 515 . . DOI: 10.1007/978-3-319-46478-7_31 http://doi.org/10.1007/978-3-319-46478-7_31 . .

Z Wu , , , S Song , , , A Khosla , , , 等 . . 3D shapenets: a deep representation for volumetric shapes . . IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.1912 - - 1920 . . DOI: 10.1109/CVPR.2015.7298801 http://doi.org/10.1109/CVPR.2015.7298801 . .

O Yamaguchi , , , K Fukui , , , K Maeda . . Face recognition using temporal image sequence . . Proc 3 rd IEEE Int Conf on Automatic Face and Gesture Recognition , , 1998 . . p.318 - - 323 . . DOI: 10.1109/AFGR.1998.670968 http://doi.org/10.1109/AFGR.1998.670968 . .

P Zhu , , , L Zhang , , , W Zuo , , , 等 . . From point to set: extend the learning of distance metrics . . IEEE Int Conf on Computer Vision , , 2013 . . p.2664 - - 2671 . . DOI: 10.1109/ICCV.2013.331 http://doi.org/10.1109/ICCV.2013.331 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Face recognition based on subset selection via metric learning on manifold

Related Author

SHAO Hong

CHEN Shuang

ZHAO Jie-yi

CUI Wen-cheng

YU Tian-shu

Related Institution

School of Information Science and Engineering, Shenyang University of Technology

The University of Texas Health Science Center at Houston

Schulich School of Engineering, University of Calgary

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰