Multiclass classification based on a deep convolutional network for head pose estimation

CAI Ying; YANG Meng-long; LI Jun

doi:10.1631/FITEE.1500125

Your Location：

Home >

Browse articles >

Multiclass classification based on a deep convolutional network for head pose estimation

Computer & Automation | Updated：2022-06-06

- Multiclass classification based on a deep convolutional network for head pose estimation
- 基于深度卷积网络的多分类法在头部姿态估计中的应用
- Frontiers of Information Technology & Electronic Engineering Vol. 16, Issue 11, Pages: 930-939(2015)
- Affiliations：
  
  1.School of Computer Science, Sichuan University, Chengdu 610065, China
  2.College of Information Engineering, Sichuan Agricultural University, Yaan 625014, China
  3.School of Aeronautics and Astronautics, Sichuan University, Chengdu 610065, China
- Author bio：
  
  [ "Ying CAI, E-mail: caiying34@qq.com" ]
  Meng-long YANG, steinbeck@163.com
  [ "Jun LI, E-mail: ljun402@163.com" ]
- Funds：
- DOI：10.1631/FITEE.1500125
  CLC： TP391
- Received：20 April 2015，
  
  Revised：2015-10-16，
  
  Published：2015-11
- Accepted：
Scan QR Code
CAI Ying, YANG Meng-long, LI Jun. Multiclass classification based on a deep convolutional network for head pose estimation[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(11): 930-939.
DOI：

CAI Ying, YANG Meng-long, LI Jun. Multiclass classification based on a deep convolutional network for head pose estimation[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(11): 930-939. DOI： 10.1631/FITEE.1500125.

摘要

目的

利用深度卷积网络的优势，解决头部姿态估计中各种关键难点，并提高分类正确率。

创新点

将人工智能的新兴方法深度卷积网络应用在头部姿态估计问题上，根据姿态估计的具体问题设计一套裁剪人脸的方法，改进卷积网络模型、优化参数，并取得了大幅度的效果提升。

方法

首先，因为深度卷积网络算法对图像旋转、尺度、光照等的鲁棒性，图像预处理阶段仅对图像做简单裁剪（图3），并对比了各种裁剪法对分类正确率的影响（表1）。然后，在训练阶段使用适合姿态估计的数据处理策略，通过少量偏移裁剪框和轻微变化图像尺度来获得更多的训练数据以提升效果，在三种公开数据库上报告了实验结果并与目前取得最好效果的三种方法做了对比（表4）。最后，设计两种不同深度的网络，对比网络深度对效果的影响（表2）。

结论

针对头部姿态估计问题，提出了切实有效的新解决方案，并取得了明显改善的效果。

Abstract

Head pose estimation has been considered an important and challenging task in computer vision. In this paper we propose a novel method to estimate head pose based on a deep convolutional neural network (DCNN) for 2D face images. We design an effective and simple method to roughly crop the face from the input image

maintaining the individual-relative facial features ratio. The method can be used in various poses. Then two convolutional neural networks are set up to train the head pose classifier and then compared with each other. The simpler one has six layers. It performs well on seven yaw poses but is somewhat unsatisfactory when mixed in two pitch poses. The other has eight layers and more pixels in input layers. It has better performance on more poses and more training samples. Before training the network

two reasonable strategies including shift and zoom are executed to prepare training samples. Finally

feature extraction filters are optimized together with the weight of the classification component through training

to minimize the classification error. Our method has been evaluated on the CAS-PEAL-R1

CMU PIE

and CUBIC FacePix databases. It has better performance than state-of-the-art methods for head pose estimation.

关键词

Keywords

references

J.A.Jr. Black , M. Gargesha , K. Kahol , 等 . A framework for performance evaluation of face recognition algorithms . SPIE , 2002 . 4862 163 DOI: 10.1117/12.473032 http://doi.org/10.1117/12.473032 .

D. Ciresan , U. Meier , J. Schmidhuber . Multi-column deep neural networks for image classification . CVPR , 2012 . 3642 - 3649 . DOI: 10.1109/CVPR.2012.6248110 http://doi.org/10.1109/CVPR.2012.6248110 .

C. Farabet , C. Couprie , L. Najman , 等 . Learning hierarchical features for scene labeling . IEEE Trans. Patt. Anal. Mach. Intell , 2013 . 35 ( 8 ): 1915 - 1929 . DOI: 10.1109/TPAMI.2012.231 http://doi.org/10.1109/TPAMI.2012.231 .

Y. Fu , T.S. Huang . Graph embedded analysis for head pose estimation . 7th Int. Conf. on Automatic Face and Gesture Recognition , 2006 . 1 - 6 . DOI: 10.1109/FGR.2006.60 http://doi.org/10.1109/FGR.2006.60 .

K. Fukushima . Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position . Biol. Cybern , 1980 . 36 ( 4 ): 193 - 202 . DOI: 10.1007/BF00344251 http://doi.org/10.1007/BF00344251 .

W. Gao , B. Cao , S.G. Shan , 等 . The CAS-PEAL large-scale Chinese face database and baseline evaluations . IEEE Trans. Syst. Man. Cybern. A , 2008 . 38 ( 1 ): 149 - 161 . DOI: 10.1109/TSMCA.2007.909557 http://doi.org/10.1109/TSMCA.2007.909557 .

C. Huang , X.Q. Ding , C. Fang . Head pose estimation based on random forests for multiclass classification . ICPR , 2010 . p.934 - 937 . DOI: 10.1109/ICPR.2010.234 http://doi.org/10.1109/ICPR.2010.234 .

K. Jarrett , K. Kavukcuoglu , M. Ranzato , 等 . What is the best multi-stage architecture for object recognition . ICCV , 2009 . p.2146 - 2153 . DOI: 10.1109/ICCV.2009.5459469 http://doi.org/10.1109/ICCV.2009.5459469 .

A. Krizhevsky , I. Sutskever , G.E. Hinton . ImageNet classification with deep convolutional neural networks . NIPS , 2012 . p.1097 - 1105 . .

A. Lanitis , C.J. Taylor , T.F. Cootes , 等 . Automatic interpretation of human faces and hand gestures using flexible models . Int. Workshop on Automatic Face-and Gesture-Recognition , 1995 . p.98 - 103 . .

Y. LeCun , Y. Bengio . Convolutional networks for images, speech, and time series . In: Arbib M.A., (Ed.), The Handbook of Brain Theory and Neural Networks. MIT Press, Cambridge , 1995 .

Y. LeCun , L.D. Jackel , B. Boser , 等 . Handwritten digit recognition: applications of neural network chips and automatic learning . IEEE Commun. Mag , 1989 . 27 ( 11 ): 41 - 46 . DOI: 10.1109/35.41400 http://doi.org/10.1109/35.41400 .

Y. LeCun , I. Kanter , S.A. Solla . Eigenvalues of covariance matrices: application to neural-network learning . Phys. Rev. Lett , 1991 . 66 ( 18 ): 2396 DOI: 10.1103/PhysRevLett.66.2396 http://doi.org/10.1103/PhysRevLett.66.2396 .

Y. LeCun , L. Bottou , Y. Bengio , 等 . Gradientbased learning applied to document recognition . Proc. IEEE , 1998 . 86 ( 11 ): 2278 - 2324 . DOI: 10.1109/5.726791 http://doi.org/10.1109/5.726791 .

P. Luo , X.G. Wang , X.O. Tang . Hierarchical face parsing via deep learning . CVPR , 2012 . p.2480 - 2487 . DOI: 10.1109/CVPR.2012.6247963 http://doi.org/10.1109/CVPR.2012.6247963 .

B.P. Ma , W.C. Zhang , S.G. Shan , 等 . Robust head pose estimation using LGBP . ICPR , 2006 . p.512 - 515 . DOI: 10.1109/ICPR.2006.1006 http://doi.org/10.1109/ICPR.2006.1006 .

B.P. Ma , X.J. Chai , T.J. Wang . A novel feature descriptor based on biologically inspired feature for head pose estimation . Neurocomputing , 2013 . 115 ( 4 ): 1 - 10 . DOI: 10.1016/jneucom.2012.11.005 http://doi.org/10.1016/jneucom.2012.11.005 .

M. Matsugu , P. Cardon . Unsupervised feature selection for multi-class object detection using convolutional neural networks . ISNN , 2004 . p.864 - 869 . DOI: 10.1007/978-3-540-28647-9_142 http://doi.org/10.1007/978-3-540-28647-9_142 .

E. Murphy-Chutorian , M.M. Trivedi . Head pose estimation in computer vision: a survey . IEEE Trans. Patt. Anal. Mach. Intell , 2009 . 31 ( 4 ): 607 - 626 . DOI: 10.1109/TPAMI.2008.106 http://doi.org/10.1109/TPAMI.2008.106 .

B. Raytchev , I. Yoda , K. Sakaue . Head pose estimation by nonlinear manifold learning . ICPR , 2004 . p.462 - 466 . DOI: 10.1109/ICPR.2004.1333802 http://doi.org/10.1109/ICPR.2004.1333802 .

D. Scherer , A. Müller , S. Behnke . Evaluation of pooling operations in convolutional architectures for object recognition . Proc. 20th Int. Conf. on Artificial Neural Networks , 2010 . p.92 - 101 . DOI: 10.1007/978-3-642-15825-4_10 http://doi.org/10.1007/978-3-642-15825-4_10 .

T. Sim , S. Baker , M. Bsat . The CMU pose, illumination, and expression (PIE) database . 5th IEEE Int. Conf. on Automatic Face and Gesture Recognition , 2002 . p.46 - 51 . DOI: 10.1109/AFGR.2002.1004130 http://doi.org/10.1109/AFGR.2002.1004130 .

P.Y. Simard , D. Steinkraus , J.C. Platt . Best practices for convolutional neural networks applied to visual document analysis . 7th Int. Conf. on Document Analysis and Recognition , 2003 . p.958 - 963 . DOI: 10.1109/ICDAR.2003.1227801 http://doi.org/10.1109/ICDAR.2003.1227801 .

M. Storer , M. Urschler , H. Bischof . 3D-MAM: 3D morphable appearance model for efficient fine head pose estimation from still images . ICCV , 2009 . p.192 - 199 . DOI: 10.1109/ICCVW.2009.5457701 http://doi.org/10.1109/ICCVW.2009.5457701 .

Y. Sun , X.G. Wang , X.O. Tang . Deep convolutional network cascade for facial point detection . CVPR , 2013 . p.3476 - 3483 . DOI: 10.1109/CVPR.2013.446 http://doi.org/10.1109/CVPR.2013.446 .

Y.Q. Tang , Z.N. Sun , T.N. Tan . A survey on head pose estimation . Patt. Recogn. Artif. Intell , 2014 . 27 ( 3 ): 213 - 225 . .

J.G. Wang , E. Sung . EM enhancement of 3D head pose estimated by point at infinity . Image Vis. Comput , 2007 . 25 ( 12 ): 1864 - 1874 . DOI: 10.1016/jimavis.2005.12.017 http://doi.org/10.1016/jimavis.2005.12.017 .

X.W. Wang , X.Y. Huang , J.Z. Gao , 等 . Illumination and person-insensitive head pose estimation using distance metric learning . ECCV , 2008 . p.624 - 637 . DOI: 10.1007/978-3-540-88688-4_46 http://doi.org/10.1007/978-3-540-88688-4_46 .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Unseen head pose prediction using dense multivariate label distribution

Related Author

Gao-li SANG

Hu CHEN

Ge HUANG

Qi-jun ZHAO

Related Institution

State Key Laboratory of Fundamental Science on Synthetic Vision,College of Computer Science, Sichuan University

College of Mathematics and Information Engineering, Jiaxing University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰