基于曲率尺度空间的单视深度图像手部特征提取

Shao-fan WANG; Chun LI; De-hui KONG; Bao-cai YIN

doi:10.1631/FITEE.1500126

Your Location：

Home >

Browse articles >

基于曲率尺度空间的单视深度图像手部特征提取

常规文章 | Updated：2022-05-19

- 基于曲率尺度空间的单视深度图像手部特征提取
- Extracting hand articulations from monocular depth images using curvature scale space descriptors
- 信息与电子工程前沿（英文） 2016年17卷第1期页码：41-54
- Affiliations：
  
  1Beijing Key Laboratory of Multimedia and Intelligent Software Technology, College of Metropolitan Transportation, Beijing University of Technology, Beijing 100124, China
  2School of Software Technology, Dalian University of Technology, Dalian 116024, China
  3Collaborative Innovation Center of Electric Vehicles in Beijing, Beijing 100081, China
- Author bio：
  
  E-mail:wangshaofan@bjut.edu.cn
  E-mail:kdh@bjut.edu.cn
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 61227004, 61370120, 61390510, 61300065, and 61402024), Beijing Municipal Natural Science Foundation, China (No. 4142010), Beijing Municipal Commission of Education, China (No. km201410005013), and the Funding Project for Academic Human Resources Development in Institutions of Higher Learning under the Jurisdiction of Beijing Municipality, China
- DOI：10.1631/FITEE.1500126
  中图分类号： TP391; TP751
- 收稿：2015-04-20，
  
  录用：2015-26-10，
  
  纸质出版：2016-01
- Accepted：
Scan QR Code
基于曲率尺度空间的单视深度图像手部特征提取[J]. 信息与电子工程前沿（英文）, 2016,17(1):41-54.

Shao-fan WANG, Chun LI, De-hui KONG, et al. Extracting hand articulations from monocular depth images using curvature scale space descriptors[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(1): 41-54.
基于曲率尺度空间的单视深度图像手部特征提取[J]. 信息与电子工程前沿（英文）, 2016,17(1):41-54. DOI： 10.1631/FITEE.1500126.

Shao-fan WANG, Chun LI, De-hui KONG, et al. Extracting hand articulations from monocular depth images using curvature scale space descriptors[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(1): 41-54. DOI： 10.1631/FITEE.1500126.

摘要

目的

从深度图像、彩色图像提取手部特征（如指尖、指根、手指关节、手形轮廓）是人机交互与虚拟现实领域的重要研究课题。由于人的手部运动自由度较多，受环境光照和噪声影响较大，以及手部出现自遮挡现象，手部特征提取的研究仍亟待解决。数据手套和微软Kinect体感设备的开发，一定程度上解决了手部特征提取的问题，但前者需用户穿戴设备，后者获取精度不高。本文提出一类基于曲率尺度空间特征描述符的手部特征点定位方法，实现从单视深度图像获取手部特征点的鲁棒算法。

创新点

提出改进的曲率尺度空间特征描述符，从手形轮廓提取手指的指尖点、指谷点；通过角度区域与手形轮廓及手部深度差异计算未检测的四指指尖；通过五个指根点以及手形轮廓的起始点构成的七边形计算未检测的大拇指指尖。

方法

通过openNI对单幅深度图像提取手部部分并提取手形轮廓点。将传统的曲率尺度空间特征描述符改进为适当阈值范围内的特征点提取算法，从手形轮廓提取手指的指尖点、指谷点；对未检测的指尖点通过角度阈值进行弯曲判断，通过角度区域与手形轮廓及手部深度差异逐一计算未检测的手部特征点。

结论

与传统的基于角度阈值、轮廓凸包等方法相比，改进的曲率尺度空间特征描述鲁棒性更佳，适合从手部轮廓中提取手部的指尖点和指谷点。在此基础上通过角度区域、手形轮廓及手部深度差等方法可逐一计算未检测的手部特征点。

Abstract

We propose a framework of hand articulation detection from a monocular depth image using curvature scale space (CSS) descriptors. We extract the hand contour from an input depth image

and obtain the fingertips and finger-valleys of the contour using the local extrema of a modified CSS map of the contour. Then we recover the undetected fingertips according to the local change of depths of points in the interior of the contour. Compared with traditional appearance-based approaches using either angle detectors or convex hull detectors

the modified CSS descriptor extracts the fingertips and finger-valleys more precisely since it is more robust to noisy or corrupted data; moreover

the local extrema of depths recover the fingertips of bending fingers well while traditional appearance-based approaches hardly work without matching models of hands. Experimental results show that our method captures the hand articulations more precisely compared with three state-of-the-art appearance-based approaches.

关键词

Keywords

references

S Abbasi , , , F Mokhtarian , , , J Kittler . . Curvature scale space image in shape similarity retrieval . . Multimedia Syst. , , 1999 . . 7 ( ( 8 ): ): 467 - - 476 . . DOI: 10.1007/s005300050147 http://doi.org/10.1007/s005300050147 . .

V Athitsos , , , S Sclaroff . . An appearance-based framework for 3D hand shape classification and camera viewpoint estimation . . 2002 . . Proc. 5th IEEE Int. Conf. on Automatic Face and Gesture Recognition . . p.40 - - 45 . . DOI: 10.1109/AFGR.2002.1004129 http://doi.org/10.1109/AFGR.2002.1004129 . .

V Athitsos , , , S Sclaroff . . Estimating 3D hand pose from a cluttered image . . 2003 . . Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition . . p.432 - - 439 . . DOI: 10.1109/CVPR.2003.1211500 http://doi.org/10.1109/CVPR.2003.1211500 . .

Cerezo, T 3D hand and finger recognition using Kinect , , 2012 . . : Technical Report : Universidad de Granada, Spain , , Available at http://frantracerkinectft.codeplex.com http://frantracerkinectft.codeplex.com . . .

WY Chang , , , CS Chen , , , YD Jian . . Visual tracking in high-dimensional state space by appearanceguided particle filtering . . IEEE Trans. Image Process , , 2008 . . 17 ( ( 7 ): ): 1054 - - 1067 . . DOI: 10.1109/TIP.2008.924283 http://doi.org/10.1109/TIP.2008.924283 . .

M de La Gorce , , , DJ Fleet , , , N Paragios . . Modelbased 3D hand pose estimation from monocular video . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2011 . . 33 ( ( 9 ): ): 1793 - - 1805 . . DOI: 10.1109/TPAMI.2011.33 http://doi.org/10.1109/TPAMI.2011.33 . .

Z Feng , , , B Yang , , , Y Chen , , , 等 . . Features extraction from hand images based on new detection operators . . Patt. Recog. , , 2011 . . 44 ( ( 5 ): ): 1089 - - 1105 . . DOI: 10.1016/j.patcog.2010.08.007 http://doi.org/10.1016/j.patcog.2010.08.007 . .

C Keskin , , , F Kıraç , , , YE Kara , , , 等 . . Real time hand pose estimation using depth sensors , , 2011 . . : London : In: Fossati, A., Gall, J., Grabner, H., et al. (Eds.), Consumer Depth Cameras for Computer Vision, Springer , , p.119 - - 137 . . DOI: 10.1007/978-1-4471-4640-7_7 http://doi.org/10.1007/978-1-4471-4640-7_7 . .

F Kirac , , , YE Kara , , , L Akarun . . Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data . . Patt. Recog. Lett. , , 2014 . . 50 ( ( 3 ): ): 415 - - 422 . . DOI: 10.1016/j.patrec.2013.09.003 http://doi.org/10.1016/j.patrec.2013.09.003 . .

D Lee , , , S Lee . . Vision-based finger action recognition by angle detection and contour analysis . . ETRI J. , , 2011 . . 33 ( ( 3 ): ): 415 - - 422 . . DOI: 10.4218/etrij.11.0110.0313 http://doi.org/10.4218/etrij.11.0110.0313 . .

Z Ma , , , E Wu . . Real-time and robust hand tracking with a single depth camera . . Vis. Comput. , , 2014 . . 30 ( ( 10 ): ): 1133 - - 1144 . . DOI: 10.1007/s00371-013-0894-1 http://doi.org/10.1007/s00371-013-0894-1 . .

M Maisto , , , M Panella , , , L Liparulo , , , 等 . . An accurate algorithm for the identification of fingertips using an RGB-D camera . . IEEE J. Emerg. Sel. Topics Circ. Syst. , , 2013 . . 3 ( ( 2 ): ): 272 - - 283 . . DOI: 10.1109/JETCAS.2013.2256830 http://doi.org/10.1109/JETCAS.2013.2256830 . .

M Morshidi , , , T Tjahjadi . . Gravity optimised particle filter for hand tracking . . Patt. Recog. , , 2014 . . 47 ( ( 1 ): ): 194 - - 207 . . DOI: 10.1016/j.patcog.2013.06.032 http://doi.org/10.1016/j.patcog.2013.06.032 . .

S Nagarajan , , , T Subashini , , , V Ramalingam . . Vision based real time finger counter for hand gesture recognition . . Int. J. Technol. , , 2012 . . 2 ( ( 2 ): ): 1 - - 5 . . . .

I Oikonomidis , , , N Kyriazis , , , AA Argyros . . Efficient model-based 3D tracking of hand articulations using Kinect . . BMVC , , 2011 . . 1 ( ( 2 ): ): 1 - - 11 . . . .

C Qian , , , X Sun , , , Y Wei , , , 等 . . Realtime and robust hand tracking from depth . . 2014 . . Proc. IEEE Conf. on Computer Vision and Pattern Recognition . . p.1106 - - 1113 . . DOI: 10.1109/CVPR.2014.145 http://doi.org/10.1109/CVPR.2014.145 . .

Z Ren , , , J Yuan , , , Z Zhang . . Robust hand gesture recognition based on finger-earth moverâ€™s distance with a commodity depth camera . . 2011 . . Proc. 19th ACM Int. Conf. on Multimedia . . p.1093 - - 1096 . . DOI: 10.1145/2072298.2071946 http://doi.org/10.1145/2072298.2071946 . .

R Rosales , , , V Athitsos , , , L Sigal , , , 等 . . 3D hand pose reconstruction using specialized mappings . . 2001 . . Proc. 8th IEEE Int. Conf. on Computer Vision . . p.378 - - 385 . . DOI: 10.1109/ICCV.2001.937543 http://doi.org/10.1109/ICCV.2001.937543 . .

M Schlattmann , , , F Kahlesz , , , R Sarlette , , , 等 . . Markerless 4 gestures 6 DOF real-time visual tracking of the human hand with automatic initialization . . Comput. Graph. Forum , , 2007 . . 26 ( ( 3 ): ): 467 - - 476 . . DOI: 10.1111/j.1467-8659.2007.01069.x http://doi.org/10.1111/j.1467-8659.2007.01069.x . .

C Tomasi , , , S Petrov , , , A Sastry . . 3D tracking = classification + interpolation . . 2003 . . Proc. 9th IEEE Int. Conf. on Computer Vision . . p.1441 - - 1448 . . DOI: 10.1109/ICCV.2003.1238659 http://doi.org/10.1109/ICCV.2003.1238659 . .

J Tompson , , , M Stein , , , Y Lecun , , , 等 . . Real-time continuous pose recovery of human hands using convolutional networks . . ACM Trans. Graph. , , 2014 . . 33 ( ( 5 ): ): 169.1 - - 169.10 . . DOI: 10.1109/ICCV.2003.1238659 http://doi.org/10.1109/ICCV.2003.1238659 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据