Extracting hand articulations from monocular depth images using curvature scale space descriptors

Shao-fan WANG; Chun LI; De-hui KONG; Bao-cai YIN

doi:10.1631/FITEE.1500126

Your Location：

Home >

Browse articles >

Extracting hand articulations from monocular depth images using curvature scale space descriptors

Regular Papers | Updated：2022-05-19

- Extracting hand articulations from monocular depth images using curvature scale space descriptors
- 基于曲率尺度空间的单视深度图像手部特征提取
- Frontiers of Information Technology & Electronic Engineering Vol. 17, Issue 1, Pages: 41-54(2016)
- Affiliations：
  
  1Beijing Key Laboratory of Multimedia and Intelligent Software Technology, College of Metropolitan Transportation, Beijing University of Technology, Beijing 100124, China
  2School of Software Technology, Dalian University of Technology, Dalian 116024, China
  3Collaborative Innovation Center of Electric Vehicles in Beijing, Beijing 100081, China
- Author bio：
  
  E-mail:wangshaofan@bjut.edu.cn
  E-mail:kdh@bjut.edu.cn
- Funds：
- DOI：10.1631/FITEE.1500126
  CLC： TP391; TP751
- Received：20 April 2015，
  
  Accepted：10 February 2017，
  
  Published：2016-01
- Accepted：
Scan QR Code
Shao-fan WANG, Chun LI, De-hui KONG, et al. Extracting hand articulations from monocular depth images using curvature scale space descriptors[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(1): 41-54.
DOI：

Shao-fan WANG, Chun LI, De-hui KONG, et al. Extracting hand articulations from monocular depth images using curvature scale space descriptors[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(1): 41-54. DOI： 10.1631/FITEE.1500126.

摘要

目的

从深度图像、彩色图像提取手部特征（如指尖、指根、手指关节、手形轮廓）是人机交互与虚拟现实领域的重要研究课题。由于人的手部运动自由度较多，受环境光照和噪声影响较大，以及手部出现自遮挡现象，手部特征提取的研究仍亟待解决。数据手套和微软Kinect体感设备的开发，一定程度上解决了手部特征提取的问题，但前者需用户穿戴设备，后者获取精度不高。本文提出一类基于曲率尺度空间特征描述符的手部特征点定位方法，实现从单视深度图像获取手部特征点的鲁棒算法。

创新点

提出改进的曲率尺度空间特征描述符，从手形轮廓提取手指的指尖点、指谷点；通过角度区域与手形轮廓及手部深度差异计算未检测的四指指尖；通过五个指根点以及手形轮廓的起始点构成的七边形计算未检测的大拇指指尖。

方法

通过openNI对单幅深度图像提取手部部分并提取手形轮廓点。将传统的曲率尺度空间特征描述符改进为适当阈值范围内的特征点提取算法，从手形轮廓提取手指的指尖点、指谷点；对未检测的指尖点通过角度阈值进行弯曲判断，通过角度区域与手形轮廓及手部深度差异逐一计算未检测的手部特征点。

结论

与传统的基于角度阈值、轮廓凸包等方法相比，改进的曲率尺度空间特征描述鲁棒性更佳，适合从手部轮廓中提取手部的指尖点和指谷点。在此基础上通过角度区域、手形轮廓及手部深度差等方法可逐一计算未检测的手部特征点。

Abstract

We propose a framework of hand articulation detection from a monocular depth image using curvature scale space (CSS) descriptors. We extract the hand contour from an input depth image

and obtain the fingertips and finger-valleys of the contour using the local extrema of a modified CSS map of the contour. Then we recover the undetected fingertips according to the local change of depths of points in the interior of the contour. Compared with traditional appearance-based approaches using either angle detectors or convex hull detectors

the modified CSS descriptor extracts the fingertips and finger-valleys more precisely since it is more robust to noisy or corrupted data; moreover

the local extrema of depths recover the fingertips of bending fingers well while traditional appearance-based approaches hardly work without matching models of hands. Experimental results show that our method captures the hand articulations more precisely compared with three state-of-the-art appearance-based approaches.

关键词

Keywords

references

S Abbasi , , , F Mokhtarian , , , J Kittler . . Curvature scale space image in shape similarity retrieval . . Multimedia Syst. , , 1999 . . 7 ( ( 8 ): ): 467 - - 476 . . DOI: 10.1007/s005300050147 http://doi.org/10.1007/s005300050147 . .

V Athitsos , , , S Sclaroff . . An appearance-based framework for 3D hand shape classification and camera viewpoint estimation . . 2002 . . Proc. 5th IEEE Int. Conf. on Automatic Face and Gesture Recognition . . p.40 - - 45 . . DOI: 10.1109/AFGR.2002.1004129 http://doi.org/10.1109/AFGR.2002.1004129 . .

V Athitsos , , , S Sclaroff . . Estimating 3D hand pose from a cluttered image . . 2003 . . Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition . . p.432 - - 439 . . DOI: 10.1109/CVPR.2003.1211500 http://doi.org/10.1109/CVPR.2003.1211500 . .

Cerezo, T 3D hand and finger recognition using Kinect , , 2012 . . : Technical Report : Universidad de Granada, Spain , , Available at http://frantracerkinectft.codeplex.com http://frantracerkinectft.codeplex.com . . .

WY Chang , , , CS Chen , , , YD Jian . . Visual tracking in high-dimensional state space by appearanceguided particle filtering . . IEEE Trans. Image Process , , 2008 . . 17 ( ( 7 ): ): 1054 - - 1067 . . DOI: 10.1109/TIP.2008.924283 http://doi.org/10.1109/TIP.2008.924283 . .

M de La Gorce , , , DJ Fleet , , , N Paragios . . Modelbased 3D hand pose estimation from monocular video . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2011 . . 33 ( ( 9 ): ): 1793 - - 1805 . . DOI: 10.1109/TPAMI.2011.33 http://doi.org/10.1109/TPAMI.2011.33 . .

Z Feng , , , B Yang , , , Y Chen , , , 等 . . Features extraction from hand images based on new detection operators . . Patt. Recog. , , 2011 . . 44 ( ( 5 ): ): 1089 - - 1105 . . DOI: 10.1016/j.patcog.2010.08.007 http://doi.org/10.1016/j.patcog.2010.08.007 . .

C Keskin , , , F Kıraç , , , YE Kara , , , 等 . . Real time hand pose estimation using depth sensors , , 2011 . . : London : In: Fossati, A., Gall, J., Grabner, H., et al. (Eds.), Consumer Depth Cameras for Computer Vision, Springer , , p.119 - - 137 . . DOI: 10.1007/978-1-4471-4640-7_7 http://doi.org/10.1007/978-1-4471-4640-7_7 . .

F Kirac , , , YE Kara , , , L Akarun . . Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data . . Patt. Recog. Lett. , , 2014 . . 50 ( ( 3 ): ): 415 - - 422 . . DOI: 10.1016/j.patrec.2013.09.003 http://doi.org/10.1016/j.patrec.2013.09.003 . .

D Lee , , , S Lee . . Vision-based finger action recognition by angle detection and contour analysis . . ETRI J. , , 2011 . . 33 ( ( 3 ): ): 415 - - 422 . . DOI: 10.4218/etrij.11.0110.0313 http://doi.org/10.4218/etrij.11.0110.0313 . .

Z Ma , , , E Wu . . Real-time and robust hand tracking with a single depth camera . . Vis. Comput. , , 2014 . . 30 ( ( 10 ): ): 1133 - - 1144 . . DOI: 10.1007/s00371-013-0894-1 http://doi.org/10.1007/s00371-013-0894-1 . .

M Maisto , , , M Panella , , , L Liparulo , , , 等 . . An accurate algorithm for the identification of fingertips using an RGB-D camera . . IEEE J. Emerg. Sel. Topics Circ. Syst. , , 2013 . . 3 ( ( 2 ): ): 272 - - 283 . . DOI: 10.1109/JETCAS.2013.2256830 http://doi.org/10.1109/JETCAS.2013.2256830 . .

M Morshidi , , , T Tjahjadi . . Gravity optimised particle filter for hand tracking . . Patt. Recog. , , 2014 . . 47 ( ( 1 ): ): 194 - - 207 . . DOI: 10.1016/j.patcog.2013.06.032 http://doi.org/10.1016/j.patcog.2013.06.032 . .

S Nagarajan , , , T Subashini , , , V Ramalingam . . Vision based real time finger counter for hand gesture recognition . . Int. J. Technol. , , 2012 . . 2 ( ( 2 ): ): 1 - - 5 . . . .

I Oikonomidis , , , N Kyriazis , , , AA Argyros . . Efficient model-based 3D tracking of hand articulations using Kinect . . BMVC , , 2011 . . 1 ( ( 2 ): ): 1 - - 11 . . . .

C Qian , , , X Sun , , , Y Wei , , , 等 . . Realtime and robust hand tracking from depth . . 2014 . . Proc. IEEE Conf. on Computer Vision and Pattern Recognition . . p.1106 - - 1113 . . DOI: 10.1109/CVPR.2014.145 http://doi.org/10.1109/CVPR.2014.145 . .

Z Ren , , , J Yuan , , , Z Zhang . . Robust hand gesture recognition based on finger-earth moverâ€™s distance with a commodity depth camera . . 2011 . . Proc. 19th ACM Int. Conf. on Multimedia . . p.1093 - - 1096 . . DOI: 10.1145/2072298.2071946 http://doi.org/10.1145/2072298.2071946 . .

R Rosales , , , V Athitsos , , , L Sigal , , , 等 . . 3D hand pose reconstruction using specialized mappings . . 2001 . . Proc. 8th IEEE Int. Conf. on Computer Vision . . p.378 - - 385 . . DOI: 10.1109/ICCV.2001.937543 http://doi.org/10.1109/ICCV.2001.937543 . .

M Schlattmann , , , F Kahlesz , , , R Sarlette , , , 等 . . Markerless 4 gestures 6 DOF real-time visual tracking of the human hand with automatic initialization . . Comput. Graph. Forum , , 2007 . . 26 ( ( 3 ): ): 467 - - 476 . . DOI: 10.1111/j.1467-8659.2007.01069.x http://doi.org/10.1111/j.1467-8659.2007.01069.x . .

C Tomasi , , , S Petrov , , , A Sastry . . 3D tracking = classification + interpolation . . 2003 . . Proc. 9th IEEE Int. Conf. on Computer Vision . . p.1441 - - 1448 . . DOI: 10.1109/ICCV.2003.1238659 http://doi.org/10.1109/ICCV.2003.1238659 . .

J Tompson , , , M Stein , , , Y Lecun , , , 等 . . Real-time continuous pose recovery of human hands using convolutional networks . . ACM Trans. Graph. , , 2014 . . 33 ( ( 5 ): ): 169.1 - - 169.10 . . DOI: 10.1109/ICCV.2003.1238659 http://doi.org/10.1109/ICCV.2003.1238659 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰