基于特定区域纹理描述和面部动态变化的自发性微笑判别技术

吴平平; 刘宏; Xue-wu ZHANG; Yuan GAO

doi:10.1631/FITEE.1600041

Your Location：

Home >

Browse articles >

基于特定区域纹理描述和面部动态变化的自发性微笑判别技术

常规文章 | Updated：2022-05-19

- 基于特定区域纹理描述和面部动态变化的自发性微笑判别技术
- Spontaneous versus posed smile recognition via region-specific texture descriptor and geometric facial dynamics
- 信息与电子工程前沿（英文） 2017年18卷第7期页码：955-967
- Affiliations：
  
  MOE Key Laboratory of Machine Perception, Peking University, Beijing 100871, China
  Engineering Lab on Intelligent Perception for Internet of Things, Peking University Shenzhen Graduate School, Shenzhen 518055, China
  Department of Computer Science, Christian-Albrechts University, Kiel 24118, Germany
- Author bio：
  
  [ "Ping-ping WU, E-mail:pingpingwu@pku.edu.cn" ]
  Hong LIU, E-mail:hongliu@pku.edu.cn
  [ "Xue-wu ZHANG, E-mail:zhangxuewu@sz.pku.edu.cn" ]
  [ "Yuan GAO, E-mail:yuan.gao@stu.uni-kiel.de" ]
- Funds：
  
  Project supported by the National Natural Science Foundation of China (No. 60675025), the National High-Tech R & D Program (863) of China (No. 2006AA04Z247), the Scientific and Technical Innovation Commission of Shenzhen Municipality, China (Nos. JCYJ20130331144631730 and JCYJ20130331144716089), and the Specialized Research Fund for the Doctoral Program of Higher Education, China (No. 20130001110011)
- DOI：10.1631/FITEE.1600041
  中图分类号： TP37
- 纸质出版日期：2017-07，
  
  收稿日期：2016-01-27，
  
  修回日期：2017-06-03，
- Accepted：
Scan QR Code
吴平平, 刘宏, Xue-wu ZHANG, 等. 基于特定区域纹理描述和面部动态变化的自发性微笑判别技术[J]. 信息与电子工程前沿（英文）, 2017,18(7):955-967.

PING-PING WU, HONG LIU, XUE-WU ZHANG, et al. Spontaneous versus posed smile recognition via region-specific texture descriptor and geometric facial dynamics. [J]. Frontiers of information technology & electronic engineering, 2017, 18(7): 955-967.
吴平平, 刘宏, Xue-wu ZHANG, 等. 基于特定区域纹理描述和面部动态变化的自发性微笑判别技术[J]. 信息与电子工程前沿（英文）, 2017,18(7):955-967. DOI： 10.1631/FITEE.1600041.

PING-PING WU, HONG LIU, XUE-WU ZHANG, et al. Spontaneous versus posed smile recognition via region-specific texture descriptor and geometric facial dynamics. [J]. Frontiers of information technology & electronic engineering, 2017, 18(7): 955-967. DOI： 10.1631/FITEE.1600041.

摘要

微笑作为一种典型的生物多样性特征信号，在社会交往中有较大影响力，它揭示了人的情感感受和内心状态。自发性的微笑与假笑由不同大脑系统发出，在形态学和动力学上均存在差异。区分这两种类型的微笑仍具有挑战性，因为其中细微差别很难被肉眼观察到，仍有待被识别捕捉。已有相关研究大多是提取自发性微笑的几何特征，而外观特征并没有被充分利用，导致纹理信息的丢失。本文提出一种基于特定区域纹理描述来表示不同面部区域的局部模式变化，从而弥补几何特征研究的局限性。每个面部区域的时间相位是通过计算相应的面部区域强度来划分，而非仅考虑嘴巴区域强度。同时利用支持向量机的中层融合策略，将两种特征类型结合起来。实验结果表明，本文提出的外观表示法及其与基于几何形状的人脸动力学的结合技术，在BBC、SPOS、MMI和UvA-NEMO四个基准数据库中得到很好的应用。

Abstract

As a typical biometric cue with great diversities

smile is a fairly influential signal in social interaction

which reveals the emotional feeling and inner state of a person. Spontaneous and posed smiles initiated by different brain systems have differences in both morphology and dynamics. Distinguishing the two types of smiles remains challenging as discriminative subtle changes need to be captured

which are also uneasily observed by human eyes. Most previous related works about spontaneous versus posed smile recognition concentrate on extracting geometric features while appearance features are not fully used

leading to the loss of texture information. In this paper

we propose a region-specific texture descriptor to represent local pattern changes of different facial regions and compensate for limitations of geometric features. The temporal phase of each facial region is divided by calculating the intensity of the corresponding facial region rather than the intensity of only the mouth region. A mid-level fusion strategy of support vector machine is employed to combine the two feature types. Experimental results show that both our proposed appearance representation and its combination with geometry-based facial dynamics achieve favorable performances on four baseline databases: BBC

SPOS

MMI

and UvA-NEMO.

关键词

面部特征定位几何特征外貌特征笑容识别

Keywords

Facial landmark localizationGeometric featureAppearance featureSmile recognition

references

Z. Ambadar, , , J.F. Cohn, , , L. Ian Reed. . All smiles are not created equal: morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. . J. Nonverb. Behav., , 2009. . 33((1):):17--34. . DOI:10.1007/s10919-008-0059-5http://doi.org/10.1007/s10919-008-0059-5..

S. Baron-Cohen, , , H.A. Ring, , , E.T. Bullmore, , , 等. . The amygdala theory of autism. . Neurosci. Biobehav. Rev., , 2000. . 24((3):):355--364. . DOI:10.1016/S0149-7634(00)00011-7http://doi.org/10.1016/S0149-7634(00)00011-7..

X.P. Burgos-Artizzu, , , P. Perona, , , P. Dollár. . Robust face landmark estimation under occlusion. . Proc. IEEE Int. Conf. on Computer Vision, , 2013. . p.1513--1520. . DOI:10.1109/ICCV.2013.191http://doi.org/10.1109/ICCV.2013.191..

M.G. Calvo, , , L. Nummenmaa. . Time course of discrimination between emotional facial expressions: the role of visual saliency. . Vis. Res., , 2011. . 51((15):):1751--1759. . DOI:10.1016/j.visres.2011.06.001http://doi.org/10.1016/j.visres.2011.06.001..

M.G. Calvo, , , A. Gutiérrez-García, , , P. Avero, , , 等. . Attentional mechanisms in judging genuine and fake smiles: eye-movement patterns. . Emotion, , 2013. . 13((4):):792--802. . DOI:10.1037/a0032317http://doi.org/10.1037/a0032317..

X. Cao, , , Y. Wei, , , F. Wen, , , 等. . Face alignment by explicit shape regression. . Int. J. Comput. Vis., , 2014. . 107((2):):177--190. . DOI:10.1007/s11263-013-0667-3http://doi.org/10.1007/s11263-013-0667-3..

J.F. Cohn, , , K.L. Schmidt. . The timing of facial motion in posed and spontaneous smiles. . Int. J. Wavel. Multiresol. Inform. Process., , 2004. . 2((2):):121--132. . DOI:10.1142/S021969130400041Xhttp://doi.org/10.1142/S021969130400041X..

T.F. Cootes, , , G.J. Edwards, , , C.J. Taylor. . Active appearance models. . IEEE Trans. Patt. Anal. Mach. Intell., , 2001. . 23((6):):681--685. . DOI:10.1109/34.927467http://doi.org/10.1109/34.927467..

N. Dalal, , , B. Triggs. . Histograms of oriented gradients for human detection. . Proc. IEEE Conf. on Computer Vision and Pattern Recognition, , 2005. . p.886--893. . DOI:10.1109/CVPR.2005.177http://doi.org/10.1109/CVPR.2005.177..

H. Dibeklioğlu, , , R. Valenti, , , A.A. Salah, , , 等. . Eyes do not lie: spontaneous versus posed smiles. . Proc. Int. Conf. on Multimedia, , 2010. . p.703--706. . DOI:10.1145/1873951.1874056http://doi.org/10.1145/1873951.1874056..

H. Dibeklioğlu, , , A. Salah, , , T. Gevers. . Are you really smiling at me?. . Spontaneous versus posed enjoymentsmiles. Proc. European Conf. on Computer Vision, , 2012. . p.525--538. . DOI:10.1007/978-3-642-33712-3_38http://doi.org/10.1007/978-3-642-33712-3_38..

H. Dibeklioğlu, , , A. Salah, , , T. Gevers. . Recognition of genuine smiles. . IEEE Trans. Multimed., , 2015. . 17((3):):279--294. . DOI:10.1109/TMM.2015.2394777http://doi.org/10.1109/TMM.2015.2394777..

P. Dollár, , , P. Welinder, , , P. Perona. . Cascaded pose regression. . Proc. IEEE Conf. on Computer Vision and Pattern Recognition, , 2010. . p.1078--1085. . DOI:10.1109/CVPR.2010.5540094http://doi.org/10.1109/CVPR.2010.5540094..

P. Ekman. . Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage. . W. W. Norton & Company, New York, , 2009. . p.140--143. . ..

P. Ekman, , , E.L. Rosenberg. . What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS), , Oxford University Press, , 1997. ..

M.G. Frank, , , P. Ekman. . Not all smiles are created equal: the differences between enjoyment and nonenjoyment smiles. . Humor, , 1993. . 6((1):):9--26. . DOI:10.1515/humr.1993.6.1.9http://doi.org/10.1515/humr.1993.6.1.9..

M. Hoque, , , D. McDuff, , , R. Picard. . Exploring temporal patterns in classifying frustrated and delighted smiles. . IEEE Trans. Affect. Comput., , 2012. . 3((3):):323--334. . DOI:10.1109/T-AFFC.2012.11http://doi.org/10.1109/T-AFFC.2012.11..

M.R. Khokher, , , A. Bouzerdoum, , , S.L. Phung. . Crowd behavior recognition using dense trajectories. . Proc. Int. Conf. on Digital lmage Computing: Techniques and Applications, , 2014. . p.1--7. . DOI:10.1109/DICTA.2014.7008098http://doi.org/10.1109/DICTA.2014.7008098..

V. Le, , , J. Brandt, , , Z. Lin, , , 等. . Interactive facial feature localization. . Proc. European Conf. on Computer Vision, , 2012. . p.679--692. . DOI:10.1007/978-3-642-33712-3_49http://doi.org/10.1007/978-3-642-33712-3_49..

W.S. Li, , , C.L. Zhou, , , J.T. Xu. . A novel face recognition method with feature combination. . J. Zhejiang Univ.-Sci., , 2005. . 6A((5):):454--459. . DOI:10.1631/jzus.2005.A0454http://doi.org/10.1631/jzus.2005.A0454..

H. Liu, , , X. Sun. . A partial least squares based ranker for fast and accurate age estimation. . Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, , 2016. . p.2792--2796. . DOI:10.1109/ICASSP.2016.7472186http://doi.org/10.1109/ICASSP.2016.7472186..

H. Liu, , , P. Wu. . Comparison of methods for smile deceit detection by training AU6 and AU12 simultaneously. . Proc. IEEE Int. Conf. on Image Processing, , 2012. . p.1805--1808. . DOI:10.1109/ICIP.2012.6467232http://doi.org/10.1109/ICIP.2012.6467232..

H. Liu, , , Y. Gao, , , C. Wang. . Gender identification in unconstrained scenarios using self-similarity of gradients features. . Proc. IEEE Int. Conf. on Image Processing, , 2014. . p.5911--5915. . DOI:10.1109/ICIP.2014.7026194http://doi.org/10.1109/ICIP.2014.7026194..

A. Miehlke, , , U. Fisch, , , C.M. Eneroth. . Surgery of the Facial Nerve. . Saunders, Philadelphia., , 1973. ..

T. Ojala, , , M. Pietikäinen, , , T. Mäenpää. . Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. . IEEE Trans. Patt. Anal. Mach. Intell., , 2002. . 24((7):):971--987. . DOI:10.1109/TPAMI.2002.1017623http://doi.org/10.1109/TPAMI.2002.1017623..

H. Peng, , , F. Long, , , C. Ding. . Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. . IEEE Trans. Patt. Anal. Mach. Intell., , 2005. . 27((8):):1226--1238. . DOI:10.1109/TPAMI.2005.159http://doi.org/10.1109/TPAMI.2005.159..

T. Pfister, , , X. Li, , , G. Zhao, , , 等. . Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework. . Proc. IEEE Int. Conf. on Computer Vision Workshops, , 2011a. . p.868--875. . DOI:10.1109/ICCVW.2011.6130343http://doi.org/10.1109/ICCVW.2011.6130343..

T. Pfister, , , X. Li, , , G. Zhao, , , 等. . Recognising spontaneous facial micro-expressions. . Proc. IEEE Int. Conf. on Computer Vision, , 2011b. . p.1449--1456. . DOI:10.1109/ICCV.2011.6126401http://doi.org/10.1109/ICCV.2011.6126401..

W.E. Rinn. . The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions. . Psychol. Bull., , 1984. . 95((1):):52--77. . DOI:10.1037/0033-2909.95.1.52http://doi.org/10.1037/0033-2909.95.1.52..

E. Sariyanidi, , , H. Gunes, , , A. Cavallaro. . Automatic analysis of facial affect: a survey of registration, representation, and recognition. . IEEE Trans. Patt. Anal. Mach. Intell., , 2015. . 37((6):):1113--1133. . DOI:10.1109/TPAMI.2014.2366127http://doi.org/10.1109/TPAMI.2014.2366127..

X.B. Shen, , , Q. Wu, , , X.L. Fu. . Effects of the duration of expressions on the recognition of microexpressions. . J. Zhejiang Univ.-Sci. B, , 2012. . 13((3):):221--230. . DOI:10.1631/jzus.B1100063http://doi.org/10.1631/jzus.B1100063..

M. Valstar, , , M. Pantic. . Induced disgust, happiness and surprise: an addition to the MMI facial expression database. . Proc. 3rd Int. Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect, , 2010. . p.65--70. . ..

M.F. Valstar, , , H. Gunes, , , M. Pantic. . How to distinguish posed from spontaneous smiles using geometric features. . Proc. Int. Conf. on Multimodal Interfaces, , 2007. . p.38--45. . DOI:10.1145/1322192.1322202http://doi.org/10.1145/1322192.1322202..

J. Wang, , , J. Yang, , , K. Yu, , , 等. . Locality-constrained linear coding for image classification. . Proc. IEEE Conf. on Computer Vision and Pattern Recognition, , 2010. . p.3360--3367. . DOI:10.1109/CVPR.2010.5540018http://doi.org/10.1109/CVPR.2010.5540018..

X. Wang, , , L. Wang, , , Y. Qiao. . A comparative study of encoding, pooling and normalization methods for action recognition. . Proc. Asian Conf. on Computer Vision, , 2012. . p.572--585. . DOI:10.1007/978-3-642-37431-9_44http://doi.org/10.1007/978-3-642-37431-9_44..

J. Whitehill, , , M.S. Bartlett, , , J.R. Movellan. . Automatic facial expression recognition. . In: Gratch J., Marsella, S. (Eds.), Social Emotions in Nature Artifact. Oxford Scholarship Online., , 2013. . DOI:10.1093/acprof:oso/9780195387643.003.0007http://doi.org/10.1093/acprof:oso/9780195387643.003.0007..

P.P. Wu, , , H. Liu, , , X.W. Zhang. . Spontaneous versus posed smile recognition using discriminative local spatio-temporal descriptors. . Proc. Int. IEEE Conf. on Acoustics, Speech and Signal Processing, , 2014. . p.1249--1253. . DOI:10.1109/ICASSP.2014.6853795http://doi.org/10.1109/ICASSP.2014.6853795..

Q. Wu, , , X.B. Shen, , , X.L. Fu. . The machine knows what you are hiding: an automatic micro-expression recognition system. . LNCS, , 2011. . 6975152--162. . DOI:10.1007/978-3-642-24571-8_16http://doi.org/10.1007/978-3-642-24571-8_16..

J.C. Yang, , , K. Yu, , , Y.H. Gong, , , 等. . Linear spatial pyramid matching using sparse coding for image classification. . Proc. IEEE Conf. on Computer Vision and Pattern Recognition, , 2009. . p.1794--1801. . DOI:10.1109/CVPR.2009.5206757http://doi.org/10.1109/CVPR.2009.5206757..

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据