View-invariant human action recognition via robust locally adaptive multi-view learning*

Feng Jia-geng; Xiao Jun

doi:10.1631/FITEE.1500080

Your Location：

Home >

Browse articles >

View-invariant human action recognition via robust locally adaptive multi-view learning*

Computer & Automation | Updated：2022-06-06

- View-invariant human action recognition via robust locally adaptive multi-view learning*
- 基于鲁棒局部自适应多视角学习的视点无关人体行为识别
- Frontiers of Information Technology & Electronic Engineering Vol. 16, Issue 11, Pages: 917-929(2015)
- Affiliations：
  
  Institute of Artificial Intelligence, College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  †E-mail:fengjiageng@126.com
- Funds：
- DOI：10.1631/FITEE.1500080
  CLC：
- Received：18 March 2015，
  
  Revised：2015-10-12，
  
  Accepted：14 September 2015，
  
  Published：2015-11
- Accepted：
Scan QR Code
Feng Jia-geng, Xiao Jun. View-invariant human action recognition via robust locally adaptive multi-view learning*[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(11): 917-929.
DOI：

Feng Jia-geng, Xiao Jun. View-invariant human action recognition via robust locally adaptive multi-view learning*[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(11): 917-929. DOI： 10.1631/FITEE.1500080.

摘要

目的

基于视觉的人体行为识别是一个非常活跃的研究领域。它在智能监控、感知接口和基于内容的视频检索等领域具有广泛的应用前景。然而，一些现实应用场景仍然阻碍行为识别技术的发展，比如现实场景中的动作往往是从任意角度拍摄的。因此与视点无关的行为识别显得十分重要。大量研究者开始致力于行为识别的视点无关性。本文提出一种基于多视角学习的视点无关人体行为识别方法。

创新点

针对现有多视角学习算法在构建近邻图时缺乏数据自适应性的问题，本文提出一种自适应多视角学习算法。此外，还提出一种迭代优化求解方法对所构建的目标函数进行优化求解。

方法

对于单个视角下的所有样本特征数据，构建一个该视角下的L1图。在获得数据的稀疏图结构后，对于单视角下的数据，希望学习一种最优的降维方法，在对原始数据进行降维的同时，最大程度地保持数据内在的局部结构信息；对于不同的视角，取一个非负的权重向量来衡量不同视角的重要程度。对于全部的视角可以统一起来得到目标函数。最后利用迭代优化求解，用支持向量机（SVM）分类。

结论

将本文所提算法应用到视点无关的行为识别中，实验结果表明：该算法能够自适应地选择近邻数与不同特征的权重；与其他几种对比算法相比，本文所提算法的分类准确率更高。

Abstract

Human action recognition is currently one of the most active research areas in computer vision. It has been widely used in many applications

such as intelligent surveillance

perceptual interface

and content-based video retrieval. However

some extrinsic factors are barriers for the development of action recognition; e.g.

human actions may be observed from arbitrary camera viewpoints in realistic scene. Thus

view-invariant analysis becomes important for action recognition algorithms

and a number of researchers have paid much attention to this issue. In this paper

we present a multi-view learning approach to recognize human actions from different views. As most existing multi-view learning algorithms often suffer from the problem of lacking data adaptiveness in the nearest neighborhood graph construction procedure

a robust locally adaptive multi-view learning algorithm based on learning multiple local L1-graphs is proposed. Moreover

an efficient iterative optimization method is proposed to solve the proposed objective function. Experiments on three public view-invariant action recognition datasets

i.e.

ViHASi

IXMAS

and WVU

demonstrate data adaptiveness

effectiveness

and efficiency of our algorithm. More importantly

when the feature dimension is correctly selected (i.e.

>60)

the proposed algorithm stably outperforms state-of-the-art counterparts and obtains about 6% improvement in recognition accuracy on the three datasets.

关键词

Keywords

references

AB Ashraf , S Lucey , T Chen . Learning patch correspondences for improved viewpoint invariant face recognition . 2008 . Proc. IEEE Conf. on Computer Vision and Pattern Recognition . 1 - 8 . DOI: 10.1109/CVPR.2008.4587754 http://doi.org/10.1109/CVPR.2008.4587754 .

S Balakrishnama , A Ganapathiraju . Linear Discriminant Analysis—a Brief Tutorial . 1998 . Institute for Signal and Information Processing, Mississippi State University, USA . .

M Balasubramanian , EL Schwartz . The isomap algorithm and topological stability . Science , 2002 . 295 ( 5552 ): 7 DOI: 10.1126/science.295.5552.9r http://doi.org/10.1126/science.295.5552.9r .

A Blum , T Mitchell . Combining labeled and unlabeled data with co-training . 1998 . Proc. 11th Annual Conf. on Computational Learning Theory . 92 - 100 . DOI: 10.1145/279943.279962 http://doi.org/10.1145/279943.279962 .

AF Bobick , JW Davis . The recognition of human movement using temporal templates . IEEE Trans Patt Anal Mach Intell , 2001 . 23 ( 3 ): 257 - 267 . DOI: 10.1109/34.910878 http://doi.org/10.1109/34.910878 .

F Brémond , M Thonnat , M Zℝñiga . Video-understanding framework for automatic behavior recognition . Behav Res Methods , 2006 . 38 ( 3 ): 416 - 426 . DOI: 10.3758/BF03192795 http://doi.org/10.3758/BF03192795 .

E Candès , J Romberg . l 1 -Magic: Recovery of Sparse Signals via Convex Programming . 2005 .

C Chen , YT Zhuang , J Xiao . Silhouette representation and matching for 3D pose discrimination—a comparative study . Image Vis Comput , 2010 . 28 ( 4 ): 654 - 667 . DOI: 10.1016/j.imavis.2009.10.008 http://doi.org/10.1016/j.imavis.2009.10.008 .

HS Chen , HT Chen , Y Chen , 等 . Human action recognition using star skeleton . 2006 . Proc. 4th ACM Int. Workshop on Video Surveillance and Sensor Networks . 171 - 178 . DOI: 10.1145/1178782.1178808 http://doi.org/10.1145/1178782.1178808 .

B Cheng , J Yang , S Yan , 等 . Learning with l 1 -graph for image analysis . IEEE Trans Image Process , 2010 . 19 ( 4 ): 858 - 866 . DOI: 10.1109/TIP.2009.2038764 http://doi.org/10.1109/TIP.2009.2038764 .

R de Sa Virginia . Spectral clustering with two views . 2005 . Proc. 22nd Annual Int. Conf. on Machine Learning . 20 - 27 . .

DL Donoho . For most large underdetermined systems of linear equations the minimal l 1 -norm solution is also the sparsest solution . Commun Pure Appl Math , 2006 . 59 ( 6 ): 797 - 829 . DOI: 10.1002/cpa.20132 http://doi.org/10.1002/cpa.20132 .

DL Donoho , M Elad , VN Temlyakov . Stable recovery of sparse overcomplete representations in the presence of noise . IEEE Trans Inform Theory , 2006 . 52 ( 1 ): 6 - 18 . DOI: 10.1109/TIT.2005.860430 http://doi.org/10.1109/TIT.2005.860430 .

JG Feng , J Xiao . View-invariant action recognition: a survey . J Image Graph , 2013 . 18 ( 2 ): 157 - 168 . (in Chinese) DOI: 10.11834/jig.20130205 http://doi.org/10.11834/jig.20130205 .

Y Fu , YM Xian . Image classification based on multi-feature and improved SVM ensemble . Comput Eng , 2001 . 37 ( 21 ): 196 - 198 . DOI: 10.3969/j.issn.1000-3428.2011.21.067 http://doi.org/10.3969/j.issn.1000-3428.2011.21.067 .

XF He , D Cai , S Yan , 等 . Neighborhood preserving embedding . 2005 . Proc. 10th IEEE Int. Conf. on Computer Vision . 1208 - 1213 . DOI: 10.1109/ICCV.2005.167 http://doi.org/10.1109/ICCV.2005.167 .

F Jean , R Bergevin , AB Albu . Trajectories normalization for viewpoint invariant gait recognition . 2008 . Proc. 19th Int. Conf. on Pattern Recognition . 1 - 4 . DOI: 10.1109/ICPR.2008.4761312 http://doi.org/10.1109/ICPR.2008.4761312 .

IN Junejo , E Dexter , I Laptev , 等 . Cross-view action recognition from temporal self-similarities . 2008 . Proc. 10th European Conf. on Computer Vision . 293 - 306 . DOI: 10.1007/978-3-540-88688-4_22 http://doi.org/10.1007/978-3-540-88688-4_22 .

DD Lee , HS Seung . Learning the parts of objects by non-negative matrix factorization . Nature , 1999 . 401 ( 6755 ): 788 - 791 . DOI: 10.1038/44565 http://doi.org/10.1038/44565 .

M Lewandowski , J Martinez-del-Rincon , D Makris , 等 . Temporal extension of Laplacian eigenmaps for unsupervised dimensionality reduction of time series . 2010 . Proc. 20th Int. Conf. on Pattern Recognition . 161 - 164 . DOI: 10.1109/ICPR.2010.48 http://doi.org/10.1109/ICPR.2010.48 .

B Long , PS Yu , ZF Zhang . A general model for multiple view unsupervised learning . SIAM , 2008 . 822 - 833 . .

Y Luo , T Wu , J Hwang . Object-based analysis and interpretation of human motion in sports video sequences by dynamic Bayesian networks . Comput Vis Image Understand , 2003 . 92 ( 2-3 ): 196 - 216 . DOI: 10.1016/j.cviu.2003.08.001 http://doi.org/10.1016/j.cviu.2003.08.001 .

JL Mao . Adaptive multi-view learning and its application to image classification . J Comput Appl , 2013 . 33 ( 7 ): 1955 - 1959 . (in Chinese) DOI: 10.11772/j.issn.1001-9081.2013.07.1955 http://doi.org/10.11772/j.issn.1001-9081.2013.07.1955 .

P Natarajan , R Nevatia . View and scale invariant action recognition using multiview shape-ﬂow models . 2008 . Proc. IEEE Conf. on Computer Vision and Pattern Recognition . 1 - 8 . DOI: 10.1109/CVPR.2008.4587716 http://doi.org/10.1109/CVPR.2008.4587716 .

P Natarajan , VK Singh , R Nevatia . Learning 3D action models from a few 2D videos for view invariant action recognition . 2010 . Proc. IEEE Conf. on Computer Vision and Pattern Recognition . 2006 - 2013 . DOI: 10.1109/CVPR.2010.5539876 http://doi.org/10.1109/CVPR.2010.5539876 .

V Parameswaran , R Chellappa . View invariance for human action recognition . Int J Comput Vis , 2006 . 66 ( 1 ): 83 - 101 . DOI: 10.1007/s11263-005-3671-4 http://doi.org/10.1007/s11263-005-3671-4 .

C Rao , A Yilmaz , M Shah . View-invariant representation and recognition of actions . Int J Comput Vis , 2002 . 50 ( 2 ): 203 - 226 . DOI: 10.1023/A:1020350100748 http://doi.org/10.1023/A:1020350100748 .

B Raytchev , Y Kikutsugi , T Tamaki , 等 . Class-speciﬁc low-dimensional representation of local features for viewpoint invariant object recognition . 2010 . Proc. 10th Asian Conf. on Computer Vision . 250 - 261 . DOI: 10.1007/978-3-642-19318-7_20 http://doi.org/10.1007/978-3-642-19318-7_20 .

M Roh , H Shin , S Lee . View-independent human action recognition with volume motion template on single stereo camera . Patt Recogn Lett , 2010 . 31 ( 7 ): 639 - 647 . DOI: 10.1016/j.patrec.2009.11.017 http://doi.org/10.1016/j.patrec.2009.11.017 .

ST Roweis , LK Saul . Nonlinear dimensionality reduction by locally linear embedding . Science , 2000 . 290 ( 5500 ): 2323 - 2326 . DOI: 10.1126/science.290.5500.2323 http://doi.org/10.1126/science.290.5500.2323 .

B Shen , L Si . Nonnegative matrix factorization clustering on multiple manifolds . 2010 . Proc. 24th AAAI Conf. on Artificial Intelligence . 575 - 580 . .

P Srestasathiern , A Yilmaz . View invariant object recognition . 2008 . Proc. 19th Int. Conf. on Pattern Recognition . 1 - 4 . DOI: 10.1109/ICPR.2008.4761238 http://doi.org/10.1109/ICPR.2008.4761238 .

T Syeda-Mahmood , A Vasilescu , S Sethi . Recognizing action events from multiple viewpoints . 2001 . Proc. IEEE Workshop on Detection and Recognition of Events in Video . 64 - 72 . DOI: 10.1109/EVENT.2001.938868 http://doi.org/10.1109/EVENT.2001.938868 .

YF Tang , ZM Huang , RJ Huang , 等 . Texture image classification based on multi-feature extraction and SVM classifier . Comput Appl Softw , 2011 . 28 ( 6 ): 22 - 46 . (in Chinese) DOI: 10.3969/j.issn.1000-386X.2011.06.006 http://doi.org/10.3969/j.issn.1000-386X.2011.06.006 .

C Tian , G Fan , X Gao . Multi-view face recognition by nonlinear tensor decomposition . 2008 . Proc. 19th Int. Conf. on Pattern Recognition . 1 - 4 . DOI: 10.1109/ICPR.2008.4761195 http://doi.org/10.1109/ICPR.2008.4761195 .

Y Wang , K Huang , T Tan . Multi-view gymnastic activity recognition with fused HMM . 2007 . Proc. 8th Asian Conf. on Computer Vision . 667 - 677 . DOI: 10.1007/978-3-540-76386-4_63 http://doi.org/10.1007/978-3-540-76386-4_63 .

D Weinland , R Ronfard , E Boyer . Free viewpoint action recognition using motion history volumes . Comput Vis Image Understand , 2006 . 104 ( 2-3 ): 249 - 257 . DOI: 10.1016/j.cviu.2006.07.013 http://doi.org/10.1016/j.cviu.2006.07.013 .

D Weinland , E Boyer , R Ronfard . Action recognition from arbitrary views using 3D exemplars . 2007 . Proc. IEEE 11th Int. Conf. on Computer Vision . 1 - 7 . DOI: 10.1109/ICCV.2007.4408849 http://doi.org/10.1109/ICCV.2007.4408849 .

JH Wen , Z Tian , W Lin , 等 . Feature extraction based on supervised locally linear embedding for classification of hyperspectral images . J Comput Appl , 2011 . 31 ( 3 ): 715 - 717 . DOI: 10.3724/SP.J.1087.2011.00715 http://doi.org/10.3724/SP.J.1087.2011.00715 .

S Wold , K Esbensen , P Geladi . Principal component analysis . Chemometr Intell Lab Syst , 1987 . 2 ( 1-3 ): 37 - 52 . DOI: 10.1016/0169-7439(87)80084-9 http://doi.org/10.1016/0169-7439(87)80084-9 .

J Wright , AY Yang , A Ganesh , 等 . Robust face recognition via sparse representation . IEEE Trans Patt Anal Mach Intell , 2009 . 31 ( 2 ): 210 - 227 . DOI: 10.1109/TPAMI.2008.79 http://doi.org/10.1109/TPAMI.2008.79 .

T Xia , DC Tao , T Mei , 等 . Multiview spectral embedding . IEEE Trans Syst Man Cybern , 2010 . 40 ( 6 ): 1438 - 1446 . DOI: 10.1109/TSMCB.2009.2039566 http://doi.org/10.1109/TSMCB.2009.2039566 .

P Yan , SM Khan , M Shah . Learning 4D action feature models for arbitrary view action recognition . 2008 . Proc. IEEE Conf. on Computer Vision and Pattern Recognition . 1 - 7 . DOI: 10.1109/CVPR.2008.4587737 http://doi.org/10.1109/CVPR.2008.4587737 .

J Yang , YG Jiang , AG Hauptmann , 等 . Evaluating bag-of-visual-words representations in scene classification . 2007 . Proc. Int. Workshop on Multimedia Information Retrieval . 197 - 206 . DOI: 10.1145/1290082.1290111 http://doi.org/10.1145/1290082.1290111 .

A Yilmaz , M Shah . Actions as objects: a novel action representation . 2005 . Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition . 984 - 989 . DOI: 10.1109/CVPR.2005.58 http://doi.org/10.1109/CVPR.2005.58 .

H Yu , G Sun , W Song , 等 . Human motion recognition based on neural network . 2005 . Proc. Int. Conf. on Communications, Circuits and Systems . 979 - 982 . DOI: 10.1109/ICCCAS.2005.1495271 http://doi.org/10.1109/ICCCAS.2005.1495271 .

SE Zheng , SZ Ye . Semi-supervision and active relevance feedback algorithm for content-based image retrieval . Comput Eng Appl , 2006 . S1 81 - 87 . (in Chinese) .

D Zhou , CJC Burges . Spectral clustering and transductive learning with multiple views . 2007 . Proc. 24th Int. Conf. on Machine Learning . 1159 - 1166 . DOI: 10.1145/1273496.1273642 http://doi.org/10.1145/1273496.1273642 .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Temporal fidelity enhancement for video action recognition

Learning robust principal components from L1-norm maximization

Related Author

Shaowu XU

Xibin JIA

Qianmei SUN

Jing CHANG

Ding-cheng FENG

Feng CHEN

Wen-li XU

Related Institution

Faculty of Information Technology, Beijing University of Technology

Beijing Chao-yang Hospital, Capital Medical University

Tsinghua National Laboratory for Information Science and Technology, Tsinghua University

Department of Automation, Tsinghua University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰