Unsupervised feature selection via joint local learning and group sparse regression

Yue WU; Can WANG; Yue-qing ZHANG; Jia-jun BU

doi:10.1631/FITEE.1700804

Your Location：

Home >

Browse articles >

Unsupervised feature selection via joint local learning and group sparse regression

Regular Papers | Updated：2022-05-19

- Unsupervised feature selection via joint local learning and group sparse regression
- 联合局部学习和组稀疏回归的无监督特征选择
- Frontiers of Information Technology & Electronic Engineering Vol. 20, Issue 4, Pages: 538-553(2019)
- Affiliations：
  
  Zhejiang Provincial Key Laboratory of Service Robot, College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
  Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Hangzhou 310027, China
- Author bio：
  
  [ "Yue WU, E-mail: wy1988@zju.edu.cn" ]
  Can WANG, E-mail: wcan@zju.edu.cn
  [ "Yue-qing ZHANG, E-mail: 704787221@qq.com" ]
  [ "Jia-jun BU, E-mail: bjj@zju.edu.cn" ]
- Funds：
- DOI：10.1631/FITEE.1700804
  CLC： TP391.4
- Received：02 December 2017，
  
  Revised：2019-;4-11，
  
  Published：2019-04
- Accepted：
Scan QR Code
Yue WU, Can WANG, Yue-qing ZHANG, et al. Unsupervised feature selection via joint local learning and group sparse regression[J]. Frontiers of Information Technology & Electronic Engineering, 2019, 20(4): 538-553.
DOI：

Yue WU, Can WANG, Yue-qing ZHANG, et al. Unsupervised feature selection via joint local learning and group sparse regression[J]. Frontiers of Information Technology & Electronic Engineering, 2019, 20(4): 538-553. DOI： 10.1631/FITEE.1700804.

摘要

近十年，特征选择备受关注。通过挑选特征子集，可有效提升学习算法效率。由于难以获取标签信息，无监督特征选择算法相较于有监督特征选择算法应用更为广泛，其关键在于找出更能反映数据分布的特征集合。由于数据集中存在冗余和噪声，使用全部特征并不能很好展现数据的真实分布。为解决这一问题，本文提出联合局部学习和组稀疏回归的无监督特征选择算法。将基于局部学习聚类方法与组稀疏回归算法有机整合，选出有效反映数据流形分布同时保持组稀疏结构的特征。通过迭代算法，回归系数汇聚到重要特征上，选出能得到更优聚类效果的特征。对多个实际数据集（图像、声音和网页）的实验证明了该算法的有效性。

Abstract

Feature selection has attracted a great deal of interest over the past decades. By selecting meaningful feature subsets

the performance of learning algorithms can be effectively improved. Because label information is expensive to obtain

unsupervised feature selection methods are more widely used than the supervised ones. The key to unsupervised feature selection is to find features that effectively reflect the underlying data distribution. However

due to the inevitable redundancies and noise in a dataset

the intrinsic data distribution is not best revealed when using all features. To address this issue

we propose a novel unsupervised feature selection algorithm via joint local learning and group sparse regression (JLLGSR). JLLGSR incorporates local learning based clustering with group sparsity regularized regression in a single formulation

and seeks features that respect both the manifold structure and group sparse structure in the data space. An iterative optimization method is developed in which the weights finally converge on the important features and the selected features are able to improve the clustering results. Experiments on multiple real-world datasets (images

voices

and web pages) demonstrate the effectiveness of JLLGSR.

关键词

Keywords

references

M Belkin , , , P Niyogi . . Laplacian eigenmaps and spectral techniques for embedding and clustering . . 14 th Int Conf on Neural Information Processing Systems: Natural and Synthetic , , 2001 . . p.585 - - 591 . . . .

RE Bellman . . Adaptive Control Processes: a Guided Tour , , : : Princeton, NJ Princeton University Press , , . .

D Cai , , , C Zhang , , , X He . . Unsupervised feature selection for multi-cluster data . . 16 th Int Conf on Knowledge Discovery and Data Mining , , 2010 . . p.333 - - 342 . . DOI: 10.1145/1835804.1835848 http://doi.org/10.1145/1835804.1835848 . .

XJ Chang , , , FP Nie , , , Y Yang , , , 等 . . Convex sparse PCA for unsupervised feature learning . . ACM Trans Knowl Dis Data , , 2016 . . 11 ( ( 1 ): ): 3 DOI: 10.1145/2910585 http://doi.org/10.1145/2910585 . .

Y Cheung , , , H Zeng . . Local kernel regression score for selecting features of high-dimensional data . . IEEE Trans Knowl Data Eng , , 2009 . . 21 ( ( 12 ): ): 1798 - - 1802 . . DOI: 10.1109/TKDE.2009.23 http://doi.org/10.1109/TKDE.2009.23 . .

G Doquire , , , M Verleysen . . Mutual information-based feature selection for multilabel classification . . Neurocomputing , , 2013 . . 122 148 - - 155 . . DOI: 10.1016/j.neucom.2013.06.035 http://doi.org/10.1016/j.neucom.2013.06.035 . .

L Du , , , YD Shen . . Unsupervised feature selection with adaptive structure learning . . 21 st Int Conf on Knowledge Discovery and Data Mining , , 2015 . . p.209 - - 218 . . DOI: 10.1145/2783258.2783345 http://doi.org/10.1145/2783258.2783345 . .

M Fanty , , , R Cole . . Spoken letter recognition . . Conf on Advances in Neural Information Processing Systems , , 1990 . . p.220 - - 226 . . DOI: 10.3115/116580.116725 http://doi.org/10.3115/116580.116725 . .

AS Georghiades , , , PN Belhumeur , , , DJ Kriegman . . From few to many: illumination cone models for face recognition under variable lighting and pose . . IEEE Trans Patt Anal Mach Intell , , 2001 . . 23 ( ( 6 ): ): 643 - - 660 . . DOI: 10.1109/34.927464 http://doi.org/10.1109/34.927464 . .

I Guyon , , , A Elisseeff . . An introduction to variable and feature selection . . J Mach Learn Res , , 2003 . . 3 1157 - - 1182 . . DOI: 10.1162/153244303322753616 http://doi.org/10.1162/153244303322753616 . .

I Guyon , , , J Weston , , , S Barnhill , , , 等 . . Gene selection for cancer classification using support vector machines . . Mach Learn , , 2002 . . 46 ( ( 1-3 ): ): 389 - - 422 . . DOI: 10.1023/A:1012487302797 http://doi.org/10.1023/A:1012487302797 . .

YH Han , , , F Wu , , , Q Tian , , , 等 . . Image annotation by input-output structural grouping sparsity . . IEEE Trans Image Proc , , 2012 . . 21 ( ( 6 ): ): 3066 - - 3079 . . DOI: 10.1109/TIP.2012.2183880 http://doi.org/10.1109/TIP.2012.2183880 . .

YH Han , , , Y Yang , , , Y Yan , , , 等 . . Semisupervised feature selection via spline regression for video semantic recognition . . IEEE Trans Neur Netw Learn Syst , , 2015 . . 26 ( ( 2 ): ): 252 - - 264 . . DOI: 10.1109/TNNLS.2014.2314123 http://doi.org/10.1109/TNNLS.2014.2314123 . .

X He , , , P Niyogi . . Locality preserving projections . . Conf on Advances in Neural Information Processing Systems , , 2004 . . p.153 - - 160 . . . .

X He , , , D Cai , , , P Niyogi . . Laplacian score for feature selection . . Conf on Advances in Neural Information Processing Systems , , 2005 . . p.507 - - 514 . . . .

CP Hou , , , FP Nie , , , XL Li , , , 等 . . Joint embedding learning and sparse regression: a framework for unsupervised feature selection . . IEEE Trans Cybern , , 2014 . . 44 ( ( 6 ): ): 793 - - 804 . . DOI: 10.1109/TCYB.2013.2272642 http://doi.org/10.1109/TCYB.2013.2272642 . .

JJ Hull . . A database for handwritten text recognition research . . IEEE Trans Patt Anal Mach Intell , , 1994 . . 16 ( ( 5 ): ): 550 - - 554 . . DOI: 10.1109/34.291440 http://doi.org/10.1109/34.291440 . .

Y Jiang , , , JT Ren . . Eigenvalue sensitive feature selection . . 28 th Int Conf on Machine Learning , , 2011 . . p.89 - - 96 . . . .

IT Jolliffe . . Principal Component Analysis (2 nd Ed.) . . Springer, New York , , 2002 . . .

A Krizhevsky . . Learning Multiple Layers of Features from Tiny Images. . . Science Department, University of Toronto, Tech, Toronto , , 2009 . . .

HW Kuhn . . The Hungarian method for the assignment problem . . Nav Res Log Q , , 1955 . . 2 ( ( 1-2 ): ): 83 - - 97 . . DOI: 10.1002/nav.3800020109 http://doi.org/10.1002/nav.3800020109 . .

KC Lee , , , J Ho , , , DJ Kriegman . . Acquiring linear subspaces for face recognition under variable lighting . . IEEE Trans Patt Anal Mach Intell , , 2005 . . 27 ( ( 5 ): ): 684 - - 698 . . DOI: 10.1109/TPAMI.2005.92 http://doi.org/10.1109/TPAMI.2005.92 . .

MN Luo , , , FP Nie , , , XJ Chang , , , 等 . . Adaptive unsupervised feature selection with structure regularization . . IEEE Trans Neur Netw Learn Syst , , 2018 . . 29 ( ( 4 ): ): 944 - - 956 . . DOI: 10.1109/TNNLS.2017.2650978 http://doi.org/10.1109/TNNLS.2017.2650978 . .

J Munkres . . Algorithms for the assignment and transportation problems . . J Soc Ind Appl Math , , 1957 . . 5 ( ( 1 ): ): 32 - - 38 . . DOI: 10.1137/0105003 http://doi.org/10.1137/0105003 . .

FP Nie , , , SM Xiang , , , YQ Jia , , , 等 . . Trace ratio criterion for feature selection . . 23 rd Int Conf on Artificial Intelligence , , 2008 . . p.671 - - 676 . . . .

FP Nie , , , SM Xiang , , , YQ Song , , , 等 . . Orthogonal locality minimizing globality maximizing projections for feature extraction . . Opt Eng , , 2009 . . 48 ( ( 1 ): ): 017202 DOI: 10.1117/1.3067869 http://doi.org/10.1117/1.3067869 . .

FP Nie , , , H Huang , , , X Cai , , , 等 . . Efficient and robust feature selection via joint l 2, 1 -norms minimization . . 23 rd Int Conf on Neural Information Processing Systems , , 2010a . . p.1813 - - 1821 . . . .

FP Nie , , , D Xu , , , IWH Tsang , , , 等 . . Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction . . IEEE Trans Image Proc , , 2010b . . 19 ( ( 7 ): ): 1921 - - 1932 . . DOI: 10.1109/TIP.2010.2044958 http://doi.org/10.1109/TIP.2010.2044958 . .

FP Nie , , , ZN Zeng , , , IW Tsang , , , 等 . . Spectral embedded clustering: a framework for in-sample and out-ofsample spectral clustering . . IEEE Trans Neur Netw , , 2011 . . 22 ( ( 11 ): ): 1796 - - 1808 . . DOI: 10.1109/TNN.2011.2162000 http://doi.org/10.1109/TNN.2011.2162000 . .

FP Nie , , , XQ Wang , , , MI Jordan , , , 等 . . The constrained Laplacian rank algorithm for graph-based clustering . . 30 th AAAI Conf on Artificial Intelligence , , 2016a . . p.1969 - - 1976 . . . .

FP Nie , , , W Zhu , , , XI Li . . Unsupervised feature selection with structured graph optimization . . 30 th AAAI Conf on Artificial Intelligence , , 2016b . . p.1302 - - 1308 . . . .

HC Peng , , , FH Long , , , C Ding . . Feature selection based on mutual information criteria of max-dependency, maxrelevance, and min-redundancy . . IEEE Trans Patt Anal Mach Intell , , 2005 . . 27 ( ( 8 ): ): 1226 - - 1238 . . DOI: 10.1109/TPAMI.2005.159 http://doi.org/10.1109/TPAMI.2005.159 . .

ST Roweis , , , LK Saul . . Nonlinear dimensionality reduction by locally linear embedding . . Science , , 2000 . . 290 ( ( 5500 ): ): 2323 - - 2326 . . DOI: 10.1126/science.290.5500.2323 http://doi.org/10.1126/science.290.5500.2323 . .

YJ Sun , , , S Todorovic , , , S Goodison . . Local-learningbased feature selection for high-dimensional data analysis . . IEEE Trans Patt Anal Mach Intell , , 2010 . . 32 ( ( 9 ): ): 1610 - - 1626 . . DOI: 10.1109/TPAMI.2009.190 http://doi.org/10.1109/TPAMI.2009.190 . .

MK Tan , , , L Wang , , , IW Tsang . . Learning sparse SVM for feature selection on very high dimensional datasets . . 27 th Int Conf on Machine Learning , , 2010 . . p.1047 - - 1054 . . . .

JB Tenenbaum , , , V de Silva , , , JC Langford . . A global geometric framework for nonlinear dimensionality reduction . . Science , , 2000 . . 290 ( ( 5500 ): ): 2319 - - 2323 . . DOI: 10.1126/science.290.5500.2319 http://doi.org/10.1126/science.290.5500.2319 . .

R Tibshirani . . Regression shrinkage and selection via the Lasso . . J R Stat Soc B , , 1996 . . 58 ( ( 1 ): ): 267 - - 288 . . . .

M Verleysen . . Learning high-dimensional data.In: Ablameyko S, Goras L, Gori M (Eds.), Limitations and Future Trends in Neural Computation . . IOS Press, Amsterdam , , 2003 . . p.141 - - 162 . . . .

D Wang , , , FP Nie , , , H Huang . . Unsupervised feature selection via unified trace ratio formulation and Kmeans clustering (TRACK) . . European Conf on Machine Learning and Knowledge Discovery in Databases , , 2014 . . p.306 - - 321 . . DOI: 10.1007/978-3-662-44845-8_20 http://doi.org/10.1007/978-3-662-44845-8_20 . .

Y Wu , , , C Wang , , , JJ Bu , , , 等 . . Group sparse feature selection on local learning based clustering . . Neurocomputing , , 2016 . . 171 1118 - - 1130 . . DOI: 10.1016/j.neucom.2015.07.045 http://doi.org/10.1016/j.neucom.2015.07.045 . .

Y Yang , , , HT Shen , , , ZG Ma , , , 等 . . l2, 1-norm regularized discriminative feature selection for unsupervised learning . . 22 nd Int Joint Conf on Artificial Intelligence , , 2011 . . p.1589 - - 1594 . . DOI: 10.5591/978-1-57735-516-8/ijcai11-267 http://doi.org/10.5591/978-1-57735-516-8/ijcai11-267 . .

H Zeng , , , YM Cheung . . Feature selection for local learning based clustering . . 13 th Pacific-Asia Conf on Advances in Knowledge Discovery and Data Mining , , 2009 . . p.414 - - 425 . . DOI: 10.1007/978-3-642-01307-2_38 http://doi.org/10.1007/978-3-642-01307-2_38 . .

H Zeng , , , YM Cheung . . Feature selection and kernel learning for local learning-based clustering . . IEEE Trans Patt Anal Mach Intell , , 2011 . . 33 ( ( 8 ): ): 1532 - - 1547 . . DOI: 10.1109/TPAMI.2010.215 http://doi.org/10.1109/TPAMI.2010.215 . .

Z Zhao , , , H Liu . . Spectral feature selection for supervised and unsupervised learning . . 24 th Int Conf on Machine Learning , , 2007 . . p.1151 - - 1157 . . DOI: 10.1145/1273496.1273641 http://doi.org/10.1145/1273496.1273641 . .

H Zou , , , T Hastie . . Regularization and variable selection via the elastic net . . J R Stat Soc Ser B , , 2005 . . 67 ( ( 2 ): ): 301 - - 320 . . DOI: 10.1111/j.1467-9868.2005.00503.x http://doi.org/10.1111/j.1467-9868.2005.00503.x . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Feature selection techniques for microarray datasets: a comprehensive review, taxonomy, and future directions

A software defect prediction method with metric compensation based on feature selection and transfer learning

FAAD: an unsupervised fast and accurate anomaly detection method for a multi-dimensional sequence over data stream

A new feature selection method for handling redundant information in text classification

Battle damage assessment based on an improved Kullback-Leibler divergence sparse autoencoder

Related Author

Ramasamy DHANALAKSHMI

Kulanthaivel BALAKRISHNAN

Jinfu CHEN

Haibo CHEN

Jingyi CHEN

Jiaping XU

Saihua CAI

Xiaoli WANG

Related Institution

Department of Computer Science and Engineering, Indian Institute of Information Technology

School of Computer Science and Communication Engineering, Jiangsu University

Jiangsu Key Laboratory of Security Technology for Industrial Cyberspace, Jiangsu University

Science and Technology on Parallel and Distributed Processing Laboratory, College of Computer, National University of Defense Technology

Block Chain Research Institute of LianLian Pay

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰