Preservation of local linearity by neighborhood subspace scaling for solving the pre-image problem

Sheng-kai YANG; Jian-yi MENG; Hai-bin SHEN

doi:10.1631/jzus.C1300248

Your Location：

Home >

Browse articles >

Preservation of local linearity by neighborhood subspace scaling for solving the pre-image problem

Computer & Automation | Updated：2022-05-19

- Preservation of local linearity by neighborhood subspace scaling for solving the pre-image problem
- 基于邻域子空间缩放维持局部线性的原像算法
- Frontiers of Information Technology & Electronic Engineering Vol. 15, Issue 4, Pages: 254-264(2014)
- Affiliations：
  
  Institute of VLSI Design, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  [ "Sheng-kai YANG, E-mail: skyangzju@gmail.com" ]
  [ "Jian-yi MENG, E-mail: mengjy@vlsi.zju.edu.cn" ]
  Hai-bin SHEN, E-mail: shb@vlsi.zju.edu.cn
- Funds：
- DOI：10.1631/jzus.C1300248
  CLC： TP391
- Received：06 September 2013，
  
  Revised：2014-;3-17，
  
  Published：2014-04
- Accepted：
Scan QR Code
Sheng-kai YANG, Jian-yi MENG, Hai-bin SHEN. Preservation of local linearity by neighborhood subspace scaling for solving the pre-image problem[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(4): 254-264.
DOI：

Sheng-kai YANG, Jian-yi MENG, Hai-bin SHEN. Preservation of local linearity by neighborhood subspace scaling for solving the pre-image problem[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(4): 254-264. DOI： 10.1631/jzus.C1300248.

摘要

研究目的

原像问题广泛存在于各种核方法中，是一个悬而未决的重要问题。由于该问题是不适定的，其解往往不唯一甚至不存在。本文对目前主流的原像求解算法进行了分析和归纳，并探讨求解原像的更有效方法。

创新要点

基于局部线性保持的基本思想，结合邻域子空间的缩放变换，提出了一种新的逆映射构造方法。

研究方法

该算法包含两个步骤：（1）通过近邻子空间缩放变换以及局部线性保持，建立训练样本的局部逆映射过程；（2）基于该局部逆映射，利用光滑性假设得到测试样本的局部逆映射，进而求解得到原像。

重要结论

实验结果表明，本文提出的算法在图像降噪性能上优于目前的主流原像算法，同时具有较小的时间复杂度。

Abstract

An important issue involved in kernel methods is the pre-image problem. However

it is an ill-posed problem

as the solution is usually nonexistent or not unique. In contrast to direct methods aimed at minimizing the distance in feature space

indirect methods aimed at constructing approximate equivalent models have shown outstanding performance. In this paper

an indirect method for solving the pre-image problem is proposed. In the proposed algorithm

an inverse mapping process is constructed based on a novel framework that preserves local linearity. In this framework

a local nonlinear transformation is implicitly conducted by neighborhood subspace scaling transformation to preserve the local linearity between feature space and input space. By extending the inverse mapping process to test samples

we can obtain pre-images in input space. The proposed method is non-iterative

and can be used for any kernel functions. Experimental results based on image denoising using kernel principal component analysis (PCA) show that the proposed method outperforms the state-of-the-art methods for solving the pre-image problem.

关键词

Keywords

references

T.J. Abrahamsen , , , L.K. Hansen . . Regularized preimage estimation for kernel PCA de-noising . . J. Signal Process. Syst. , , 2011 . . 65 ( ( 3 ): ): 403 - - 412 . . DOI: 10.1007/s11265-010-0515-4 http://doi.org/10.1007/s11265-010-0515-4 . .

O. Arif , , , P.A. Vela , , , W. Daley . . Pre-image problem in manifold learning and dimensional reduction methods . . Proc. 9th Int. Conf. on Machine Learning and Applications , , 2010 . . p.921 - - 924 . . DOI: 10.1109/icmla.2010.146 http://doi.org/10.1109/icmla.2010.146 . .

G.H. Bakir , , , J. Weston , , , B. Schölkopf . . Learning to find pre-images . . Adv. Neur. Inform. Process. Syst. , , 2003 . . 16 ( ( 7 ): ): 449 - - 456 . . . .

A.S. Georghiades , , , P.N. Belhumeur , , , D.J. Kriegman . . From few to many: illumination cone models for face recognition under variable lighting and pose . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2001 . . 23 ( ( 6 ): ): 643 - - 660 . . DOI: 10.1109/34.927464 http://doi.org/10.1109/34.927464 . .

P. Gruber , , , K. Stadlthanner , , , M. Böhm , , , 等 . . Denoising using local projective subspace methods . . Neurocomputing , , 2006 . . 69 ( ( 13-15 ): ): 1485 - - 1501 . . DOI: 10.1016/j.neucom.2005.12.025 http://doi.org/10.1016/j.neucom.2005.12.025 . .

P. Honeine , , , C. Richard . . A closed-form solution for the pre-image problem in kernel-based machines . . J. Signal Process. Syst. , , 2011a . . 65 ( ( 3 ): ): 289 - - 299 . . DOI: 10.1007/s11265-010-0482-9 http://doi.org/10.1007/s11265-010-0482-9 . .

P. Honeine , , , C. Richard . . Preimage problem in kernelbased machine learning . . IEEE Signal Process. Mag. , , 2011b . . 28 ( ( 2 ): ): 77 - - 88 . . DOI: 10.1109/msp.2010.939747 http://doi.org/10.1109/msp.2010.939747 . .

D. Huang , , , Y.D. Tian , , , F. de la Torre . . Local isomorphism to solve the pre-image problem in kernel methods . . Proc. IEEE Conf. on Computer Vision and Pattern Recognition , , 2011 . . p.2761 - - 2768 . . DOI: 10.1109/cvpr.2011.5995685 http://doi.org/10.1109/cvpr.2011.5995685 . .

A. J. Izenman , , , 等 . . Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning , , : : New York, USA Springer , , 2008 . . .

R. Jenssen . . Kernel entropy component analysis . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2010 . . 32 ( ( 5 ): ): 847 - - 860 . . DOI: 10.1109/tpami.2009.100 http://doi.org/10.1109/tpami.2009.100 . .

M. Kallas , , , P. Honeine , , , C. Richard , , , 等 . . Nonnegativity constraints on the pre-image for pattern recognition with kernel machines . . Pattern Recog. , , 2013 . . 46 ( ( 11 ): ): 3066 - - 3080 . . DOI: 10.1016/j.patcog.2013.03.021 http://doi.org/10.1016/j.patcog.2013.03.021 . .

K.I. Kim , , , M.O. Franz , , , B. Schölkopf . . Iterative kernel principal component analysis for image modeling . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2005 . . 27 ( ( 9 ): ): 1351 - - 1366 . . DOI: 10.1109/TPAMI.2005.181 http://doi.org/10.1109/TPAMI.2005.181 . .

J.T.Y. Kwok , , , I.W. Tsang . . The pre-image problem in kernel methods . . IEEE Trans. Neur. Networks , , 2004 . . 15 ( ( 6 ): ): 1517 - - 1525 . . DOI: 10.1109/tnn.2004.837781 http://doi.org/10.1109/tnn.2004.837781 . .

K.C. Lee , , , J. Ho , , , D. Kriegman . . Acquiring linear subspaces for face recognition under variable lighting . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2005 . . 27 ( ( 5 ): ): 684 - - 698 . . DOI: 10.1109/TPAMI.2005.92 http://doi.org/10.1109/TPAMI.2005.92 . .

J.W. Li , , , L. Su . . Combining KPCA and PSO for pattern denoising . . Proc. Chinese Conf. on Pattern Recognition , , 2008 . . p.1 - - 6 . . DOI: 10.1109/ccpr.2008.10 http://doi.org/10.1109/ccpr.2008.10 . .

J.W. Li , , , L. Su , , , C. Cheng . . Finding pre-images via evolution strategies . . Appl. Soft Comput. , , 2011 . . 11 ( ( 6 ): ): 4183 - - 4194 . . DOI: 10.1016/j.asoc.2011.03.011 http://doi.org/10.1016/j.asoc.2011.03.011 . .

S. Mika , , , B. Schölkopf , , , A. Smola , , , 等 . . Kernel PCA and de-noising in feature spaces . . Adv. Neur. Inform. Process. Syst. , , 1998 . . 11 536 - - 542 . . . .

M.H. Nguyen , , , F. de la Torre . . Robust kernel principal component analysis . . Adv. Neur. Inform. Process. Syst. , , 2008 . . 21 1185 - - 1192 . . . .

J. Park , , , D. Kang , , , J. Kim , , , 等 . . SVDD-based pattern denoising . . Neur. Comput. , , 2007 . . 19 ( ( 7 ): ): 1919 - - 1938 . . DOI: 10.1162/neco.2007.19.7.1919 http://doi.org/10.1162/neco.2007.19.7.1919 . .

Y. Rathi , , , S. Dambreville , , , A. Tannenbaum . . Statistical shape analysis using kernel PCA . . SPIE , , 2006 . . 6064 60641B DOI: 10.1117/12.641417 http://doi.org/10.1117/12.641417 . .

L.K. Saul , , , S.T. Roweis . . Think globally, fit locally: unsupervised learning of low dimensional manifolds . . J. Mach. Learn. Res. , , 2003 . . 4 119 - - 155 . . . .

B. Schölkopf , , , A. Smola , , , K.R. Mȹller . . Kernel principal component analysis . . Proc. 7th Int. Conf. on Artificial Neural Networks , , 1997 . . p.583 - - 588 . . DOI: 10.1007/bfb0020217 http://doi.org/10.1007/bfb0020217 . .

B. Schölkopf , , , R. Herbrich , , , A.J. Smola . . A generalized representer theorem . . Proc. 14th Annual Conf. on Computational Learning Theory and 5th European Conf. on Computational Learning Theory , , 2001 . . p.416 - - 426 . . DOI: 10.1007/3-540-44581-1_27 http://doi.org/10.1007/3-540-44581-1_27 . .

A.R. Teixeira , , , A.M. Tomé , , , K. Stadlthanner , , , 等 . . KPCA denoising and the pre-image problem revisited . . Digit. Signal Process. , , 2008 . . 18 ( ( 4 ): ): 568 - - 580 . . DOI: 10.1016/j.dsp.2007.08.001 http://doi.org/10.1016/j.dsp.2007.08.001 . .

W.S. Zheng , , , J.H. Lai . . Regularized locality preserving learning of pre-image problem in kernel principal component analysis . . Proc. 18th Int. Conf. on Pattern Recognition , , 2006 . . p.456 - - 459 . . DOI: 10.1109/icpr.2006.991 http://doi.org/10.1109/icpr.2006.991 . .

W.S. Zheng , , , J.H. Lai , , , P.C. Yuen . . Penalized preimage learning in kernel principal component analysis . . IEEE Trans. Neur. Networks , , 2010 . . 21 ( ( 4 ): ): 551 - - 570 . . DOI: 10.1109/tnn.2009.2039647 http://doi.org/10.1109/tnn.2009.2039647 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Kernel sparse representation for MRI image analysis in automatic brain tumor segmentation

Fast global kernel fuzzy c-means clustering algorithm for consonant/vowel segmentation of speech signal

Related Author

Ji-jun TONG

Peng ZHANG

Yu-xiang WENG

Dan-hua ZHU

Xian ZANG

Felipe P. VISTA IV

Kil To CHONG

Related Institution

School of Information Science and Technology, Zhejiang Sci-Tech University

Department of Neurosurgery, The First Affiliated Hospital, School of Medicine, Zhejiang University

State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, The First Affiliated Hospital, School of Medicine, Zhejiang University

Department of Electronic Engineering, Jeonbuk National University,-si

Advanced Electronics and Information Research Center, Jeonbuk National University,-si

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰