Interactive image segmentation with a regression based ensemble learning paradigm

Jin ZHANG; Zhao-hui TANG; Wei-hua GUI; Qing CHEN; Jin-ping LIU

doi:10.1631/FITEE.1601401

Your Location：

Home >

Browse articles >

Interactive image segmentation with a regression based ensemble learning paradigm

Regular Papers | Updated：2022-05-19

- Interactive image segmentation with a regression based ensemble learning paradigm
- 基于回归预测集成学习的交互式图像分割
- Frontiers of Information Technology & Electronic Engineering Vol. 18, Issue 7, Pages: 1002-1020(2017)
- Affiliations：
  
  School of Information Science and Engineering, Central South University, Changsha 410083, China
  College of Computer and Communication, Hunan University of Technology, Zhuzhou 412007, China
  College of Mathematics and Computer Science, Hunan Normal University, Changsha 410083, China
- Author bio：
  
  [ "Jin ZHANG, E-mail:zhang_jin@csu.edu.cn" ]
  Zhao-hui TANG, E-mail: zhtang@csu.edu.cn
- Funds：
- DOI：10.1631/FITEE.1601401
  CLC： TP391.4
- Received：18 July 2016，
  
  Revised：2017-;7-11，
  
  Published：2017-07
- Accepted：
Scan QR Code
Jin ZHANG, Zhao-hui TANG, Wei-hua GUI, et al. Interactive image segmentation with a regression based ensemble learning paradigm[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(7): 1002-1020.
DOI：

Jin ZHANG, Zhao-hui TANG, Wei-hua GUI, et al. Interactive image segmentation with a regression based ensemble learning paradigm[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(7): 1002-1020. DOI： 10.1631/FITEE.1601401.

摘要

对于复杂场景下的自然图像，全自动图像分割方法难以获得与真实情况吻合的结果，人们常常采用交互式分割手段实现精确分割。然而，当前及背景中存在颜色相似的区域时，传统半监督图像分割方法只能通过大量增加手工标记获得精确分割结果。为此，本文提出一种结合半监督学习的基于回归预测的集成学习交互式图像分割方法。通过集成两个互补的样条回归函数，将图像分割视为一个非线性预测问题。首先，基于已标记样本训练出两个在属性上互补的多元自适应回归样条学习器（multivariate adaptive regression splines

MARS）和薄板样条回归学习器（thin plate spline regression

TPSR）；接着，提出一种基于聚类假设和半监督学习的回归器增强算法，该算法从未标记样本中抽选部分样本辅助训练MARS和TPSR；然后，引入支持向量回归方法（support vector regression

SVR）集成MARS和TPSR的预测结果；最后，对SVR集成结果进行GraphCut图像分割。在标准数据库BSDS500和PascalVOC上进行大量实验，验证了所提算法的有效性。大量对比实验证实，所提算法在交互式自然图像分割上的表现与当前最先进算法相当。

Abstract

To achieve fine segmentation of complex natural images

people often resort to an interactive segmentation paradigm

since fully automatic methods often fail to obtain a result consistent with the ground truth. However

when the foreground and background share some similar areas in color

the fine segmentation result of conventional interactive methods usually relies on the increase of manual labels. This paper presents a novel interactive image segmentation method via a regression-based ensemble model with semi-supervised learning. The task is formulated as a non-linear problem integrating two complementary spline regressors and strengthening the robustness of each regressor via semi-supervised learning. First

two spline regressors with a complementary nature are constructed based on multivariate adaptive regression splines (MARS) and smooth thin plate spline regression (TPSR). Then

a regressor boosting method based on a clustering hypothesis and semi-supervised learning is proposed to assist the training of MARS and TPSR by using the region segmentation information contained in unlabeled pixels. Next

a support vector regression (SVR) based decision fusion model is adopted to integrate the results of MARS and TPSR. Finally

the GraphCut is introduced and combined with the SVR ensemble results to achieve image segmentation. Extensive experimental results on benchmark datasets of BSDS500 and Pascal VOC have demonstrated the effectiveness of our method

and the comparison with experiment results has validated that the proposed method is comparable with the state-of-the-art methods for interactive natural image segmentation.

关键词

Keywords

references

J. Adamowski , , , H.F. Chan , , , S.O. Prasher , , , 等 . . Comparison of multivariate adaptive regression splines with coupled wavelet transform artificial neural networks for runoff forecasting in Himalayan micro-watersheds with limited data . . J. Hydroinform. , , 2012 . . 14 ( ( 3 ): ): 731 - - 744 . . DOI: 10.2166/hydro.2011.044 http://doi.org/10.2166/hydro.2011.044 . .

M.F. Balcan , , , A. Blum , , , K. Yang . . Co-training and expansion: towards bridging theory and practice . . 17th Int. Conf. on Neural Information Processing Systems , , 2004 . . p.89 - - 96 . . . .

A. Blum , , , T. Mitchell . . Combining labeled and unlabeled data with co-training . . 11th Annual Conf. on Computational Learning Theory , , 1998 . . p.92 - - 100 . . DOI: 10.1145/279943.279962 http://doi.org/10.1145/279943.279962 . .

Y.Y. Boykov , , , M.P. Jolly . . Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images . . 8th IEEE Int. Conf. on Computer Vision , , 2001 . . p.105 - - 112 . . DOI: 10.1109/ICCV.2001.937505 http://doi.org/10.1109/ICCV.2001.937505 . .

Y.Y. Boykov , , , O. Veksler , , , R. Zabih . . Fast approximate energy minimization via graph cuts . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2001 . . 23 ( ( 11 ): ): 1222 - - 1239 . . DOI: 10.1109/ICCV.1999.791245 http://doi.org/10.1109/ICCV.1999.791245 . .

J.J. Ding , , , C.J. Lin , , , I.F. Lu , , , 等 . . Real-time interactive image segmentation using improved superpixels . . IEEE Int. Conf. on Digital Signal Processing , , 2015 . . p.740 - - 744 . . DOI: 10.1109/ICDSP.2015.7251974 http://doi.org/10.1109/ICDSP.2015.7251974 . .

M. Everingham , , , L. van Gool , , , C.K. Williams , , , 等 . . The Pascal Visual Object Classes (VOC) challenge . . Int. J. Comput. Vis. , , 2010 . . 88 ( ( 2 ): ): 303 - - 338 . . DOI: 10.1007/s11263-009-0275-4 http://doi.org/10.1007/s11263-009-0275-4 . .

J.H. Friedman . . Multivariate adaptive regression splines . . Ann. Statist. , , 1991 . . 19 ( ( 1 ): ): 1 - - 67 . . DOI: 10.1214/aos/1176347963 http://doi.org/10.1214/aos/1176347963 . .

Z. Fu , , , L. Wang , , , D. Zhang . . An improved multi-label classification ensemble learning algorithm . . In : Li S., Liu C., Wang, Y. (Eds.), Pattern Recognition. Springer Berlin Heidelberg , , 2014 . . p.243 - - 252 . . DOI: 10.1007/978-3-662-45646-0_25 http://doi.org/10.1007/978-3-662-45646-0_25 . .

M. Galar , , , A. Fernandez , , , E. Barrenechea , , , 等 . . A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches . . IEEE Trans. Syst. Man Cybern. C , , 2012 . . 42 ( ( 4 ): ): 463 - - 484 . . DOI: 10.1109/TSMCC.2011.2161285 http://doi.org/10.1109/TSMCC.2011.2161285 . .

L. Ge , , , R. Ju , , , T. Ren , , , 等 . . Interactive RGB-D image segmentation using hierarchical graph cut and geodesic distance . . In : Ho Y.S., Sang J., Ro Y.M., et al. (Eds.), Advances in Multimedia Information Processing. Springer International Publishing , , 2015 . . p.114 - - 124 . . DOI: 10.1007/978-3-319-24075-6_12 http://doi.org/10.1007/978-3-319-24075-6_12 . .

V. Gulshan , , , C. Rother , , , A. Criminisi , , , 等 . . Geodesic star convexity for interactive image segmentation . . IEEE Computer Society Conf. on Computer Vision and Pattern Recognition , , 2010 . . p.3129 - - 3136 . . DOI: 10.1109/CVPR.2010.5540073 http://doi.org/10.1109/CVPR.2010.5540073 . .

M. Jian , , , C. Jung . . Interactive image segmentation using adaptive constraint propagation . . IEEE Trans. Image Process. , , 2016 . . 25 ( ( 3 ): ): 1301 - - 1311 . . DOI: 10.1109/TIP.2016.2518480 http://doi.org/10.1109/TIP.2016.2518480 . .

A.M. Jobst , , , D.G. Kingston , , , N.J. Cullen , , , 等 . . Combining thin-plate spline interpolation with a lapse rate model to produce daily air temperature estimates in a data-sparse alpine catchment . . Int. J. Climatol. , , 2016 . . 37 ( ( 1 ): ): 214 - - 229 . . DOI: 10.1002/joc.4699 http://doi.org/10.1002/joc.4699 . .

C. Jung , , , M. Jian , , , J. Liu , , , 等 . . Interactive image segmentation via kernel propagation . . Patt. Recogn. , , 2014 . . 47 ( ( 8 ): ): 2745 - - 2755 . . DOI: 10.1016/j.patcog.2014.02.010 http://doi.org/10.1016/j.patcog.2014.02.010 . .

V. Kolmogorov , , , R. Zabih . . What energy functions can be minimized via graph cuts? . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2004 . . 26 ( ( 2 ): ): 147 - - 159 . . DOI: 10.1109/TPAMI.2004.1262177 http://doi.org/10.1109/TPAMI.2004.1262177 . .

A. Lazaridis , , , I. Mporas , , , T. Ganchev , , , 等 . . Support vector regression fusion scheme in phone duration modeling . . IEEE Int. Conf. on Acoustics, Speech and Signal Processing , , 2011 . . p.4732 - - 4735 . . DOI: 10.1109/ICASSP.2011.5947412 http://doi.org/10.1109/ICASSP.2011.5947412 . .

Y.S. Lee , , , S.B. Cho . . Activity recognition with Android phone using mixture-of-experts co-trained with labeled and unlabeled data . . Neurocomputing , , 2014 . . 126 106 - - 115 . . DOI: 10.1016/j.neucom.2013.05.044 http://doi.org/10.1016/j.neucom.2013.05.044 . .

Y. Li , , , J. Sun , , , C.K. Tang , , , 等 . . Lazy snapping . . ACM Trans. Graph. , , 2004 . . 23 ( ( 3 ): ): 303 - - 308 . . DOI: 10.1145/1015706.1015719 http://doi.org/10.1145/1015706.1015719 . .

Y. Liu , , , Y. Yu . . Interactive image segmentation based on level sets of probabilities . . IEEE Trans. Visual. Comput. Graph. , , 2012 . . 18 ( ( 2 ): ): 202 - - 213 . . DOI: 10.1109/TVCG.2011.77 http://doi.org/10.1109/TVCG.2011.77 . .

D. Martin , , , C. Fowlkes , , , D. Tal , , , 等 . . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics . . 8th IEEE Int. Conf. on Computer Vision , , 2001 . . p.416 - - 423 . . DOI: 10.1109/ICCV.2001.937655 http://doi.org/10.1109/ICCV.2001.937655 . .

R. Menon , , , G. Bhat , , , G.R. Saade , , , 等 . . Multivariate adaptive regression splines analysis to predict biomarkers of spontaneous preterm birth . . Acta Obstetr. Gynecol. Scandinav. , , 2014 . . 93 ( ( 4 ): ): 382 - - 391 . . DOI: 10.1111/aogs.12344 http://doi.org/10.1111/aogs.12344 . .

T.N.A. Nguyen , , , J. Cai , , , J. Zhang , , , 等 . . Robust interactive image segmentation using convex active contours . . IEEE Trans. Image Process. , , 2012 . . 21 ( ( 8 ): ): 3734 - - 3743 . . DOI: 10.1109/TIP.2012.2191566 http://doi.org/10.1109/TIP.2012.2191566 . .

J. Ning , , , L. Zhang , , , D. Zhang , , , 等 . . Interactive image segmentation by maximal similarity based region merging . . Patt. Recogn. , , 2010 . . 43 ( ( 2 ): ): 445 - - 456 . . DOI: 10.1016/j.patcog.2009.03.004 http://doi.org/10.1016/j.patcog.2009.03.004 . .

D. Opitz , , , R. Maclin . . Popular ensemble methods: an empirical study . . J. Artif. Intell. Res. , , 1999 . . 11 169 - - 198 . . DOI: 10.1613/jair.614 http://doi.org/10.1613/jair.614 . .

Y. Pauchard , , , T. Fitze , , , D. Browarnik , , , 等 . . Interactive graph-cut segmentation for fast creation of finite element models from clinical CT data for hip fracture prediction . . Comput. Methods Biomech. Biomed. Eng. , , 2016 . . 19 ( ( 16 ): ): 1693 - - 1703 . . DOI: 10.1080/10255842.2016.1181173 http://doi.org/10.1080/10255842.2016.1181173 . .

B. Peng , , , L. Zhang , , , D. Zhang . . A survey of graph theoretical approaches to image segmentation . . Patt. Recogn. , , 2013 . . 46 ( ( 3 ): ): 1020 - - 1038 . . DOI: 10.1016/j.patcog.2012.09.015 http://doi.org/10.1016/j.patcog.2012.09.015 . .

C. Qin , , , G. Zhang , , , Y. Zhou , , , 等 . . Integration of the saliency-based seed extraction and random walks for image segmentation . . Neurocomputing , , 2014 . . 129 378 - - 391 . . DOI: 10.1016/j.neucom.2013.09.021 http://doi.org/10.1016/j.neucom.2013.09.021 . .

C. Rother , , , V. Kolmogorov , , , A. Blake . . GrabCut: interactive foreground extraction using iterated graph cuts . . ACM Trans. Graph. , , 2004 . . 23 ( ( 3 ): ): 309 - - 314 . . DOI: 10.1145/1015706.1015720 http://doi.org/10.1145/1015706.1015720 . .

B.M. Shahshahani , , , D.A. Landgrebe . . The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon . . IEEE Trans. Geosci. Remote Sens. , , 1994 . . 32 ( ( 5 ): ): 1087 - - 1095 . . DOI: 10.1109/36.312897 http://doi.org/10.1109/36.312897 . .

M. Tang , , , L. Gorelick , , , O. Veksler , , , 等 . . GrabCut in one cut . . IEEE Int. Conf. on Computer Vision , , 2013 . . p.1769 - - 1776 . . DOI: 10.1109/ICCV.2013.222 http://doi.org/10.1109/ICCV.2013.222 . .

T. Wang , , , Q. Sun , , , Z. Ji , , , 等 . . Multi-layer graph constraints for interactive image segmentation via game theory . . Patt. Recogn. , , 2016 . . 55 28 - - 44 . . DOI: 10.1016/j.patcog.2016.01.018 http://doi.org/10.1016/j.patcog.2016.01.018 . .

X.Y. Wang , , , Q.Y. Wang , , , H.Y. Yang , , , 等 . . Color image segmentation using automatic pixel classification with support vector machine . . Neurocomputing , , 2011a . . 74 ( ( 18 ): ): 3898 - - 3911 . . DOI: 10.1016/j.neucom.2011.08.004 http://doi.org/10.1016/j.neucom.2011.08.004 . .

X.Y. Wang , , , T. Wang , , , J. Bu . . Color image segmentation using pixel wise support vector machine classification . . Patt. Recogn. , , 2011b . . 44 ( ( 4 ): ): 777 - - 787 . . DOI: 10.1016/j.patcog.2010.08.008 http://doi.org/10.1016/j.patcog.2010.08.008 . .

J. Wu , , , Y. Zhao , , , J.Y. Zhu , , , 等 . . MILCut: a sweeping line multiple instance learning paradigm for interactive image segmentation . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2014 . . p.256 - - 263 . . DOI: 10.1109/CVPR.2014.40 http://doi.org/10.1109/CVPR.2014.40 . .

S. Xiang , , , F. Nie , , , C. Zhang , , , 等 . . Interactive natural image segmentation via spline regression . . IEEE Trans. Image Process. , , 2009 . . 18 ( ( 7 ): ): 1623 - - 1632 . . DOI: 10.1109/TIP.2009.2018570 http://doi.org/10.1109/TIP.2009.2018570 . .

S. Xiang , , , F. Nie , , , C. Zhang . . Semi-supervised classification via local spline regression . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2010 . . 32 ( ( 11 ): ): 2039 - - 2053 . . DOI: 10.1109/TPAMI.2010.35 http://doi.org/10.1109/TPAMI.2010.35 . .

W. Yang , , , J. Cai , , , J. Zheng , , , 等 . . User-friendly interactive image segmentation through unified combinatorial user inputs . . IEEE Trans. Image Process. , , 2010 . . 19 ( ( 9 ): ): 2470 - - 2479 . . DOI: 10.1109/TIP.2010.2048611 http://doi.org/10.1109/TIP.2010.2048611 . .

J. Zhang , , , Z. Tang , , , J. Liu , , , 等 . . Recognition of flotation working conditions through froth image statistical modeling for performance monitoring . . Miner. Eng. , , 2016 . . 86 116 - - 129 . . DOI: 10.1016/j.mineng.2015.12.008 http://doi.org/10.1016/j.mineng.2015.12.008 . .

W. Zhang , , , A.T. Goh . . Evaluating seismic liquefaction potential using multivariate adaptive regression splines and logistic regression . . Geomech. Eng. , , 2016 . . 10 ( ( 3 ): ): 269 - - 284 . . DOI: 10.12989/gae.2016.10.3.269 http://doi.org/10.12989/gae.2016.10.3.269 . .

Y. Zhang , , , H. Song , , , J. Gu , , , 等 . . Interactive object extraction using hierarchical graph cuts . . Int. Conf. on Audio Language and Image Processing , , 2010 . . p.851 - - 858 . . DOI: 10.1109/ICALIP.2010.5685212 http://doi.org/10.1109/ICALIP.2010.5685212 . .

Y. Zhang , , , J. Wen , , , X. Wang , , , 等 . . Semi-supervised learning combining co-training with active learning . . Expert Syst. Appl. , , 2014 . . 41 ( ( 5 ): ): 2372 - - 2378 . . DOI: 10.1016/j.eswa.2013.09.035 http://doi.org/10.1016/j.eswa.2013.09.035 . .

W. Zhou , , , E.V. Garcia . . Nuclear image-guided approaches for cardiac resynchronization therapy (CRT) . . Curr. Cardiol. Rep. , , 2016 . . 18 ( ( 1 ): ): 1 - - 11 . . DOI: 10.1007/s11886-015-0687-4 http://doi.org/10.1007/s11886-015-0687-4 . .

W. Zhou , , , X. Hou , , , M. Piccinelli , , , 等 . . 3D fusion of LV venous anatomy on fluoroscopy venograms with epi-cardial surface on SPECT myocardial perfusion images for guiding CRT LV lead placement . . JACC Cardiov. Imag. , , 2014 . . 7 ( ( 12 ): ): 1239 - - 1248 . . DOI: 10.1016/j.jcmg.2014.09.002 http://doi.org/10.1016/j.jcmg.2014.09.002 . .

Z.H. Zhou . . When semi-supervised learning meets ensemble learning . . Front. Electr. Electron. Eng. China , , 2011 . . 6 ( ( 1 ): ): 6 - - 16 . . DOI: 10.1007/978-3-642-02326-2_53 http://doi.org/10.1007/978-3-642-02326-2_53 . .

Z.H. Zhou , , , M. Li . . Semi-supervised regression with co-training . . 19th Int. Joint Conf. on Artificial Intelligence , , 2005 . . p.908 - - 913 . . . .

Z.H. Zhou , , , M. Li . . Semisupervised regression with cotraining-style algorithms . . IEEE Trans. Knowl. Data Eng. , , 2007 . . 19 ( ( 11 ): ): 1479 - - 1493 . . DOI: 10.1109/TKDE.2007.190644 http://doi.org/10.1109/TKDE.2007.190644 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Dynamic prompting class distribution optimization for semi-supervised sound event detection

Interactive medical image segmentation with self-adaptive confidence calibration

Ensemble-transfer-learning-based channel parameter prediction in asymmetric massive MIMO systems

Vibration-based hypervelocity impact identification and localization

NGAT: attention in breadth and depth exploration for semi-supervised graph representation learning

Related Author

Yongzhao ZHAN

Qirong MAO

Yaxin SHEN

Qing ZHU

Lijian GAO

Fengping ZHU

Chuyun SHEN

Wenhao LI

Related Institution

School of Computer Science and Communication Engineering, Jiangsu University

Jiangsu Engineering Research Center of Big Data Ubiquitous Perception and Intelligent Agricultural Applications

School of Computer Science and Technology, East China Normal University

Huashan Hospital, Fudan University

Software Engineering Institute, East China Normal University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰