Visual recognition of cardiac pathology based on 3D parametric model reconstruction

Jinxiao XIAO; Yansong LI; Yun TIAN; Dongrong XU; Penghui LI; Shifeng ZHAO; Yunhe PAN

doi:10.1631/FITEE.2200102

Your Location：

Home >

Browse articles >

Visual recognition of cardiac pathology based on 3D parametric model reconstruction

Special Column on Visual Knowledge | Updated：2022-08-22

- Visual recognition of cardiac pathology based on 3D parametric model reconstruction
- 基于三维参数模型重建的心脏病理视觉识别
- Frontiers of Information Technology & Electronic Engineering Vol. 23, Issue 9, Pages: 1324-1337(2022)
- Affiliations：
  
  1.School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China
  2.Department of Psychiatry, Columbia University & New York State Psychiatric Institute, New York 10032, USA
  3.College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  † E-mail: xiaojinxiao@mail.bnu.edu.cn;
  liyansong@mail.bnu.edu.cn;
  ‡Corresponding author
- Funds：
- DOI：10.1631/FITEE.2200102
  CLC： TP391
- Received：16 March 2022，
  
  Revised：2022-08-16，
  
  Accepted：28 July 2022，
  
  Published：2022-09
- Accepted：
Scan QR Code
Jinxiao XIAO, Yansong LI, Yun TIAN, et al. Visual recognition of cardiac pathology based on 3D parametric model reconstruction[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(9): 1324-1337.
DOI：

Jinxiao XIAO, Yansong LI, Yun TIAN, et al. Visual recognition of cardiac pathology based on 3D parametric model reconstruction[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(9): 1324-1337. DOI： 10.1631/FITEE.2200102.

摘要

心脏图像的视觉识别对于心脏病理诊断和治疗具有重要意义。由于可用标注数据集有限，传统方法通常基于三维心脏图像的二维切片对病理分类特征进行提取，难以确保心脏解剖结构的整体一致性。为此，本文提出一种基于三维参数模型重建的心脏病理分类方法。首先，基于收缩末期和舒张末期时相心脏图像的多个三维心脏成像数据重建三维心脏模型。其次，基于重建的三维心脏模型，通过统计形状模型方法构建三维参数模型。然后，基于三维统计形状模型及其视觉知识约束对心脏数据进行增强。最后，提取不同时相的三维心脏模型的形状和运动特征，对心脏病理进行分类。在STACOM公开挑战赛的ACDC数据集上的实验验证了所提方法的优越性和有效性。

Abstract

Visual recognition of cardiac images is important for cardiac pathology diagnosis and treatment. Due to the limited availability of annotated datasets

traditional methods usually extract features directly from two-dimensional slices of three-dimensional (3D) heart images

followed by pathological classification. This process may not ensure the overall anatomical consistency in 3D heart. A new method for classification of cardiac pathology is therefore proposed based on 3D parametric model reconstruction. First

3D heart models are reconstructed based on multiple 3D volumes of cardiac imaging data at the end-systole (ES) and end-diastole (ED) phases. Next

based on these reconstructed 3D hearts

3D parametric models are constructed through the statistical shape model (SSM)

and then the heart data are augmented via the variation in shape parameters of one 3D parametric model with visual knowledge constraints. Finally

shape and motion features of 3D heart models across two phases are extracted to classify cardiac pathology. Comprehensive experiments on the automated cardiac diagnosis challenge (ACDC) dataset of the Statistical Atlases and Computational Modelling of the Heart (STACOM) workshop confirm the superior performance and efficiency of this proposed approach.

关键词

Keywords

references

Ammar A , Bouattane O , Youssfi M , 2021 . Automatic cardiac cine MRI segmentation and heart disease classification . Comput Med Imag Graph , 88 : 101864 . doi: 10.1016/j.compmedimag.2021.101864 http://doi.org/10.1016/j.compmedimag.2021.101864

Attar R , Pereañez M , Bowles C , et al. , 2019 . 3D cardiac shape prediction with deep neural networks: simultaneous use of images and patient metadata . Proc 22 nd Int Conf on Medical Image Computing and Computer-Assisted Intervention , p. 586 - 594 . doi: 10.1007/978-3-030-32245-8_65 http://doi.org/10.1007/978-3-030-32245-8_65

Bai WJ , Oktay O , Rueckert D , 2016 . Classification of myocardial infarcted patients by combining shape and motion features . Proc 6 th Int Workshop on Statistical Atlases and Computational Models of the Heart , p. 140 - 145 . doi: 10.1007/978-3-319-28712-6_15 http://doi.org/10.1007/978-3-319-28712-6_15

Bernard O , Lalande A , Zotti C , et al. , 2018 . Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans Med Imag , 37 ( 11 ): 2514 - 2525 . doi: 10.1109/TMI.2018.2837502 http://doi.org/10.1109/TMI.2018.2837502

Bernardino G , Benkarim O , Sanz-de la Garza M , et al. , 2020 . Handling confounding variables in statistical shape analysis-application to cardiac remodelling . Med Image Anal , 65 : 101792 . doi: 10.1016/j.media.2020.101792 http://doi.org/10.1016/j.media.2020.101792

Besl PJ , McKay ND , 1992 . A method for registration of 3-D shapes . IEEE Trans Patt Anal Mach Intell , 14 ( 2 ): 239 - 256 . doi: 10.1109/34.121791 http://doi.org/10.1109/34.121791

Biffi C , Cerrolaza JJ , Tarroni G , et al. , 2019 . 3D high-resolution cardiac segmentation reconstruction from 2D views using conditional variational autoencoders . Proc IEEE 16 th Int Symp on Biomedical Imaging , p. 1643 - 1646 . doi: 10.1109/ISBI.2019.8759328 http://doi.org/10.1109/ISBI.2019.8759328

Brock A , Donahue J , Simonyan K , 2019 . Large scale GAN training for high fidelity natural image synthesis . https://arxiv.org/abs/1809.11096 https://arxiv.org/abs/1809.11096

Cerqueira MD , Weissman NJ , Dilsizian V , et al. , 2002 . Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart—a statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association . Circulation , 105 ( 4 ): 539 - 542 . doi: 10.1161/hc0402.102975 http://doi.org/10.1161/hc0402.102975

Cetin I , Sanroma G , Petersen SE , et al. , 2017 . A radiomics approach to computer-aided diagnosis with cardiac cine-MRI . Proc 8 th Int Workshop on Statistical Atlases and Computational Models of the Heart , p. 82 - 90 . doi: 10.1007/978-3-319-75541-0_9 http://doi.org/10.1007/978-3-319-75541-0_9

Chang YK , Jung C , 2020 . Automatic cardiac MRI segmentation and permutation-invariant pathology classification using deep neural networks and point clouds . Neurocomputing , 418 : 270 - 279 . doi: 10.1016/j.neucom.2020.08.030 http://doi.org/10.1016/j.neucom.2020.08.030

Charles RQ , Su H , Mo KC , et al. , 2017 . PointNet: deep learning on point sets for 3D classification and segmentation . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 77 - 85 . doi: 10.1109/CVPR.2017.16 http://doi.org/10.1109/CVPR.2017.16

Chen X , Ravikumar N , Xia Y , et al. , 2021 . Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds . Med Imag Anal , 74 : 102228 . doi: 10.1016/j.media.2021.102228 http://doi.org/10.1016/j.media.2021.102228

Cootes TF , Taylor CJ , Cooper DH , et al. , 1995 . Active shape models—their training and application . Comput Vis Imag Underst , 61 ( 1 ): 38 - 59 . doi: 10.1006/cviu.1995.1004 http://doi.org/10.1006/cviu.1995.1004

Dryden IL , Mardia KV , 1998 . Statistical Shape Analysis . John Wiley & Sons , London, UK , p. 663 - 669 .

Frangi AF , Rueckert D , Schnabel JA , et al. , 2002 . Automatic construction of multiple-object three-dimensional statistical shape models: application to cardiac modeling . IEEE Trans Med Imag , 21 ( 9 ): 1151 - 1166 . doi: 10.1109/TMI.2002.804426 http://doi.org/10.1109/TMI.2002.804426

Gao ZF , Zhang HY , Wang DF , et al. , 2018 . Robust recovery of myocardial kinematics using dual <math id="M166"><mrow><msub><mi mathvariant="script">H</mi><mi>∞</mi></msub></mrow></math> criteria . Multim Tools Appl , 77 ( 17 ): 23043 - 23071 . doi: 10.1007/s11042-017-5395-1 http://doi.org/10.1007/s11042-017-5395-1

Gjesdal O , Bluemke DA , Lima JA , 2011 . Cardiac remodeling at the population level—risk factors, screening, and outcomes . Nat Rev Cardiol , 8 ( 12 ): 673 - 685 . doi: 10.1038/nrcardio.2011.154 http://doi.org/10.1038/nrcardio.2011.154

Isensee F , Jaeger PF , Full PM , et al. , 2018 . Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features . Proc 8 th Int Workshop on Statistical Atlases and Computational Models of the Heart , p. 120 - 129 . doi: 10.1007/978-3-319-75541-0_13 http://doi.org/10.1007/978-3-319-75541-0_13

Jolliffe IT , 2002 . Principal component analysis . J Mark Res , 87 ( 4 ): 513 .

Karras T , Laine S , Aila T , 2019 . A style-based generator architecture for generative adversarial networks . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 4396 - 4405 . doi: 10.1109/CVPR.2019.00453 http://doi.org/10.1109/CVPR.2019.00453

Khened M , Alex V , Krishnamurthi G , 2018 . Densely connected fully convolutional network for short-axis cardiac cine MR image segmentation and heart diagnosis using random forest . Proc 8 th Int Workshop on Statistical Atlases and Computational Models of the Heart , p. 140 - 151 . doi: 10.1007/978-3-319-75541-0_15 http://doi.org/10.1007/978-3-319-75541-0_15

Liu HF , Shi PC , 2009 . Maximum a posteriori strategy for the simultaneous motion and material property estimation of the heart . IEEE Trans Biomed Eng , 56 ( 2 ): 378 - 389 . doi: 10.1109/TBME.2008.2006012 http://doi.org/10.1109/TBME.2008.2006012

Pan YH , 1996 . The synthesis reasoning . Patt Recogn Artif Intell , 9 ( 3 ): 201 - 208 ( in Chinese ).

Pan YH , 2019 . On visual knowledge . Front Inform Technol Electron Eng , 20 ( 8 ): 1021 - 1025 . doi: 10.1631/FITEE.1910001 http://doi.org/10.1631/FITEE.1910001

Pan YH , 2020 . Multiple knowledge representation of artificial intelligence . Engineering , 6 ( 3 ): 216 - 217 . doi: 10.1016/j.eng.2019.12.011 http://doi.org/10.1016/j.eng.2019.12.011

Pan YH , 2021a . Miniaturized five fundamental issues about visual knowledge . Front Inform Technol Electron Eng , 22 ( 5 ): 615 - 618 . doi: 10.1631/FITEE.2040000 http://doi.org/10.1631/FITEE.2040000

Pan YH , 2021b . On visual understanding . Front Inform Technol Electron Eng , early access . doi: 10.1631/FITEE.2130000 http://doi.org/10.1631/FITEE.2130000

Parajuli N , Lu A , Duncan JS , 2016 . Left ventricle classification using active shape model and support vector machine . Proc 6 th Int Workshop on Statistical Atlases and Computational Models of the Heart , p. 154 - 161 . doi: 10.1007/978-3-319-28712-6_17 http://doi.org/10.1007/978-3-319-28712-6_17

Rodero C , Strocchi M , Marciniak M , et al. , 2021 . Linking statistical shape models and simulated function in the healthy adult human heart . PLoS Comput Biol , 17 ( 4 ): e1008851 . doi: 10.1371/journal.pcbi.1008851 http://doi.org/10.1371/journal.pcbi.1008851

Suinesiaputra A , Ablin P , Albà X , et al. , 2018 . Statistical shape modeling of the left ventricle: myocardial infarct classification challenge . IEEE J Biomed Health Inform , 22 ( 2 ): 503 - 515 . doi: 10.1109/JBHI.2017.2652449 http://doi.org/10.1109/JBHI.2017.2652449

Thermos S , Liu X , O’Neil A , et al. , 2021 . Controllable cardiac synthesis via disentangled anatomy arithmetic . Proc 24 th Int Conf on Medical Image Computing and Computer-Assisted Intervention , p. 160 - 170 . doi: 10.1007/978-3-030-87199-4_15 http://doi.org/10.1007/978-3-030-87199-4_15

Van Dijck C , Wirix-Speetjens R , Jonkers I , et al. , 2018 . Statistical shape model-based prediction of tibiofemoral cartilage . Comput Methods Biomech Biomed Eng , 21 ( 9 ): 568 - 578 . doi: 10.1080/10255842.2018.1495711 http://doi.org/10.1080/10255842.2018.1495711

Wolterink JM , Leiner T , Viergever MA , et al. , 2018 . Automatic segmentation and disease classification using cardiac cine MR images . Proc 8 th Int Workshop on Statistical Atlases and Computational Models of the Heart , p. 101 - 110 . doi: 10.1007/978-3-319-75541-0_11 http://doi.org/10.1007/978-3-319-75541-0_11

Zheng Q , Delingette H , Ayache N , 2019 . Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow . Med Imag Anal , 56 : 80 - 95 . doi: 10.1016/j.media.2019.06.001 http://doi.org/10.1016/j.media.2019.06.001

Zhou QY , Park J , Koltun V , 2016 . Fast global registration . Proc 14 th European Conf on Computer Vision , p. 766 - 782 . doi: 10.1007/978-3-319-46475-6_47 http://doi.org/10.1007/978-3-319-46475-6_47

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Significance extraction based on data augmentation for reinforcement learning

Hybrid embedding and joint training of stacked encoder for opinion question machine reading comprehension

Related Author

Yuxi HAN

Dequan LI

Yang YANG

Xiang-zhou HUANG

Si-liang TANG

Yin ZHANG

Bao-gang WEI

Related Institution

Faculty of Artificial Intelligence, Anhui University of Science and Technology

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰