A three-dimensional measurement method for binocular endoscopes based on deep learning

Hao YU; Changjiang ZHOU; Wei ZHANG; Liqiang WANG; Qing YANG; Bo YUAN

doi:10.1631/FITEE.2000679

Your Location：

Home >

Browse articles >

A three-dimensional measurement method for binocular endoscopes based on deep learning

Regular Papers | Updated：2022-05-19

- A three-dimensional measurement method for binocular endoscopes based on deep learning
  Enhanced Publication
- 基于深度学习的双目内窥镜三维测量方法
- Frontiers of Information Technology & Electronic Engineering Vol. 23, Issue 4, Pages: 653-660(2022)
- Affiliations：
  
  1.State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou310027, China
  2.Research Center for Intelligent Sensing, Zhejiang Lab, Hangzhou311100, China
- Author bio：
  
  ‡Corresponding author
- Funds：
- DOI：10.1631/FITEE.2000679
  CLC： TN29;TP391.4
- Received：07 December 2020，
  
  Accepted：22 March 2021，
  
  Published Online：24 January 2022，
  
  Published：2022-04
- Accepted：
Scan QR Code
Hao YU, Changjiang ZHOU, Wei ZHANG, et al. A three-dimensional measurement method for binocular endoscopes based on deep learning[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(4): 653-660.
DOI：

Hao YU, Changjiang ZHOU, Wei ZHANG, et al. A three-dimensional measurement method for binocular endoscopes based on deep learning[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(4): 653-660. DOI： 10.1631/FITEE.2000679.

摘要

在内窥镜临床检查中，病灶尺寸精确估计对诊断具有非常重要的意义。本文提出一种基于深度学习的双目内窥镜三维测量方法，可以克服传统双目匹配算法在弱纹理区域鲁棒性较差的缺点。利用三维扫描仪获得的目标三维数据和三维渲染软件仿真的双目相机创建虚拟双目图像数据集，用于训练视差预测模型进行三维测量。实验结果表明，所提方法相比传统双目匹配算法在视差准确度和视差图生成速度上分别提高48.9%和90.5%，能够提供更加准确、可靠的病灶尺寸信息，提高内窥镜诊断效率。

Abstract

Abstract:

In the practice of clinical endoscopy

the precise estimation of the lesion size is quite significant for diagnosis. In this paper

we propose a three-dimensional (3D) measurement method for binocular endoscopes based on deep learning

which can overcome the poor robustness of the traditional binocular matching algorithm in texture-less areas. A simulated binocular image dataset is created from the target 3D data obtained by a 3D scanner and the binocular camera is simulated by 3D rendering software to train a disparity estimation model for 3D measurement. The experimental results demonstrate that

compared with the traditional binocular matching algorithm

the proposed method improves the accuracy and disparity map generation speed by 48.9% and 90.5%

respectively. This can provide more accurate and reliable lesion size and improve the efficiency of endoscopic diagnosis.

关键词

Keywords

references

Ahmad I , Levine JB , Anderson JC , 2016 . Endoscopic measurement of colorectal polyps: how do we measure up? Gastroenterology , 150 ( 3 ): 769 - 771 . doi: 10.1053/j.gastro.2016.01.020 http://doi.org/10.1053/j.gastro.2016.01.020

Anderson BW , Smyrk TC , Anderson KS , et al ., 2016 . Endoscopic overestimation of colorectal polyp size . Gastrointest Endosc , 83 ( 1 ): 201 - 208 . doi: 10.1016/j.gie.2015.06.058 http://doi.org/10.1016/j.gie.2015.06.058

Cai HF , Wang R , Li Y , et al ., 2018 . Role of 3D reconstruction in the evaluation of patients with lower segment oesophageal cancer . J Thorac Dis , 10 ( 7 ): 3940 - 3947 . doi: 10.21037/jtd.2018.06.119 http://doi.org/10.21037/jtd.2018.06.119

Dimas G , Bianchi F , Iakovidis DK , et al ., 2020 . Endoscopic single-image size measurements . Meas Sci Technol , 31 ( 7 ): 074010 . doi: 10.1088/1361-6501/ab803c http://doi.org/10.1088/1361-6501/ab803c

Furukawa R , Aoyama M , Hiura S , et al ., 2014 . Calibration of a 3D endoscopic system based on active stereo method for shape measurement of biological tissues and specimen . Proc 36 th Annual Int Conf of the IEEE Engineering in Medicine and Biology Society , p. 4991 - 4994 . doi: 10.1109/EMBC.2014.6944745 http://doi.org/10.1109/EMBC.2014.6944745

Hirschmüller H , 2008 . Stereo processing by semiglobal matching and mutual information . IEEE Trans Patt Anal Mach Intell , 30 ( 2 ): 328 - 341 . doi: 10.1109/TPAMI.2007.1166 http://doi.org/10.1109/TPAMI.2007.1166

Kendall A , Martirosyan H , Dasgupta S , et al ., 2017 . End-to-end learning of geometry and context for deep stereo regression . Proc IEEE Int Conf on Computer Vision , p. 66 - 75 . doi: 10.1109/ICCV.2017.17 http://doi.org/10.1109/ICCV.2017.17

Khamis S , Fanello S , Rhemann C , et al ., 2018 . StereoNet: guided hierarchical refinement for real-time edge-aware depth prediction . https://arxiv.org/abs/1807.08865v1 https://arxiv.org/abs/1807.08865v1

Nomura K , Kikuchi D , Kaise M , et al ., 2019 . Comparison of 3D endoscopy and conventional 2D endoscopy in gastric endoscopic submucosal dissection: an ex vivo animal study . Surg Endosc , 33 ( 12 ): 4164 - 4170 . doi: 10.1007/s00464-019-06726-w http://doi.org/10.1007/s00464-019-06726-w

Ogino-Nishimura E , Nakagawa T , Sakamoto T , et al ., 2015 . Efficacy of three-dimensional endoscopy in endonasal surgery . Auris Nasus Larynx , 42 ( 3 ): 203 - 207 . doi: 10.1016/j.anl.2014.10.004 http://doi.org/10.1016/j.anl.2014.10.004

Omori J , Goto O , Higuchi K , et al ., 2020 . Three-dimensional flexible endoscopy can facilitate efficient and reliable endoscopic hand suturing: an ex-vivo study . Clin Endosc , 53 ( 3 ): 334 - 338 . doi: 10.5946/ce.2019.207 http://doi.org/10.5946/ce.2019.207

Sakata S , Mcivor F , Klein K , et al ., 2018 . Measurement of polyp size at colonoscopy: a proof-of-concept simulation study to address technology bias . Gut , 67 ( 2 ): 206 - 208 . doi: 10.1136/gutjnl-2016-312915 http://doi.org/10.1136/gutjnl-2016-312915

Shaw MJ , Shaukat A , 2016 . Does polyp size scatter matter? Gastrointest Endosc , 83 ( 1 ): 209 - 211 . doi: 10.1016/j.gie.2015.08.060 http://doi.org/10.1016/j.gie.2015.08.060

Wang D , Liu H , Cheng X , 2018 . A miniature binocular endoscope with local feature matching and stereo matching for 3D measurement and 3D reconstruction . Sensors , 18 ( 7 ): 2243 . doi: 10.3390/s18072243 http://doi.org/10.3390/s18072243

Wang XZ , Nie YF , Lu SP , et al ., 2020 . Deep convolutional network for stereo depth mapping in binocular endoscopy . IEEE Access , 8 : 73241 - 73249 . doi: 10.1109/ACCESS.2020.2987767 http://doi.org/10.1109/ACCESS.2020.2987767

Žbontar J , LeCun Y , 2014 . Computing the stereo matching cost with a convolutional neural network . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 1592 - 1599 . doi: 10.1109/CVPR.2015.7298767 http://doi.org/10.1109/CVPR.2015.7298767

Zhang FH , Prisacariu V , Yang RG , et al ., 2019 . GA-Net: guided aggregation net for end-to-end stereo matching . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 185 - 194 . doi: 10.1109/CVPR.2019.00027 http://doi.org/10.1109/CVPR.2019.00027

Views

134

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

MltAuxTSPP: a unified benchmark for deep learning-based traffic state prediction with multi-source auxiliary data

Q-space-coordinate-guided neural networks for high-fidelity diffusion tensor estimation from minimal diffusion-weighted images

CRGT-SA: an interlaced and spatiotemporal deep learning model for network intrusion detection

Federated model with contrastive learning and adaptive control variates for human activity recognition

A comprehensive survey of physical adversarial vulnerabilities in autonomous driving systems

Related Author

Yusong ZHOU

Xiaoyu JIANG

Shu SUN

Xinmin ZHANG

Yuanqiu MO

Zhihuan SONG

Maokun ZHENG

Zhi LI

Related Institution

School of Mathematics, Southeast University

State Key Laboratory of Industrial Control Technology, Zhejiang University

College of Control Science and Engineering, Zhejiang University

State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University

School of Electronic and Electrical Engineering, Shanghai University of Engineering Science

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰