自适应光学显微中基于深度学习的复杂波前探测方法

胡淑文; 胡乐佳; 龚薇; 李政翰; 斯科

doi:10.1631/FITEE.2000422

Your Location：

Home >

Browse articles >

自适应光学显微中基于深度学习的复杂波前探测方法

常规文章 | Updated：2022-06-06

- 自适应光学显微中基于深度学习的复杂波前探测方法
  Enhanced Publication
- Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes
- 信息与电子工程前沿（英文） 2021年22卷第10期页码：1277-1288
- Affiliations：
  
  Department of Neurology of the First Affiliated Hospital, State Key Laboratory of Modern Optical Instrumentation, Zhejiang University School of Medicine, Hangzhou 310009, China
  College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, China
  Research Units for Emotion and Emotion Disorders, Chinese Academy of Medical Sciences, MOE Frontier Science Center for Brain Research and Brain-Machine Integration, School of Brain Science and Brain Medicine, Zhejiang University, Hangzhou 310058, China
- Author bio：
  
  [ "", "Shuwen HU, first author of this invited paper, received her BS degree from Tianjin University, China. She is currently a master degree candidate at the College of Optical Science and Engineering, Zhejiang University, China. Her research interests include adaptive optics, machine learning, and deep tissue imaging" ]
  [ "", "Wei GONG, corresponding author of this invited paper, is a PI at Zhejiang University School of Medicine, China. She received her BS and MS degrees from Zhejiang University, China, and her PhD degree from the National University of Singapore, Singapore. She is a special expert of the Ministry of Education (MOE) and the Outstanding Youth of Zhejiang Province. Her research interests include biomedical imaging, optical clearing, and artificial intelligence in biomedicine. E-mail: weigong@zju.edu.cn" ]
  [ "", "Ke SI, corresponding author of this invited paper, is a professor at the College of Optical Science and Engineering, Zhejiang University, China, and a joint professor at Zhejiang University School of Medicine. He is the Vice Director of MOE Frontier Science Center for Brain Science and Brain-Machine Integration, and the Vice Dean of the School of Brain Science and Brain Medicine. He is now a corresponding expert of Front Inform Technol Electron Eng. His research focuses on biophotonics, deep tissue imaging, adaptive optics, and optogenetics" ]
- Funds：
  
  the National Natural Science Foundation of China(61735016);the National Natural Science Foundation of China(81771877);the National Natural Science Foundation of China(61975178);the Zhejiang Provincial Natural Science Foundation of China(LR20F050002);the Key R&D Program of Zhejiang Province, China(2021C03001);the CAMS Innovation Fund for Medical Sciences, China(2019-I2M-5-057);the Fundamental Research Funds for the Central Universities, China
- DOI：10.1631/FITEE.2000422
  中图分类号： O439
- 纸质出版日期：2021-10，
  
  网络出版日期：2021-03-29，
  
  收稿日期：2020-08-21，
  
  修回日期：2021-01-25，
- Accepted：
Scan QR Code
胡淑文, 胡乐佳, 龚薇, 等. 自适应光学显微中基于深度学习的复杂波前探测方法[J]. 信息与电子工程前沿（英文）, 2021,22(10):1277-1288.

SHUWEN HU, LEJIA HU, WEI GONG, et al. Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes. [J]. Frontiers of information technology & electronic engineering, 2021, 22(10): 1277-1288.
胡淑文, 胡乐佳, 龚薇, 等. 自适应光学显微中基于深度学习的复杂波前探测方法[J]. 信息与电子工程前沿（英文）, 2021,22(10):1277-1288. DOI： 10.1631/FITEE.2000422.

SHUWEN HU, LEJIA HU, WEI GONG, et al. Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes. [J]. Frontiers of information technology & electronic engineering, 2021, 22(10): 1277-1288. DOI： 10.1631/FITEE.2000422.

摘要

Shack-Hartmann波前传感器（SHWS）是自适应光学显微镜中用于波前传感的重要工具。然而，由复杂波前相位分布引起的畸变点阵限制了其探测性能。本文提出一种基于深度学习的波前探测方法，该方法结合了基于点扩散函数图像的泽尼克（Zernike）系数估计和波前相位分布拼接。该方法不仅仅使用每个子孔径的质心位移，而是通过子孔径的点扩散函数分布估计局部波前对应的Zernike系数，然后拼接局部波前进行重建。本文所提方法可实现高精度的复杂波前检测，获得的波前残差均方根误差值显著降低，在自适应光学显微中具有极大应用潜力。

Abstract

The Shack-Hartmann wavefront sensor (SHWS) is an essential tool for wavefront sensing in adaptive optical microscopes. However

the distorted spots induced by the complex wavefront challenge its detection performance. Here

we propose a deep learning based wavefront detection method which combines point spread function image based Zernike coefficient estimation and wavefront stitching. Rather than using the centroid displacements of each micro-lens

this method first estimates the Zernike coefficients of local wavefront distribution over each micro-lens and then stitches the local wavefronts for reconstruction. The proposed method can offer low root mean square wavefront errors and high accuracy for complex wavefront detection

and has potential to be applied in adaptive optical microscopes.

关键词

自适应光学波前探测深度学习泽尼克系数显微成像

Keywords

Adaptive opticsWavefront detectionDeep learningZernike coefficientsMicroscopy

references

MJ Booth. . Adaptive optical microscopy: the ongoing quest for a perfect image. . Light Sci Appl, , 2014. . 3((4):):e165DOI:10.1038/lsa.2014.46http://doi.org/10.1038/lsa.2014.46..

MJ Booth, , , MAA Neil, , , R Jukaitis, , , 等. . Adaptive aberration correction in a confocal microscope. . Proc Nat Acad Sci, , 2002. . 99((9):):5788--5792. . DOI:10.1073/pnas.082544799http://doi.org/10.1073/pnas.082544799..

SF Cheng, , , HH Li, , , YQ Luo, , , 等. . Artificial intelligence- assisted light control and computational imaging through scattering media. . J Innov Opt Health Sci, , 2019. . 12((4):):193006DOI:10.1142/s1793545819300064http://doi.org/10.1142/s1793545819300064..

A Cornea, , , PM Conn. . Fluorescence Microscopy: Super Resolution and Other Novel Techniques. . Elsevier, London, UK, , 2014. . p. 249..

M Cui. . Parallel wavefront optimization method for focusing light through random scattering media. . Opt Lett, , 2011. . 36((6):):870--872. . DOI:10.1364/ol.36.000870http://doi.org/10.1364/ol.36.000870..

BP Cumming, , , M Gu. . Direct determination of aberration functions in microscopy by an artificial neural network. . Opt Expr, , 2020. . 28((10):):14511--14521. . DOI:10.1364/OE.390856http://doi.org/10.1364/OE.390856..

GM Dai. . Wavefront Optics for Vision Correction, , ::Bellingham, USASPIE Press, , 2008. . DOI:10.1117/3.769212http://doi.org/10.1117/3.769212..

M Drozdzal, , , E Vorontsov, , , G Chartrand, , , 等. . The importance of skip connections in biomedical image segmentation. . Int Workshop on Deep Learning in Medical Image Analysis and Int Workshop on Large- Scale Annotation of Biomedical Data and Expert Label Synthesis, , 2016. . p. 179--187. . DOI:10.1007/978-3-319-46976-8_19http://doi.org/10.1007/978-3-319-46976-8_19..

TB Dubose, , , DF Gardner, , , AT Watnik. . Intensity-enhanced deep network wavefront reconstruction in ShackɃ Hartmann sensors. . Opt Lett, , 2020. . 45((7):):1699--1702. . DOI:10.1364/OL.389895http://doi.org/10.1364/OL.389895..

SLS Gómez, , , C González-Gutiérrez, , , ED Alonso, , , 等. . Improving adaptive optics reconstructions with a deep learning approach. . Int Conf on Hybrid Artificial Intelligence Systems, , 2018. . p. 74--83. . DOI:10.1007/978-3-319-92639-1_7http://doi.org/10.1007/978-3-319-92639-1_7..

LJ Hu, , , SW Hu, , , W Gong, , , 等. . Learning-based Shack- Hartmann wavefront sensor for high-order aberration detection. . Opt Expr, , 2019. . 27((23):):33504--33517. . DOI:10.1364/OE.27.033504http://doi.org/10.1364/OE.27.033504..

LJ Hu, , , SW Hu, , , YN Li, , , 等. . Reliability of wavefront shaping based on coherent optical adaptive technique in deep tissue focusing. . J Biophoton, , 2020. . 13((1):):e201900245DOI:10.1002/jbio.201900245http://doi.org/10.1002/jbio.201900245..

SW Hu, , , LJ Hu, , , BW Zhang, , , 等. . Simplifying the detection of optical distortions by machine learning. . J Innov Opt Health Sci, , 2020. . 13((3):):2040001DOI:10.1142/s1793545820400015http://doi.org/10.1142/s1793545820400015..

N Ji. . Adaptive optical fluorescence microscopy. . Nat Methods, , 2017. . 14((4):):374--280. . DOI:10.1038/nmeth.4218http://doi.org/10.1038/nmeth.4218..

YC Jin, , , YY Zhang, , , LJ Hu, , , 等. . Machine learning guided rapid focusing with sensor-less aberration corrections. . Opt Expr, , 2018. . 26((23):):30162--30171. . DOI:10.1364/OE.26.030162http://doi.org/10.1364/OE.26.030162..

ZH Li, , , ZP Yu, , , H Hui, , , 等. . Edge enhancement through scattering media enabled by optical wavefront shaping. . Photon Res, , 2020. . 8((6):):954--962. . DOI:10.1364/PRJ.388062http://doi.org/10.1364/PRJ.388062..

R Liu, , , ZY Li, , , JS Marvin, , , 等. . Direct wavefront sensing enables functional imaging of infragranular axons and spines. . Nat Methods, , 2019. . 16((7):):615--618. . DOI:10.1038/s41592-019-0434-7http://doi.org/10.1038/s41592-019-0434-7..

TL Liu, , , S Upadhyayula, , , DE Milkie, , , 等. . Observing the cell in its native state: imaging subcellular dynamics in multicellular organisms. . Science, , 2018. . 360((6386):):eaaq1392DOI:10.1126/science.aaq1392http://doi.org/10.1126/science.aaq1392..

VN Mahajan, , , GM Dai. . Orthonormal polynomials in wavefront analysis: analytical solution. . J Opt Soc Am A, , 2007. . 24((9):):2994--3016. . DOI:10.1364/JOSAA.24.002994http://doi.org/10.1364/JOSAA.24.002994..

Y Nishizaki, , , M Valdivia, , , R Horisaki, , , 等. . Deep learning wavefront sensing. . Opt Expr, , 2019. . 27((1):):240--251. . DOI:10.1364/OE.27.000240http://doi.org/10.1364/OE.27.000240..

SW Paine, , , JR Fienup. . Machine learning for improved image-based wavefront sensing. . Opt Lett, , 2018. . 43((6):):1235--1238. . DOI:10.1364/OL.43.001235http://doi.org/10.1364/OL.43.001235..

JH Park, , , LJ Kong, , , YF Zhou, , , 等. . Large-field-of-view imaging by multi-pupil adaptive optics. . Nat Methods, , 2017. . 14((6):):581--583. . DOI:10.1038/nmeth.4290http://doi.org/10.1038/nmeth.4290..

C Rodríguez, , , N Ji. . Adaptive optical microscopy for neurobiology. . Curr Opin Neurobiol, , 2018. . 5083--91. . DOI:10.1016/j.conb.2018.01.011http://doi.org/10.1016/j.conb.2018.01.011..

S Schott, , , J Bertolotti, , , JF Léger, , , 等. . Characterization of the angular memory effect of scattered light in biological tissues. . Opt Expr, , 2015. . 23((10):):13505--13516. . DOI:10.1364/OE.23.013505http://doi.org/10.1364/OE.23.013505..

R Swanson, , , M Lamb, , , C Correia, , , 等. . Wavefront reconstruction and prediction with convolutional neural networks. . Adaptive Optics Systems VI, , 2018. . Article 10703FDOI:10.1117/12.2312590http://doi.org/10.1117/12.2312590..

JY Tang, , , RN Germain, , , M Cui. . Superpenetration optical microscopy by iterative multiphoton adaptive compensation technique. . Proc Nat Acad Sci, , 2012. . 109((22):):8434--8439. . DOI:10.1073/pnas.1119590109http://doi.org/10.1073/pnas.1119590109..

PO Vanberg, , , GO de Xivry, , , O Absil, , , 等. . Machine learning for image-based wavefront sensing. . 33rd Conf on Neural Information Processing Systems, , 2019. . p. 1--6. . ..

BK Wang, , , M Barbiero, , , QM Zhang, , , 等. . Super- resolution optical microscope: principle, instrumentation, and application. . Front Inform Technol Electron Eng, , 2019. . 20((5):):608--630. . DOI:10.1631/FITEE.1800449http://doi.org/10.1631/FITEE.1800449..

K Wang, , , DE Milkie, , , A Saxena, , , 等. . Rapid adaptive optical recovery of optimal resolution over large volumes. . Nat Methods, , 2014. . 11((6):):625--628. . DOI:10.1038/nmeth.2925http://doi.org/10.1038/nmeth.2925..

K Wang, , , WZ Sun, , , CT Richie, , , 等. . Direct wavefront sensing for high-resolution in vivo imaging in scattering tissue. . Nat Commun, , 2015. . 67276DOI:10.1038/ncomms8276http://doi.org/10.1038/ncomms8276..

J Yoon, , , M Lee, , , K Lee, , , 等. . Optogenetic control of cell signaling pathway through scattering skull using wavefront shaping. . Sci Rep, , 2015. . 513289DOI:10.1038/srep13289http://doi.org/10.1038/srep13289..

ZP Yu, , , MY Xia, , , HH Li, , , 等. . Implementation of digital optical phase conjugation with embedded calibration and phase rectification. . Sci Rep, , 2019. . 9((1):):1537DOI:10.1038/s41598-018-38326-4http://doi.org/10.1038/s41598-018-38326-4..

ZP Zeng, , , H Xie, , , L Chen, , , 等. . Computational methods in super-resolution microscopy. . Front Inform Technol Electron Eng, , 2017. . 18((9):):1222--1235. . DOI:10.1631/FITEE.1601628http://doi.org/10.1631/FITEE.1601628..

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Quant 4.0: engineering quantitative investment with automated, explainable, and knowledge-driven artificial intelligence

Improved deep learning aided key recovery framework: applications to large-state block ciphers

Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning

Deep unfolding based channel estimation for wideband terahertz near-field massive MIMO systems

Combining graph neural network with deep reinforcement learning for resource allocation in computing force networks