High-resolution spectral video acquisition

Lin-sen CHEN; Tao YUE; Xun CAO; Zhan MA; David J. BRADY

doi:10.1631/FITEE.1700098

Your Location：

Home >

Browse articles >

High-resolution spectral video acquisition

Special Feature on Computational Photography | Updated：2022-05-19

- High-resolution spectral video acquisition
- 高分辨率光谱视频采集
- Frontiers of Information Technology & Electronic Engineering Vol. 18, Issue 9, Pages: 1250-1260(2017)
- Affiliations：
  
  School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China
  Department of Electrical & Computer Engineering, Duke University, Durham NC27708, USA
- Author bio：
  
  [ "CHEN Lin-sen, E-mail: njucls@163.com" ]
  Tao YUE, E-mail: yuetao@nju.edu.cn
- Funds：
- DOI：10.1631/FITEE.1700098
  CLC： TN919.8
- Received：08 February 2017，
  
  Revised：2017-;9-15，
  
  Published：2017-09
- Accepted：
Scan QR Code
Lin-sen CHEN, Tao YUE, Xun CAO, et al. High-resolution spectral video acquisition[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(9): 1250-1260.
DOI：

Lin-sen CHEN, Tao YUE, Xun CAO, et al. High-resolution spectral video acquisition[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(9): 1250-1260. DOI： 10.1631/FITEE.1700098.

摘要

与传统视频采集技术相比，高分辨率光谱视频采集技术能够为精细农业、环境监测、遥感、军事刑侦等诸多重要领域提供高维度、高精度场景特征信息。传统光谱仪主要采用的扫描系统无法满足动态场景下的采集需求，从而限制了光谱成像技术的应用范围。近来，得益于计算摄像理论和半导体技术的快速发展，光谱视频采集技术成为可能。本文首先详细介绍了传统光谱仪的基本理论与工作原理。其次，针对光谱视频采集的核心问题，从成像理论、系统结构、关键器件等方面回顾了近年出现的瞬拍式光谱采集技术。从进一步提升光谱视频采集精度的角度出发，通过主观质量评价与客观指标比较，分析和总结了各光谱成像系统的性能优势与弊端，同时讨论了计算摄像理论、半导体制备工艺对光谱视频采集技术发展的关键作用。

Abstract

Compared with conventional cameras

spectral imagers provide many more features in the spectral domain. They have been used in various fields such as material identification

remote sensing

precision agriculture

and surveillance. Traditional imaging spectrometers use generally scanning systems. They cannot meet the demands of dynamic scenarios. This limits the practical applications for spectral imaging. Recently

with the rapid development in computational photography theory and semiconductor techniques

spectral video acquisition has become feasible. This paper aims to offer a review of the state-of-the-art spectral imaging technologies

especially those capable of capturing spectral videos. Finally

we evaluate the performances of the existing spectral acquisition systems and discuss the trends for future work.

关键词

Keywords

references

F.M. Abed , , , S.H. Amirshahi , , , M.R.M. Abed . . Reconstruction of reflectance data using an interpolation technique . . J. Opt. Soc. Am. A , , 2009 . . 26 ( ( 3 ): ): 613 - - 624 . . DOI: 10.1364/JOSAA.26.000613 http://doi.org/10.1364/JOSAA.26.000613 . .

E.H. Adelson , , , J.R. Bergen . . The plenoptic function and the elements of early vision. In : Landy M.S., Movshon, J.A. (Eds.) . . Computational Models of Visual Processing. MIT Press, Cambridge , , 1991 . . p.3 - - 20 . . . .

G.R. Arce , , , D.J. Brady , , , L. Carin , , , 等 . . Compressive coded aperture spectral imaging: an introduction . . IEEE Signal Process. Mag. , , 2014 . . 31 ( ( 1 ): ): 105 - - 115 . . DOI: 10.1109/MSP.2013.2278763 http://doi.org/10.1109/MSP.2013.2278763 . .

J. Bao , , , M.G. Bawendi . . A colloidal quantum dot spectrometer . . Nature , , 2015 . . 523 ( ( 7558 ): ): 67 - - 70 . . DOI: 10.1038/nature14576 http://doi.org/10.1038/nature14576 . .

J.M. Bioucas-Dias , , , M.A. Figueiredo . . A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration . . IEEE Trans. Imag. Process. , , 2007 . . 16 ( ( 12 ): ): 2992 - - 3004 . . DOI: 10.1109/TIP.2007.909319 http://doi.org/10.1109/TIP.2007.909319 . .

A. Bodkin , , , A. Sheinis , , , A. Norton , , , 等 . . Snapshot hyperspectral imaging: the hyperpixel array camera . . SPIE , , 2009 . . 7334 73340H DOI: 10.1117/12.818929 http://doi.org/10.1117/12.818929 . .

S. Boyd , , , N. Parikh , , , E. Chu , , , 等 . . Distributed optimization and statistical learning via the alternating direction method of multipliers . . Found. Trends Mach. Learn. , , 2011 . . 3 ( ( 1 ): ): 1 - - 122 . . DOI: 10.1561/2200000016 http://doi.org/10.1561/2200000016 . .

E.J. Candès , , , M.B. Wakin . . An introduction to compressive sampling . . IEEE Signal Process. Mag. , , 2008 . . 25 ( ( 2 ): ): 21 - - 30 . . DOI: 10.1109/MSP.2007.914731 http://doi.org/10.1109/MSP.2007.914731 . .

E.J. Candès , , , J. Romberg , , , T. Tao . . Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information . . IEEE Trans. Inform. Theory , , 2006 . . 52 ( ( 2 ): ): 489 - - 509 . . DOI: 10.1109/TIT.2005.862083 http://doi.org/10.1109/TIT.2005.862083 . .

X. Cao , , , H. Du , , , X. Tong , , , 等 . . A prism-mask system for multispectral video acquisition . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2011a . . 33 ( ( 12 ): ): 2423 - - 2435 . . DOI: 10.1109/TPAMI.2011.80 http://doi.org/10.1109/TPAMI.2011.80 . .

X. Cao , , , X. Tong , , , Q. Dai , , , 等 . . High resolution multispectral video capture with a hybrid camera system . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2011b . . p.297 - - 304 . . DOI: 10.1109/CVPR.2011.5995418 http://doi.org/10.1109/CVPR.2011.5995418 . .

X. Cao , , , T. Yue , , , X. Lin , , , 等 . . Computational snapshot multispectral cameras . . IEEE Signal Process. Mag. , , 2016 . . 33 ( ( 5 ): ): 95 - - 108 . . DOI: 10.1109/MSP.2016.2582378 http://doi.org/10.1109/MSP.2016.2582378 . .

A. Chakrabarti , , , T. Zickler . . Statistics of real-world hyperspectral images . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2011 . . p.193 - - 200 . . DOI: 10.1109/CVPR.2011.5995660 http://doi.org/10.1109/CVPR.2011.5995660 . .

M. Descour , , , E. Dereniak . . Computed-tomography imaging spectrometer: experimental calibration and reconstruction results . . Appl. Opt. , , 1995 . . 34 ( ( 22 ): ): 4817 - - 4826 . . DOI: 10.1364/AO.34.004817 http://doi.org/10.1364/AO.34.004817 . .

M. Descour , , , C.E. Volin , , , B.K. Ford , , , 等 . . Snapshot hyperspectral imaging. In : Integrated Computational Imaging Systems . . OSA Publishing, Washington D.C. , , 2001 . . paper IWB4 . .

D.L. Donoho . . Compressed sensing . . IEEE Trans. Inform. Theory , , 2006 . . 52 ( ( 4 ): ): 1289 - - 1306 . . DOI: 10.1109/TIT.2006.871582 http://doi.org/10.1109/TIT.2006.871582 . .

H. Du , , , X. Tong , , , X. Cao , , , 等 . . A prism-based system for multispectral video acquisition . . IEEE 12th Int. Conf. on Computer Vision , , 2009 . . p.175 - - 182 . . DOI: 10.1109/ICCV.2009.5459162 http://doi.org/10.1109/ICCV.2009.5459162 . .

L. Gao , , , R.T. Kester , , , N. Hagen , , , 等 . . Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy . . Opt. Expr. , , 2010 . . 18 ( ( 14 ): ): 14330 - - 14344 . . DOI: 10.1364/OE.18.014330 http://doi.org/10.1364/OE.18.014330 . .

N. Gat . . Imaging spectroscopy using tunable filters: a review . . SPIE , , 2000 . . 4056 50 - - 64 . . DOI: 10.1117/12.381686 http://doi.org/10.1117/12.381686 . .

M. Golbabaee , , , P. Vandergheynst . . Compressed sensing of simultaneous low-rank and joint-sparse matrices . . 2012 . . arXiv 1211 - - 5058 . . http://arxiv.org/abs/1211.5058 http://arxiv.org/abs/1211.5058 , , . .

R.O. Green , , , M.L. Eastwood , , , C.M. Sarture , , , 等 . . Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS) . . Remote Sens. Environ. , , 1998 . . 65 ( ( 3 ): ): 227 - - 248 . . DOI: 10.1016/S0034-4257(98)00064-9 http://doi.org/10.1016/S0034-4257(98)00064-9 . .

A.R. Harvey , , , J.E. Beale , , , A.H. Greenaway , , , 等 . . Technology options for imaging spectrometry . . Int. Symp. on Optical Science and Technology , , 2000 . . p.13 - - 24 . . DOI: 10.1117/12.406592 http://doi.org/10.1117/12.406592 . .

E. Herrala , , , J.T. Okkonen , , , T.S. Hyvarinen , , , 等 . . Imaging spectrometer for process industry applications . . SPIE , , 1994 . . 2248 33 - - 40 . . DOI: 10.1117/12.194344 http://doi.org/10.1117/12.194344 . .

J. Hunicz , , , D. Piernikarski . . Investigation of combustion in a gasoline engine using spectrophotometric methods . . SPIE , , 2001 . . 4516 307 - - 314 . . DOI: 10.1117/12.435940 http://doi.org/10.1117/12.435940 . .

A.L. Kindzelskii , , , Z.Y. Yang , , , G.J. Nabel , , , 等 . . Ebola virus secretory glycoprotein (sGP) diminishes Fc $$\gamma$$ RIIIB-to-CR3 proximity on neutrophils . . J. Immun. , , 2000 . . 164 ( ( 2 ): ): 953 - - 958 . . DOI: 10.4049/jimmunol.164.2.953 http://doi.org/10.4049/jimmunol.164.2.953 . .

D. Kittle , , , K. Choi , , , A. Wagadarikar , , , 等 . . Multiframe image estimation for coded aperture snapshot spectral imagers . . Appl. Opt. , , 2010 . . 49 ( ( 36 ): ): 6824 - - 6833 . . . .

J. Lawlor , , , D. Fletcher-Holmes , , , A. Harvey , , , 等 . . In vivo hyperspectral imaging of human retina and optic disc . . Invest. Ophthalmol. Vis. Sci. , , 2002 . . 43 ( ( 13 ): ): 4350 - - 4350 . . DOI: 10.1364/AO.49.006824 http://doi.org/10.1364/AO.49.006824 . .

X. Liao , , , H. Li , , , L. Carin . . Generalized alternating projection for weighted- $$\ell_2,1$$ minimization with applications to model-based compressive sensing . . SIAM J. Imag. Sci. , , 2014 . . 7 ( ( 2 ): ): 797 - - 823 . . DOI: 10.1137/130936658 http://doi.org/10.1137/130936658 . .

X. Lin , , , Y. Liu , , , J. Wu , , , 等 . . Spatial-spectral encoded compressive hyperspectral imaging . . ACM Trans. Graph. , , 2014a . . 33 ( ( 6 ): ): Article 233 DOI: 10.1145/2661229.2661262 http://doi.org/10.1145/2661229.2661262 . .

X. Lin , , , G. Wetzstein , , , Y. Liu , , , 等 . . Dual-coded compressive hyperspectral imaging . . Opt. Lett. , , 2014b . . 39 ( ( 7 ): ): 2044 - - 2047 . . DOI: 10.1364/OL.39.002044 http://doi.org/10.1364/OL.39.002044 . .

C. Ma , , , X. Cao , , , R. Wu , , , 等 . . Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system . . Opt. Lett. , , 2014 . . 39 ( ( 4 ): ): 937 - - 940 . . DOI: 10.1364/OL.39.000937 http://doi.org/10.1364/OL.39.000937 . .

C.L. Mansfield . . Seeing into the Past . . 2005 . . http://www.nasa.gov/vision/earth/technologies/scrolls.html http://www.nasa.gov/vision/earth/technologies/scrolls.html , , . .

P.A. Mitchell . . Hyperspectral digital imagery collection experiment (HYDICE) . . SPIE , , 1995 . . 2587 70 - - 95 . . DOI: 10.1117/12.226807 http://doi.org/10.1117/12.226807 . .

J.M. Mooney , , , V.E. Vickers , , , M. An , , , 等 . . High-throughput hyperspectral infrared camera . . J. Opt. Soc. Am. A , , 1997 . . 14 ( ( 11 ): ): 2951 - - 2961 . . DOI: 10.1364/JOSAA.14.002951 http://doi.org/10.1364/JOSAA.14.002951 . .

P. Morovic , , , G.D. Finlayson . . Metamer-set-based approach to estimating surface reflectance from camera RGB . . J. Opt. Soc. Am. A , , 2006 . . 23 ( ( 8 ): ): 1814 - - 1822 . . DOI: 10.1364/JOSAA.23.001814 http://doi.org/10.1364/JOSAA.23.001814 . .

H.R. Morris , , , C.C. Hoyt , , , P.J. Treado . . Imaging spectrometers for fluorescence and Raman microscopy: acousto-optic and liquid crystal tunable filters . . Appl. Spectr. , , 1994 . . 48 ( ( 7 ): ): 857 - - 866 . . . .

R.M. Nguyen , , , D.K. Prasad , , , M.S. Brown . . Training-based spectral reconstruction from a single RGB image . . European Conf. on Computer Vision , , 2014 . . p.186 - - 201 . . DOI: 10.1007/978-3-319-10584-0_13 http://doi.org/10.1007/978-3-319-10584-0_13 . .

W.S. Oh , , , M.S. Brown , , , M. Pollefeys , , , 等 . . Do it yourself hyperspectral imaging with everyday digital cameras . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2016 . . p.2461 - - 2469 . . DOI: 10.1109/CVPR.2016.270 http://doi.org/10.1109/CVPR.2016.270 . .

J. Radon . . Über die Bestimmung von Funktionen durch ihre Integralwerte längs gewisser Mannigfaltigkeiten . . Akad. Wiss. , , 1917 . . 69 262 - - 277 (in German) . . . .

B. Rørslett . . All you ever wanted to know about digital UV and IR photography, but could not afford to ask . . 2004 . . http://www.naturfotograf.com/UV_IR_rev00.html http://www.naturfotograf.com/UV_IR_rev00.html , , . .

Y.Y. Schechner , , , S.K. Nayar . . Generalized mosaicing: wide field of view multispectral imaging . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2002 . . 24 ( ( 10 ): ): 1334 - - 1348 . . DOI: 10.1109/TPAMI.2002.1039205 http://doi.org/10.1109/TPAMI.2002.1039205 . .

L.A. Shepp , , , Y. Vardi . . Maximum likelihood reconstruction for emission tomography . . IEEE Trans. Med. Imag. , , 1982 . . 1 ( ( 2 ): ): 113 - - 122 . . DOI: 10.1109/TMI.1982.4307558 http://doi.org/10.1109/TMI.1982.4307558 . .

L. Su , , , Z. Zhou , , , Y. Yuan , , , 等 . . A snapshot light field imaging spectrometer . . Opt.-Int. J. Light Electr. Opt. , , 2015 . . 126 ( ( 9 ): ): 877 - - 881 . . DOI: 10.1016/j.ijleo.2015.01.034 http://doi.org/10.1016/j.ijleo.2015.01.034 . .

A.A. Wagadarikar , , , N.P. Pitsianis , , , X. Sun , , , 等 . . Video rate spectral imaging using a coded aperture snapshot spectral imager . . Opt. Expr. , , 2009 . . 17 ( ( 8 ): ): 6368 - - 6388 . . DOI: 10.1364/OE.17.006368 http://doi.org/10.1364/OE.17.006368 . .

R.M. Willett , , , M.F. Duarte , , , M.A. Davenport , , , 等 . . Sparsity and structure in hyperspectral imaging: sensing, reconstruction, and target detection . . IEEE Signal Process. Mag. , , 2014 . . 31 ( ( 1 ): ): 116 - - 126 . . DOI: 10.1109/MSP.2013.2279507 http://doi.org/10.1109/MSP.2013.2279507 . .

Y. Wu , , , I.O. Mirza , , , G.R. Arce , , , 等 . . Development of a digital-micromirror-device-based multishot snapshot spectral imaging system . . Opt. Lett. , , 2011 . . 36 ( ( 14 ): ): 2692 - - 2694 . . DOI: 10.1364/OL.36.002692 http://doi.org/10.1364/OL.36.002692 . .

M. Yamaguchi , , , H. Haneishi , , , H. Fukuda , , , 等 . . High-fidelity video and still-image communication based on spectral information: natural vision system and its applications . . SPIE , , 2006 . . 6062 60620G DOI: 10.1117/12.649454 http://doi.org/10.1117/12.649454 . .

F. Yasuma , , , T. Mitsunaga , , , D. Iso , , , 等 . . Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum . . IEEE Trans. Imag. Process. , , 2010 . . 19 ( ( 9 ): ): 2241 - - 2253 . . DOI: 10.1109/TIP.2010.2046811 http://doi.org/10.1109/TIP.2010.2046811 . .

Z. Zhou , , , Y. Yuan , , , X.L. Bin . . Light field imaging spectrometer: conceptual design and simulated performance . . Frontiers in Optics/Laser Science XXVI , , 2010 . . paper FThM3. DOI: 10.1364/FIO.2010.FThM3 http://doi.org/10.1364/FIO.2010.FThM3 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰