基于K-最近邻域搜寻的ToF深度摄像机和被动立体深度获取的融合技术研究

Li-wei LIU; Yang LI; Ming ZHANG; Liang-hao WANG; Dong-xiao LI

doi:10.1631/jzus.C1300194

Your Location：

Home >

Browse articles >

基于K-最近邻域搜寻的ToF深度摄像机和被动立体深度获取的融合技术研究

计算机与自动化 | Updated：2022-05-19

- 基于K-最近邻域搜寻的ToF深度摄像机和被动立体深度获取的融合技术研究
- K-nearest neighborhood based integration of time-of-flight cameras and passive stereo for high-accuracy depth maps
- 信息与电子工程前沿（英文） 2014年15卷第3期页码：174-186
- Affiliations：
  
  1.Institute of Information and Communication Engineering, Zhejiang University, Hangzhou 310027, China
  2.Zhejiang Provincial Key Laboratory of Information Network Technology, Hangzhou 310027, China
- Author bio：
  
  [ "Li-wei LIU, lllw19870907@zju.edu.cn" ]
  [ "Yang LI, lychina@zju.edu.cn" ]
  Liang-hao WANG, wanglianghao@zju.edu.cn
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 61072081 and 61271338), the National High-Tech R & D Program (863) of China (No. 2012AA011505), the National Science and Technology Major Project of the Ministry of Science and Technology of China (No. 2009ZX01033-001-007), the Key Science and Technology Innovation Team of Zhejiang Province (No. 2009R50003), and the China Postdoctoral Science Foundation (No. 2012T50545)
- DOI：10.1631/jzus.C1300194
  中图分类号： TP317.4
- 收稿：2013-07-15，
  
  修回：2014-2-19，
  
  录用：2013-11-11，
  
  纸质出版：2014-03
- Accepted：
Scan QR Code
基于K-最近邻域搜寻的ToF深度摄像机和被动立体深度获取的融合技术研究[J]. 信息与电子工程前沿（英文）, 2014,15(3):174-186.

Li-wei LIU, Yang LI, Ming ZHANG, et al. K-nearest neighborhood based integration of time-of-flight cameras and passive stereo for high-accuracy depth maps[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(3): 174-186.
基于K-最近邻域搜寻的ToF深度摄像机和被动立体深度获取的融合技术研究[J]. 信息与电子工程前沿（英文）, 2014,15(3):174-186. DOI： 10.1631/jzus.C1300194.

Li-wei LIU, Yang LI, Ming ZHANG, et al. K-nearest neighborhood based integration of time-of-flight cameras and passive stereo for high-accuracy depth maps[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(3): 174-186. DOI： 10.1631/jzus.C1300194.

摘要

研究目的

场景深度获取是目前三维显示研究领域中最关键的技术。目前主流的深度获取方式主要有两种：基于双目图像的被动立体匹配技术和基于主动光源测距的ToF深度摄像系统。两种方式获取的深度各有优缺点，本文旨在分析两者优缺点，融合两种方法获取的结果，形成质量更好的场景深度。

创新要点

利用ToF深度摄像机的优势区域指导立体匹配的过程，优化了立体匹配的结果，同时提出了一种新的代价优化深度融合算法，将ToF深度摄像机的采集结果和立体匹配产生的深度融合成精度更高的深度图。

研究方法

主要包含两部分算法，流程可见图1。首先，利用ToF深度摄像机提供的深度测量图和对应的光强振幅图构建能量函数，利用该能量函数，结合K-最近邻域算法，指导原始立体匹配过程。然后，将优化后的立体匹配结果和TOF深度图结合，构建代价函数，选取最优深度解，作为最终融合深度。

重要结论

实验结果显示，本文采用的算法获取的深度图优于单一主动或被动方法获取的深度图，也优于另一类全局优化的深度融合算法。

Abstract

Both time-of-flight (ToF) cameras and passive stereo can provide the depth information for their corresponding captured real scenes

but they have innate limitations. ToF cameras and passive stereo are intrinsically complementary for certain tasks. It is desirable to appropriately leverage all the available information by ToF cameras and passive stereo. Although some fusion methods have been presented recently

they fail to consider ToF reliability detection and ToF based improvement of passive stereo. As a result

this study proposes an approach to integrating ToF cameras and passive stereo to obtain high-accuracy depth maps. The main contributions are: (1) An energy cost function is devised to use data from ToF cameras to boost the stereo matching of passive stereo; (2) A fusion method is used to combine the depth information from both ToF cameras and passive stereo to obtain high-accuracy depth maps. Experiments show that the proposed approach achieves improved results with high accuracy and robustness.

关键词

Keywords

references

M. Attamimi , A. Mizutani , T. Nakamura , 等 . Real-time 3D visual sensor for robust object recognition . IEEE/RSJ Int. Conf. on Intelligent Robots and Systems , 2010 . p.4560 - 4565 . DOI: 10.1109/IROS.2010.5650455 http://doi.org/10.1109/IROS.2010.5650455 .

C. Buehler , M. Bosse , L. McMillan , 等 . Unstructured lumigraph rendering . Proc. 28th Annual Conf. on Computer Graphics and Interactive Techniques , 2001 . p.425 - 432 . DOI: 10.1145/383259.383309 http://doi.org/10.1145/383259.383309 .

Canesta . Canestavision Electronic Perception Development Kit . Available from , 2006 . http://www.canesta.com/ http://www.canesta.com/ , .

Q. Chen , D. Li , C. Tang . KNN matting . Proc. IEEE Conf. on Computer Vision and Pattern Recognition , 2012 . p.869 - 876 . DOI: 10.1109/CVPR.2012.6247760 http://doi.org/10.1109/CVPR.2012.6247760 .

L. De-Maeztu , S. Mattoccia , A. Villanueva , 等 . Linear stereo matching . Proc. IEEE Int. Conf. on Computer Vision , 2011 . p.1708 - 1715 . DOI: 10.1109/ICCV.2011.6126434 http://doi.org/10.1109/ICCV.2011.6126434 .

J. Diebel , S. Thrun . An application of Markov random fields to range sensing . Adv. Neur. Inform. Process. Syst. , 2005 . 18 291 - 298 . .

V. Gandhiy , J. Cech , R. Horaud . High-resolution depth maps based on TOF-stereo fusion . IEEE Int. Conf. on Robotics and Automation , 2012 . p.4742 - 4749 . DOI: 10.1109/ICRA.2012.6224771 http://doi.org/10.1109/ICRA.2012.6224771 .

S.A. Gudmundsson , H. Aanaes , R. Larsen . Fusion of stereo vision and time-of-flight imaging for improved 3D estimation . Int. J. Intell. Syst. Technol. Appl. , 2008 . 5 ( 3-4 ): 425 - 433 . DOI: 10.1504/IJISTA.2008.021305 http://doi.org/10.1504/IJISTA.2008.021305 .

T. Kanade , M. Okutomi . A stereo matching algorithm with an adaptive window: theory and experiment . IEEE Trans. Patt. Anal. Mach. Intell. , 1994 . 16 ( 9 ): 920 - 932 . DOI: 10.1109/34.310690 http://doi.org/10.1109/34.310690 .

C. Lee , H. Song , B. Choi , 等 . 3D scene capturing using stereoscopic cameras and a time-of-flight camera . IEEE Trans. Consum. Electron. , 2011 . 57 ( 3 ): 1370 - 1376 . DOI: 10.1109/TCE.2011.6018896 http://doi.org/10.1109/TCE.2011.6018896 .

M. Lindner , A. Kolb , K. Hartmann . Data-fusion of PMD-based distance-information and high-resolution RGB-images . Proc. Int. Symp. on Signals, Circuits and Systems , 2007 . p.1 - 4 . DOI: 10.1109/ISSCS.2007.4292666 http://doi.org/10.1109/ISSCS.2007.4292666 .

S. May , B. Werner , H. Surmann , 等 . 3D timeof-flight cameras for mobile robotics . IEEE/RSJ Int. Conf. on Intelligent Robots and Systems , 2006 . p.790 - 795 . DOI: 10.1109/IROS.2006.281670 http://doi.org/10.1109/IROS.2006.281670 .

Opencv . Open Source Computer Vision Library (opencv) . Available from , 2012 . http://www.intel.com/technology/computing/opencv/ http://www.intel.com/technology/computing/opencv/ , .

PMD . Camcube Series . Available from , 2009 . http://www.pmdtec.com/ http://www.pmdtec.com/ , .

PMD . Camcube 3.0 Products . Available from , 2010 . http://www.pmdtec.com/products-services/pmdvisionr-cameras/pmdvisionr-camcube-30/ http://www.pmdtec.com/products-services/pmdvisionr-cameras/pmdvisionr-camcube-30/ , .

T. Ringbeck , B. Hagebeuker . A 3D time of flight camera for object detection . Proc. 8th Conf. on Optical 3-D Measurement Techniques , 2007 .

D. Scharstein , R. Szeliski . A taxonomy and evaluation of dense two-frame stereo correspondence algorithms . Int. J. Comput. Vis. , 2002a . 47 ( 1-3 ): 7 - 42 . DOI: 10.1023/A:1014573219977 http://doi.org/10.1023/A:1014573219977 .

D. Scharstein , R. Szeliski . Middlebury Stereo Evaluation - version 2 . Available from , 2002b . http://vision.middlebury.edu/stereo/eval http://vision.middlebury.edu/stereo/eval , .

L. Wang , L. Lou , C. Yang , 等 . Portrait drawing from corresponding range and intensity images . J. Zhejiang Univ.-Sci. C (Comput. & Electron.) , 2013 . 14 ( 7 ): 530 - 541 . DOI: 10.1631/jzus.CIDE1306 http://doi.org/10.1631/jzus.CIDE1306 .

Z. Xu , R. Schwarte , H. Heinol , 等 . Smart pixelphotonic mixer device (PMD) . Proc. 5th Int. Conf. on Mechatronics and Machine Vision in Practice , 1998 . p.259 - 264 . .

Q. Yang . A non-local cost aggregation method for stereo matching . Proc. IEEE Conf. on Computer Vision and Pattern Recognition , 2012 . p.1402 - 1409 . DOI: 10.1109/CVPR.2012.6247827 http://doi.org/10.1109/CVPR.2012.6247827 .

L. Yao , D. Li , J. Zhang , 等 . Accurate real-time stereo correspondence using intra- and inter-scanline optimization . J. Zhejiang Univ.-Sci. C (Comput. & Electron.) , 2012 . 13 ( 6 ): 472 - 482 . DOI: 10.1631/jzus.C1100311 http://doi.org/10.1631/jzus.C1100311 .

Z-cam . 3DV Systems . Available from , 2004 . http://www.3dvsystems.com http://www.3dvsystems.com , .

J. Zhang , D. Li , M. Zhang . Fast stereo matching algorithm based on adaptive window . Proc. Int. Conf. on Audio Language and Image Processing , 2010 . p.138 - 142 . DOI: 10.1109/ICALIP.2010.5684994 http://doi.org/10.1109/ICALIP.2010.5684994 .

Z. Zhang . Flexible camera calibration by viewing a plane from unknown orientations . Proc. 7th IEEE Int. Conf. on Computer Vision , 1999 . p.666 - 673 . DOI: 10.1109/ICCV.1999.791289 http://doi.org/10.1109/ICCV.1999.791289 .

J. Zhu , L. Wang , R. Yang , 等 . Fusion of time-offlight depth and stereo for high accuracy depth maps . Proc. IEEE Conf. on Computer Vision and Pattern Recognition , 2008 . p.1 - 8 . DOI: 10.1109/CVPR.2008.4587761 http://doi.org/10.1109/CVPR.2008.4587761 .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据