Accurate real-time stereo correspondence using intra- and inter-scanline optimization

Li YAO; Dong-xiao LI; Jing ZHANG; Liang-hao WANG; Ming ZHANG

doi:10.1631/jzus.C1100311

Your Location：

Home >

Browse articles >

Accurate real-time stereo correspondence using intra- and inter-scanline optimization

电子工程 | Updated：2022-12-09

- Accurate real-time stereo correspondence using intra- and inter-scanline optimization
- Accurate real-time stereo correspondence using intra- and inter-scanline optimization
- 信息与电子工程前沿（英文） 2012年13卷第6期页码：472-482
- Affiliations：
  
  1.Institute of Information and Communication Engineering, Zhejiang University, Hangzhou 310027, China
  2.Zhejiang Provincial Key Laboratory of Information Network Technology, Hangzhou 310027, China
- Author bio：
  
  [ "Li YAO, E-mail: zjyaoli@163.com" ]
  Dong-xiao LI, E-mail: lidx@zju.edu.cn
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 60802013 and 61072081), the National Science and Technology Major Project of the Ministry of Science and Technology of China (No. 2009ZX01033-001-007), and the China Postdoctoral Science Foundation (No. 20110491804)
- DOI：10.1631/jzus.C1100311
  中图分类号： TN911.73
- 收稿：2011-10-23，
  
  修回：2012-;5-;4，
  
  纸质出版：2012-06
- Accepted：
Scan QR Code
Accurate real-time stereo correspondence using intra- and inter-scanline optimization[J]. 信息与电子工程前沿（英文）, 2012,13(6):472-482.

Li YAO, Dong-xiao LI, Jing ZHANG, et al. Accurate real-time stereo correspondence using intra- and inter-scanline optimization[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(6): 472-482.
Accurate real-time stereo correspondence using intra- and inter-scanline optimization[J]. 信息与电子工程前沿（英文）, 2012,13(6):472-482. DOI： 10.1631/jzus.C1100311.

Li YAO, Dong-xiao LI, Jing ZHANG, et al. Accurate real-time stereo correspondence using intra- and inter-scanline optimization[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(6): 472-482. DOI： 10.1631/jzus.C1100311.

摘要

Abstract

This paper deals with a novel stereo algorithm that can generate accurate dense disparity maps in real time. The algorithm employs an effective cross-based variable support aggregation strategy within a scanline optimization framework. Rather than matching intensities directly

the use of adaptive support aggregation allows for precisely handling the weak textured regions as well as depth discontinuities. To improve the disparity results with global reasoning

we reformulate the energy function on a tree structure over the whole 2D image area

as opposed to dynamic programming of individual scanlines. By applying both intra- and inter-scanline optimizations

the algorithm reduces the typical 'streaking' artifact while maintaining high computational efficiency. The experimental results are evaluated on the Middlebury stereo dataset

showing that our approach is among the best for all real-time approaches. We implement the algorithm on a commodity graphics card with CUDA architecture

running at about 35 fames/s for a typical stereo pair with a resolution of 384×288 and 16 disparity levels.

关键词

Keywords

references

A.F. Bobick , , , S.S. Intille . . Large occlusion stereo . . Int. J. Comput. Vis. , , 1999 . . 33 ( ( 3 ): ): 181 - - 200 . . DOI: 10.1023/A:1008150329890 http://doi.org/10.1023/A:1008150329890 . .

A. Criminisi , , , A. Blake , , , C. Rother , , , J. Shotton , , , P.H.S. Torr . . Efficient dense stereo with occlusions for new view-synthesis by four-state dynamic programming . . Int. J. Comput. Vis. , , 2007 . . 71 ( ( 1 ): ): 89 - - 110 . . DOI: 10.1007/s11263-006-8525-1 http://doi.org/10.1007/s11263-006-8525-1 . .

F.C. Crow . . Summed-Area Tables for Texture Mapping. Proc. 11th Annual Conf . . on Computer Graphics and Interactive Techniques , , 1984 . . 207 - - 212 . . DOI: 10.1145/800031.808600 http://doi.org/10.1145/800031.808600 . .

A. Darabiha , , , J. Rose , , , W.J. MacLean . . Video-Rate Stereo Depth Measurement on Programmable Hardware . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2003 . . p.203 - - 210 . . DOI: 10.1109/CVPR.2003.1211355 http://doi.org/10.1109/CVPR.2003.1211355 . .

M. Gong , , , Y. Yang . . Near Real-Time Reliable Stereo Matching Using Programmable Graphics Hardware . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2005 . . 924 - - 931 . . DOI: 10.1109/CVPR.2005.246 http://doi.org/10.1109/CVPR.2005.246 . .

M. Gong , , , Y. Yang . . Real-time stereo matching using orthogonal reliability-based dynamic programming . . IEEE Trans. Image Process. , , 2007 . . 16 ( ( 3 ): ): 879 - - 884 . . DOI: 10.1109/TIP.2006.891344]MathSciNet http://doi.org/10.1109/TIP.2006.891344]MathSciNet . .

R.K. Gupta , , , S.Y. Cho . . Real-Time Stereo Matching Using Adaptive Binary Window . . Proc. 3DPVT, Session 2 , , 2010 . . p.1 - - 8 . . . .

R.I. Hartley , , , A. Zisserman . . Multiple View Geometry . . Cambridge University Press, Cambridge, UK , , 2004 . . .

H. Hirschmüller , , , P. Innocent , , , J. Garibaldi . . Real-time correlation-based stereo vision with reduced border errors . . Int. J. Comput. Vis. , , 2002 . . 47 ( ( 1-3 ): ): 229 - - 246 . . DOI: 10.1023/A:1014554110407 http://doi.org/10.1023/A:1014554110407 . .

ISO/IEC . . HHI Test Material for 3D Video . . M15413 , , 2008 . . .

J.C. Kim , , , K.M. Lee , , , B.T. Choi , , , S.U. Lee . . A Dense Stereo Matching Using Two-Pass Dynamic Programming with Generalized Ground Control Points . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2005 . . p.1075 - - 1082 . . DOI: 10.1109/CVPR.2005.22 http://doi.org/10.1109/CVPR.2005.22 . .

K. Konolige . . Small Vision System: Hardware and Implementation . . Proc. Int. Symp. on Robotics Research , , 1997 . . p.111 - - 116 . . . .

S. Kosov , , , T. Thormählen , , , H. Seidel . . Accurate real-time disparity estimation with variational methods . . LNCS , , 2009 . . 5875 796 - - 807 . . DOI: 10.1007/978-3-642-10331-5_74 http://doi.org/10.1007/978-3-642-10331-5_74 . .

C. Lei , , , J. Selzer , , , Y. Yang . . Region-Tree Based Stereo Using Dynamic Programming Optimization . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2006 . . 2378 - - 2385 . . DOI: 10.1109/CVPR.2006.251 http://doi.org/10.1109/CVPR.2006.251 . .

S. Mattoccia , , , F. Tombari , , , L. di Stefano . . Stereo vision enabling precise border localization within a scanline optimization framework . . LNCS , , 2007 . . 4844 517 - - 527 . . DOI: 10.1007/978-3-540-76390-1_51 http://doi.org/10.1007/978-3-540-76390-1_51 . .

M.J. McDonnell . . Box-filtering techniques . . Comput. Graph. Image Process. , , 1981 . . 17 ( ( 1 ): ): 65 - - 70 . . DOI: 10.1016/S0146-664X(81)80009-3 http://doi.org/10.1016/S0146-664X(81)80009-3 . .

C.C. Moallemi , , , B. van Roy . . Convergence of min-sum message-passing for convex optimization . . IEEE Trans. Inform. Theory , , 2010 . . 56 ( ( 4 ): ): 2041 - - 2050 . . DOI: 10.1109/TIT.2010.2040863]MathSciNet http://doi.org/10.1109/TIT.2010.2040863]MathSciNet . .

J. Mulligan , , , V. Isler , , , K. Daniilidis . . Trinocular stereo: a real-time algorithm and its evaluation . . Int. J. Comput. Vis. , , 2002 . . 47 ( ( 1-3 ): ): 51 - - 61 . . DOI: 10.1023/A:1014525320885 http://doi.org/10.1023/A:1014525320885 . .

J. Salmen , , , M. Schlipsing , , , J. Edelbrunner , , , S. Hegemann , , , S. Lueke . . Real-time stereo vision: making more out of dynamic programming . . LNCS , , 2009 . . 5702 1096 - - 1103 . . DOI: 10.1007/978-3-642-03767-2_133 http://doi.org/10.1007/978-3-642-03767-2_133 . .

R. Sara . . How to Teach Stereoscopic Matching . . Proc. 52nd Int. Symp. ELMAR , , 2010 . . p.445 - - 453 . . . .

D. Scharstein , , , R. Szeliski . . Middlebury Stereo Vision Page . . Available from , , http://vision.middlebury.edu/stereo/ http://vision.middlebury.edu/stereo/ , , [Accessed on Oct. 20, 2011] 2001 . . .

D. Scharstein , , , R. Szeliski . . A taxonomy and evaluation of dense two-frame stereo correspondence algorithms . . Int. J. Comput. Vis. , , 2002 . . 47 ( ( 1 ): ): 7 - - 42 . . DOI: 10.1023/A:1014573219977 http://doi.org/10.1023/A:1014573219977 . .

F. Tombari , , , S. Mattoccia , , , L. di Stefano , , , E. Addimanda . . Classification and Evaluation of Cost Aggregation Methods for Stereo Correspondence . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2008a . . p.1 - - 8 . . DOI: 10.1109/CVPR.2008.4587677 http://doi.org/10.1109/CVPR.2008.4587677 . .

F. Tombari , , , S. Mattoccia , , , L. di Stefano , , , E. Addimanda . . Near Real-Time Stereo Based on Effective Cost Aggregation . . Int. Conf. on Pattern Recognition , , 2008b . . p.1 - - 4 . . DOI: 10.1109/ICPR.2008.4761024 http://doi.org/10.1109/ICPR.2008.4761024 . .

O. Veksler . . Efficient Graph-Based Energy Minimization Methods in Computer Vision . . PhD Thesis, Cornell University, New York, USA , , 1999 . . .

O. Veksler . . Fast Variable Window for Stereo Correspondence Using Integral Images . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2003 . . p.556 - - 561 . . DOI: 10.1109/CVPR.2003.1211403 http://doi.org/10.1109/CVPR.2003.1211403 . .

O. Veksler . . Stereo Correspondence by Dynamic Programming on a Tree . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2005 . . p.384 - - 390 . . DOI: 10.1109/CVPR.2005.334 http://doi.org/10.1109/CVPR.2005.334 . .

L. Wang , , , M. Liao , , , M. Gong , , , R. Yang , , , D. Nister . . High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming . . 3rd Int. Symp. on 3DPVT , , 2006 . . p.798 - - 805 . . DOI: 10.1109/3DPVT.2006.75 http://doi.org/10.1109/3DPVT.2006.75 . .

Y. Weiss , , , W.T. Freeman . . On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs . . IEEE Trans. Inform. Theory , , 2001 . . 47 ( ( 2 ): ): 736 - - 744 . . DOI: 10.1109/18.910585]MathSciNet http://doi.org/10.1109/18.910585]MathSciNet . .

J. Woodfill , , , B.V. Herzen . . Real-Time Stereo Vision on the PARTS Reconfigurable Computer . . Proc. IEEE Symp. on FPGAs for Custom Computing Machines , , 1997 . . p.201 - - 210 . . DOI: 10.1109/FPGA.1997.624620 http://doi.org/10.1109/FPGA.1997.624620 . .

Q. Yang , , , L. Wang , , , R. Yang , , , S. Wang , , , M. Liao , , , D. Nister . . Real-Time Global Stereo Matching Using Hierarchical Belief Propagation . . The British Machine Vision Conf. , , 2006 . . p.989 - - 998 . . . .

R. Yang , , , G. Welch , , , G. Bishop . . Real-Time Consensus-Based Scene Reconstruction Using Commodity Graphics Hardware . . Proc. 10th Pacific Conf. on Computer Graphics and Applications , , 2002 . . p.225 - - 234 . . DOI: 10.1109/PCCGA.2002.1167864 http://doi.org/10.1109/PCCGA.2002.1167864 . .

K. Yoon , , , I. Kweon . . Adaptive support-weight approach for correspondence search . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2006 . . 28 ( ( 4 ): ): 650 - - 656 . . DOI: 10.1109/TPAMI.2006.70 http://doi.org/10.1109/TPAMI.2006.70 . .

K. Zhang , , , J. Lu , , , G. Lafruit . . Cross-based local stereo matching using orthogonal integral images . . IEEE Trans. Circ. Syst. Video Technol. , , 2009 . . 19 ( ( 7 ): ): 1073 - - 1079 . . DOI: 10.1109/TCSVT.2009.2020478 http://doi.org/10.1109/TCSVT.2009.2020478 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

A hybrid genetic algorithm to optimize device allocation in industrial Ethernet networks with real-time constraints