Accurate real-time stereo correspondence using intra- and inter-scanline optimization

Li YAO; Dong-xiao LI; Jing ZHANG; Liang-hao WANG; Ming ZHANG

doi:10.1631/jzus.C1100311

Your Location：

Home >

Browse articles >

Accurate real-time stereo correspondence using intra- and inter-scanline optimization

Electronic Engineering | Updated：2022-12-09

- Accurate real-time stereo correspondence using intra- and inter-scanline optimization
- Accurate real-time stereo correspondence using intra- and inter-scanline optimization
- Frontiers of Information Technology & Electronic Engineering Vol. 13, Issue 6, Pages: 472-482(2012)
- Affiliations：
  
  1.Institute of Information and Communication Engineering, Zhejiang University, Hangzhou 310027, China
  2.Zhejiang Provincial Key Laboratory of Information Network Technology, Hangzhou 310027, China
- Author bio：
  
  [ "Li YAO, E-mail: zjyaoli@163.com" ]
  Dong-xiao LI, E-mail: lidx@zju.edu.cn
- Funds：
- DOI：10.1631/jzus.C1100311
  CLC： TN911.73
- Received：23 October 2011，
  
  Revised：2012-;5-;4，
  
  Published：2012-06
- Accepted：
Scan QR Code
Li YAO, Dong-xiao LI, Jing ZHANG, et al. Accurate real-time stereo correspondence using intra- and inter-scanline optimization[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(6): 472-482.
DOI：

Li YAO, Dong-xiao LI, Jing ZHANG, et al. Accurate real-time stereo correspondence using intra- and inter-scanline optimization[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(6): 472-482. DOI： 10.1631/jzus.C1100311.

摘要

Abstract

This paper deals with a novel stereo algorithm that can generate accurate dense disparity maps in real time. The algorithm employs an effective cross-based variable support aggregation strategy within a scanline optimization framework. Rather than matching intensities directly

the use of adaptive support aggregation allows for precisely handling the weak textured regions as well as depth discontinuities. To improve the disparity results with global reasoning

we reformulate the energy function on a tree structure over the whole 2D image area

as opposed to dynamic programming of individual scanlines. By applying both intra- and inter-scanline optimizations

the algorithm reduces the typical 'streaking' artifact while maintaining high computational efficiency. The experimental results are evaluated on the Middlebury stereo dataset

showing that our approach is among the best for all real-time approaches. We implement the algorithm on a commodity graphics card with CUDA architecture

running at about 35 fames/s for a typical stereo pair with a resolution of 384×288 and 16 disparity levels.

关键词

Keywords

references

A.F. Bobick , , , S.S. Intille . . Large occlusion stereo . . Int. J. Comput. Vis. , , 1999 . . 33 ( ( 3 ): ): 181 - - 200 . . DOI: 10.1023/A:1008150329890 http://doi.org/10.1023/A:1008150329890 . .

A. Criminisi , , , A. Blake , , , C. Rother , , , J. Shotton , , , P.H.S. Torr . . Efficient dense stereo with occlusions for new view-synthesis by four-state dynamic programming . . Int. J. Comput. Vis. , , 2007 . . 71 ( ( 1 ): ): 89 - - 110 . . DOI: 10.1007/s11263-006-8525-1 http://doi.org/10.1007/s11263-006-8525-1 . .

F.C. Crow . . Summed-Area Tables for Texture Mapping. Proc. 11th Annual Conf . . on Computer Graphics and Interactive Techniques , , 1984 . . 207 - - 212 . . DOI: 10.1145/800031.808600 http://doi.org/10.1145/800031.808600 . .

A. Darabiha , , , J. Rose , , , W.J. MacLean . . Video-Rate Stereo Depth Measurement on Programmable Hardware . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2003 . . p.203 - - 210 . . DOI: 10.1109/CVPR.2003.1211355 http://doi.org/10.1109/CVPR.2003.1211355 . .

M. Gong , , , Y. Yang . . Near Real-Time Reliable Stereo Matching Using Programmable Graphics Hardware . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2005 . . 924 - - 931 . . DOI: 10.1109/CVPR.2005.246 http://doi.org/10.1109/CVPR.2005.246 . .

M. Gong , , , Y. Yang . . Real-time stereo matching using orthogonal reliability-based dynamic programming . . IEEE Trans. Image Process. , , 2007 . . 16 ( ( 3 ): ): 879 - - 884 . . DOI: 10.1109/TIP.2006.891344]MathSciNet http://doi.org/10.1109/TIP.2006.891344]MathSciNet . .

R.K. Gupta , , , S.Y. Cho . . Real-Time Stereo Matching Using Adaptive Binary Window . . Proc. 3DPVT, Session 2 , , 2010 . . p.1 - - 8 . . . .

R.I. Hartley , , , A. Zisserman . . Multiple View Geometry . . Cambridge University Press, Cambridge, UK , , 2004 . . .

H. Hirschmüller , , , P. Innocent , , , J. Garibaldi . . Real-time correlation-based stereo vision with reduced border errors . . Int. J. Comput. Vis. , , 2002 . . 47 ( ( 1-3 ): ): 229 - - 246 . . DOI: 10.1023/A:1014554110407 http://doi.org/10.1023/A:1014554110407 . .

ISO/IEC . . HHI Test Material for 3D Video . . M15413 , , 2008 . . .

J.C. Kim , , , K.M. Lee , , , B.T. Choi , , , S.U. Lee . . A Dense Stereo Matching Using Two-Pass Dynamic Programming with Generalized Ground Control Points . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2005 . . p.1075 - - 1082 . . DOI: 10.1109/CVPR.2005.22 http://doi.org/10.1109/CVPR.2005.22 . .

K. Konolige . . Small Vision System: Hardware and Implementation . . Proc. Int. Symp. on Robotics Research , , 1997 . . p.111 - - 116 . . . .

S. Kosov , , , T. Thormählen , , , H. Seidel . . Accurate real-time disparity estimation with variational methods . . LNCS , , 2009 . . 5875 796 - - 807 . . DOI: 10.1007/978-3-642-10331-5_74 http://doi.org/10.1007/978-3-642-10331-5_74 . .

C. Lei , , , J. Selzer , , , Y. Yang . . Region-Tree Based Stereo Using Dynamic Programming Optimization . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2006 . . 2378 - - 2385 . . DOI: 10.1109/CVPR.2006.251 http://doi.org/10.1109/CVPR.2006.251 . .

S. Mattoccia , , , F. Tombari , , , L. di Stefano . . Stereo vision enabling precise border localization within a scanline optimization framework . . LNCS , , 2007 . . 4844 517 - - 527 . . DOI: 10.1007/978-3-540-76390-1_51 http://doi.org/10.1007/978-3-540-76390-1_51 . .

M.J. McDonnell . . Box-filtering techniques . . Comput. Graph. Image Process. , , 1981 . . 17 ( ( 1 ): ): 65 - - 70 . . DOI: 10.1016/S0146-664X(81)80009-3 http://doi.org/10.1016/S0146-664X(81)80009-3 . .

C.C. Moallemi , , , B. van Roy . . Convergence of min-sum message-passing for convex optimization . . IEEE Trans. Inform. Theory , , 2010 . . 56 ( ( 4 ): ): 2041 - - 2050 . . DOI: 10.1109/TIT.2010.2040863]MathSciNet http://doi.org/10.1109/TIT.2010.2040863]MathSciNet . .

J. Mulligan , , , V. Isler , , , K. Daniilidis . . Trinocular stereo: a real-time algorithm and its evaluation . . Int. J. Comput. Vis. , , 2002 . . 47 ( ( 1-3 ): ): 51 - - 61 . . DOI: 10.1023/A:1014525320885 http://doi.org/10.1023/A:1014525320885 . .

J. Salmen , , , M. Schlipsing , , , J. Edelbrunner , , , S. Hegemann , , , S. Lueke . . Real-time stereo vision: making more out of dynamic programming . . LNCS , , 2009 . . 5702 1096 - - 1103 . . DOI: 10.1007/978-3-642-03767-2_133 http://doi.org/10.1007/978-3-642-03767-2_133 . .

R. Sara . . How to Teach Stereoscopic Matching . . Proc. 52nd Int. Symp. ELMAR , , 2010 . . p.445 - - 453 . . . .

D. Scharstein , , , R. Szeliski . . Middlebury Stereo Vision Page . . Available from , , http://vision.middlebury.edu/stereo/ http://vision.middlebury.edu/stereo/ , , [Accessed on Oct. 20, 2011] 2001 . . .

D. Scharstein , , , R. Szeliski . . A taxonomy and evaluation of dense two-frame stereo correspondence algorithms . . Int. J. Comput. Vis. , , 2002 . . 47 ( ( 1 ): ): 7 - - 42 . . DOI: 10.1023/A:1014573219977 http://doi.org/10.1023/A:1014573219977 . .

F. Tombari , , , S. Mattoccia , , , L. di Stefano , , , E. Addimanda . . Classification and Evaluation of Cost Aggregation Methods for Stereo Correspondence . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2008a . . p.1 - - 8 . . DOI: 10.1109/CVPR.2008.4587677 http://doi.org/10.1109/CVPR.2008.4587677 . .

F. Tombari , , , S. Mattoccia , , , L. di Stefano , , , E. Addimanda . . Near Real-Time Stereo Based on Effective Cost Aggregation . . Int. Conf. on Pattern Recognition , , 2008b . . p.1 - - 4 . . DOI: 10.1109/ICPR.2008.4761024 http://doi.org/10.1109/ICPR.2008.4761024 . .

O. Veksler . . Efficient Graph-Based Energy Minimization Methods in Computer Vision . . PhD Thesis, Cornell University, New York, USA , , 1999 . . .

O. Veksler . . Fast Variable Window for Stereo Correspondence Using Integral Images . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2003 . . p.556 - - 561 . . DOI: 10.1109/CVPR.2003.1211403 http://doi.org/10.1109/CVPR.2003.1211403 . .

O. Veksler . . Stereo Correspondence by Dynamic Programming on a Tree . . IEEE Conf. on Computer Vision and Pattern Recognition , , 2005 . . p.384 - - 390 . . DOI: 10.1109/CVPR.2005.334 http://doi.org/10.1109/CVPR.2005.334 . .

L. Wang , , , M. Liao , , , M. Gong , , , R. Yang , , , D. Nister . . High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming . . 3rd Int. Symp. on 3DPVT , , 2006 . . p.798 - - 805 . . DOI: 10.1109/3DPVT.2006.75 http://doi.org/10.1109/3DPVT.2006.75 . .

Y. Weiss , , , W.T. Freeman . . On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs . . IEEE Trans. Inform. Theory , , 2001 . . 47 ( ( 2 ): ): 736 - - 744 . . DOI: 10.1109/18.910585]MathSciNet http://doi.org/10.1109/18.910585]MathSciNet . .

J. Woodfill , , , B.V. Herzen . . Real-Time Stereo Vision on the PARTS Reconfigurable Computer . . Proc. IEEE Symp. on FPGAs for Custom Computing Machines , , 1997 . . p.201 - - 210 . . DOI: 10.1109/FPGA.1997.624620 http://doi.org/10.1109/FPGA.1997.624620 . .

Q. Yang , , , L. Wang , , , R. Yang , , , S. Wang , , , M. Liao , , , D. Nister . . Real-Time Global Stereo Matching Using Hierarchical Belief Propagation . . The British Machine Vision Conf. , , 2006 . . p.989 - - 998 . . . .

R. Yang , , , G. Welch , , , G. Bishop . . Real-Time Consensus-Based Scene Reconstruction Using Commodity Graphics Hardware . . Proc. 10th Pacific Conf. on Computer Graphics and Applications , , 2002 . . p.225 - - 234 . . DOI: 10.1109/PCCGA.2002.1167864 http://doi.org/10.1109/PCCGA.2002.1167864 . .

K. Yoon , , , I. Kweon . . Adaptive support-weight approach for correspondence search . . IEEE Trans. Pattern Anal. Mach. Intell. , , 2006 . . 28 ( ( 4 ): ): 650 - - 656 . . DOI: 10.1109/TPAMI.2006.70 http://doi.org/10.1109/TPAMI.2006.70 . .

K. Zhang , , , J. Lu , , , G. Lafruit . . Cross-based local stereo matching using orthogonal integral images . . IEEE Trans. Circ. Syst. Video Technol. , , 2009 . . 19 ( ( 7 ): ): 1073 - - 1079 . . DOI: 10.1109/TCSVT.2009.2020478 http://doi.org/10.1109/TCSVT.2009.2020478 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

A hybrid genetic algorithm to optimize device allocation in industrial Ethernet networks with real-time constraints

Related Author

Lei ZHANG

Mattias LAMPE

Zhi WANG

Liwen Liu

Weidong Yang

Ben Fei

Related Institution

State Key Laboratory of Industrial Control Technology, Zhejiang University

Corporate Technology, Siemens Ltd.

Shanghai Key Laboratory of Data Science, the School of Computer Science, Fudan University, Shanghai, 200433, China Zhuhai Fudan Innovation Institute, Hengqin New Area

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰