无人车自主定位和障碍物感知的视觉主导多传感器融合方法

薛建儒; 王迪; 杜少毅; 崔迪潇; 黄勇; 郑南宁

doi:10.1631/FITEE.1601873

Your Location：

Home >

Browse articles >

无人车自主定位和障碍物感知的视觉主导多传感器融合方法

人工智能2.0专辑 | Updated：2022-05-19

- 无人车自主定位和障碍物感知的视觉主导多传感器融合方法
- A vision-centered multi-sensor fusing approach to self-localization and obstacle perception for robotic cars
- 信息与电子工程前沿（英文） 2017年18卷第1期页码：122-138
- Affiliations：
  
  Lab of Visual Cognitive Computing and Intelligent Vehicle, Xi'an Jiaotong University, Xi'an 710049, China
- Author bio：
  
  E-mail:jrxue@xjtu.edu.cn
- Funds：
  
  Project supported by the National Key Program Project of China (No. 2016YFB1001004) and the National Natural Science Foundation of China (Nos. 91320301 and 61273252)
- DOI：10.1631/FITEE.1601873
  中图分类号： TP181
- 收稿：2016-12-29，
  
  录用：2017-1-8，
  
  纸质出版：2017-01
- Accepted：
Scan QR Code
薛建儒, 王迪, 杜少毅, 等. 无人车自主定位和障碍物感知的视觉主导多传感器融合方法[J]. 信息与电子工程前沿（英文）, 2017,18(1):122-138.

Jian-ru XUE, Di WANG, Shao-yi DU, et al. A vision-centered multi-sensor fusing approach to self-localization and obstacle perception for robotic cars[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(1): 122-138.
薛建儒, 王迪, 杜少毅, 等. 无人车自主定位和障碍物感知的视觉主导多传感器融合方法[J]. 信息与电子工程前沿（英文）, 2017,18(1):122-138. DOI： 10.1631/FITEE.1601873.

Jian-ru XUE, Di WANG, Shao-yi DU, et al. A vision-centered multi-sensor fusing approach to self-localization and obstacle perception for robotic cars[J]. Frontiers of Information Technology & Electronic Engineering, 2017, 18(1): 122-138. DOI： 10.1631/FITEE.1601873.

摘要

人类驾驶与自主驾驶在对交通环境的理解方式上有着明显差别。首先，人主要通过视觉来理解交通场景，而机器感知需要融合多种异构的传感信息才能保证行车安全。其次，一个熟练的驾驶员可以轻松适应各种动态交通环境，但现有的机器感知系统却会经常输出有噪声的感知结果，而自主驾驶要求感知结果近乎100%准确。本文提出了一种用于无人车交通环境感知的视觉主导的多传感器融合计算框架，通过几何和语义约束融合来自相机、激光雷达（LIDAR）及地理信息系统（GIS）的信息，为无人车提供高精度的自主定位和准确鲁棒的障碍物感知，并进一步讨论了已成功集成到上述框架内的鲁棒的视觉算法，主要包括从训练数据收集、传感器数据处理、低级特征提取到障碍物识别和环境地图创建等多个层次的视觉算法。所提出的框架里已用于自主研发的无人车，并在各种真实城区环境中进行了长达八年的实地测试，实验结果验证了视觉主导的多传感融合感知框架的鲁棒性和高效性。

Abstract

Most state-of-the-art robotic cars' perception systems are quite different from the way a human driver understands traffic environments. First

humans assimilate information from the traffic scene mainly through visual perception

while the machine perception of traffic environments needs to fuse information from several different kinds of sensors to meet safety-critical requirements. Second

a robotic car requires nearly 100% correct perception results for its autonomous driving

while an experienced human driver works well with dynamic traffic environments

in which machine perception could easily produce noisy perception results. In this paper

we propose a vision-centered multi-sensor fusing framework for a traffic environment perception approach to autonomous driving

which fuses camera

LIDAR

and GIS information consistently via both geometrical and semantic constraints for efficient self-localization and obstacle perception. We also discuss robust machine vision algorithms that have been successfully integrated with the framework and address multiple levels of machine vision techniques

from collecting training data

efficiently processing sensor data

and extracting low-level features

to higher-level object and environment mapping. The proposed framework has been tested extensively in actual urban scenes with our self-developed robotic cars for eight years. The empirical results validate its robustness and efficiency.

关键词

Keywords

references

M Aeberhard , , , S Rauch , , , M Bahram , , , 等 . . Experience, results and lessons learned from automated driving on Germany's highways . . IEEE Intell. Transp. Syst. Mag. , , 2015 . . 7 ( ( 1 ): ): 42 - - 57 . . DOI: 10.1109/MITS.2014.2360306 http://doi.org/10.1109/MITS.2014.2360306 . .

JL Blanco , , , JA Fern dez-Madrigal , , , J Gonz lez . . A new approach for large-scale localization and mapping: hybrid metric-topological SLAM . . 2007 . . Proc. IEEE Int. Conf. on Robotics and Automation . . 2061 - - 2067 . . DOI: 10.1109/ROBOT.2007.363625 http://doi.org/10.1109/ROBOT.2007.363625 . .

JL Blanco , , , JA Fern dez-Madrigal , , , J Gonz lez . . Toward a unified Bayesian approach to hybrid metrictopological SLAM . . IEEE Trans. Robot. , , 2008 . . 24 ( ( 2 ): ): 259 - - 270 . . DOI: 10.1109/TRO.2008.918049 http://doi.org/10.1109/TRO.2008.918049 . .

MA Brubaker , , , A Geiger , , , R Urtasun . . Map-based probabilistic visual self-localization . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2016 . . 38 ( ( 4 ): ): 652 - - 665 . . DOI: 10.1109/TPAMI.2015.2453975 http://doi.org/10.1109/TPAMI.2015.2453975 . .

M Buehler , , , K Iagnemma , , , S Singh . . The DARPA Urban Challenge: Autonomous Vehicles in City Traffic , , 2009 . . : Springer : DOI: 10.1007/978-3-642-03991-1 http://doi.org/10.1007/978-3-642-03991-1 . .

H Cho , , , YW Seo , , , B.V.K.V Kumar , , , 等 . . A multi-sensor fusion system for moving object detection and tracking in urban driving environments . . 2014 . . IEEE Int. Conf. on Robotics and Automation . . 1836 - - 1843 . . DOI: 10.1109/ICRA.2014.6907100 http://doi.org/10.1109/ICRA.2014.6907100 . .

D Cui , , , J Xue , , , S Du , , , 等 . . Real-time global localization of intelligent road vehicles in lane-level via lane marking detection and shape registration . . 2014 . . IEEE/RSJ Int. Conf. on Intelligent Robots and Systems . . 4958 - - 4964 . . DOI: 10.1109/IROS.2014.6943267 http://doi.org/10.1109/IROS.2014.6943267 . .

DX Cui , , , JR Xue , , , NN Zheng . . Real-time global localization of robotic cars in lane level via lane marking detection and shape registration . . IEEE Trans. Intell. Transp. Syst. , , 2016 . . 17 ( ( 4 ): ): 1039 - - 1050 . . DOI: 10.1109/TITS.2015.2492019 http://doi.org/10.1109/TITS.2015.2492019 . .

M Darms , , , P Rybski , , , C Urmson . . Classification and tracking of dynamic objects with multiple sensors for autonomous driving in urban environments . . 2008 . . IEEE Intelligent Vehicles Symp . . 1197 - - 1202 . . DOI: 10.1109/IVS.2008.4621259 http://doi.org/10.1109/IVS.2008.4621259 . .

M Darms , , , P Rybski , , , C Baker , , , 等 . . Obstacle detection and tracking for the urban challenge . . IEEE Trans. Intell. Transp. Syst. , , 2009 . . 10 ( ( 3 ): ): 475 - - 485 . . DOI: 10.1109/TITS.2009.2018319 http://doi.org/10.1109/TITS.2009.2018319 . .

A J Davison , , , I D Reid , , , N D Molton , , , 等 . . MonoSLAM: real-time single camera SLAM . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2007 . . 29 ( ( 6 ): ): 1052 - - 1067 . . DOI: 10.1109/TPAMI.2007.1049 http://doi.org/10.1109/TPAMI.2007.1049 . .

M.W.M.G Dissanayake , , , P Newman , , , S Clark , , , 等 . . A solution to the simultaneous localization and map building (SLAM) problem . . IEEE Trans. Robot. Autom. , , 2001 . . 17 ( ( 3 ): ): 229 - - 241 . . DOI: 10.1109/70.938381 http://doi.org/10.1109/70.938381 . .

P Dollr , , , R Appel , , , S Belongie , , , 等 . . Fast feature pyramids for object detection . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2014 . . 36 ( ( 8 ): ): 1532 - - 1545 . . DOI: 10.1109/TPAMI.2014.2300479 http://doi.org/10.1109/TPAMI.2014.2300479 . .

B Douillard , , , D Fox , , , F Ramos . . Laser and vision based outdoor object mapping . . 2009 . . Robotics: Science and Systems Ⅳ . . 9 - - 16 . . . .

SY Du , , , NN Zheng , , , L Xiong , , , 等 . . Scaling iterative closest point algorithm for registration of m-D point sets . . J. Vis. Commun. Image Represent. , , 2010 . . 21 ( ( 5-6 ): ): 442 - - 452 . . DOI: 10.1016/j.jvcir.2010.02.005 http://doi.org/10.1016/j.jvcir.2010.02.005 . .

H Durrant-Whyte , , , T Bailey . . Simultaneous localization and mapping: part I . . IEEE Robot. Autom. Mag. , , 2006 . . 13 ( ( 2 ): ): 99 - - 110 . . DOI: 10.1109/MRA.2006.1638022 http://doi.org/10.1109/MRA.2006.1638022 . .

A Ess , , , K Schindler , , , B Leibe , , , 等 . . Object detection and tracking for autonomous navigation in dynamic environments . . Int. J. Robot. Res. , , 2010 . . 29 ( ( 14 ): ): 1707 - - 1725 . . DOI: 10.1177/0278364910365417 http://doi.org/10.1177/0278364910365417 . .

J Fuentes-Pacheco , , , J Ruiz-Ascencio , , , JM Rend n-Mancha . . Visual simultaneous localization and mapping: a survey . . Artif. Intell. Rev. , , 2015 . . 43 ( ( 1 ): ): 55 - - 81 . . DOI: 10.1007/s10462-012-9365-8 http://doi.org/10.1007/s10462-012-9365-8 . .

G Grisetti , , , R Kummerle , , , C Stachniss , , , 等 . . A tutorial on graph-based SLAM . . IEEE Intell. Transp. Syst. Mag. , , 2010 . . 2 ( ( 4 ): ): 31 - - 43 . . DOI: 10.1109/MITS.2010.939925 http://doi.org/10.1109/MITS.2010.939925 . .

RI Hartley , , , A Zisserman . . Multiple View Geometry in Computer Vision , , 2004 . . : Cambridge University Press : . .

D Held , , , J Levinson , , , S Thrun , , , 等 . . Robust realtime tracking combining 3D shape, color, and motion . . Int. J. Robot. Res. , , 2016 . . 35 ( ( 1-3 ): ): 30 - - 49 . . DOI: 10.1177/0278364915593399 http://doi.org/10.1177/0278364915593399 . .

AB Hillel , , , R Lerner , , , D Levi , , , 等 . . Recent progress in road and lane detection: a survey . . Mach. Vis. Appl. , , 2014 . . 25 ( ( 3 ): ): 727 - - 745 . . DOI: 10.1007/s00138-011-0404-2 http://doi.org/10.1007/s00138-011-0404-2 . .

D Hoiem , , , J Hays , , , JX Xiao , , , 等 . . Guest editorial: scene understanding . . Int. J. Comput. Vis. , , 2015 . . 112 ( ( 2 ): ): 131 - - 132 . . DOI: 10.1007/s11263-015-0807-z http://doi.org/10.1007/s11263-015-0807-z . .

K Konolige , , , E Marder-Eppstein , , , B Marthi . . Navigation in hybrid metric-topological maps . . 2011 . . IEEE Int. Conf. on Robotics and Automation . . 3041 - - 3047 . . DOI: 10.1109/ICRA.2011.5980074 http://doi.org/10.1109/ICRA.2011.5980074 . .

Q Li , , , NN Zheng , , , H Cheng . . Springrobot: a prototype autonomous vehicle and its algorithms for lane detection . . IEEE Trans. Intell. Transp. Syst. , , 2004 . . 5 ( ( 4 ): ): 300 - - 308 . . DOI: 10.1109/TITS.2004.838220 http://doi.org/10.1109/TITS.2004.838220 . .

C Mertz , , , LE Navarro-Serment , , , RA MacLachlan , , , 等 . . Moving object detection with laser scanners . . J. Field Robot. , , 2013 . . 30 ( ( 1 ): ): 17 - - 43 . . DOI: 10.1002/rob.21430 http://doi.org/10.1002/rob.21430 . .

M Montemerlo , , , S Thrun , , , D Koller , , , 等 . . FastSLAM: a factored solution to the simultaneous localization and mapping problem . . 2002 . . 8th National Conf. on Artificial Intelligence . . 593 - - 598 . . . .

YH Pan . . Heading toward artificial intelligence 2.0 . . Engineering , , 2016 . . 2 ( ( 4 ): ): 409 - - 413 . . DOI: 10.1016/J.ENG.2016.04.018 http://doi.org/10.1016/J.ENG.2016.04.018 . .

K Schueler , , , T Weiherer , , , E Bouzouraa , , , 等 . . 360 degree multi sensor fusion for static and dynamic obstacles . . 2012 . . IEEE Intelligent Vehicles Symp . . 692 - - 697 . . DOI: 10.1109/IVS.2012.6232253 http://doi.org/10.1109/IVS.2012.6232253 . .

S Thrun , , , JJ Leonard . . Simultaneous localization and mapping . . 2008 . . Int. Conf. on Artificial Intelligence . . 871 - - 889 . . DOI: 10.1007/978-3-540-30301-5_38 http://doi.org/10.1007/978-3-540-30301-5_38 . .

L Ulrich . . 2016's Top Ten Tech Cars . . 2016 . . Available from: http://spectrum.ieee.org/transportation/advanced-cars/2016s-top-ten-tech-cars http://spectrum.ieee.org/transportation/advanced-cars/2016s-top-ten-tech-cars . .

J Xue , , , NN Zheng , , , J Geng , , , 等 . . Tracking multiple visual targets via particle-based belief propagation . . IEEE Trans. Syst. Man Cybern. B , , 2008 . . 38 ( ( 1 ): ): 196 - - 209 . . DOI: 10.1109/TSMCB.2007.910533 http://doi.org/10.1109/TSMCB.2007.910533 . .

Z Zhang . . A flexible new technique for camera calibration . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2000 . . 22 ( ( 11 ): ): 1330 - - 1334 . . DOI: 10.1109/34.888718 http://doi.org/10.1109/34.888718 . .

NN Zheng , , , ZY Liu , , , PJ Ren , , , 等 . . Hybridaugmented intelligence: collaboration and cognition , , 2017 . . : Front. Inform. Technol. Electron. Eng., : in press DOI: 10.1631/FITEE.1700053 http://doi.org/10.1631/FITEE.1700053 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Motion planning of a quadrotor robot game using a simulation-based projected policy iteration method

Safe navigation of quadrotors with jerk limited trajectory