Inertial measurement unit-camera calibration based on incomplete inertial sensor information

Hong LIU; Yu-long ZHOU; Zhao-peng GU

doi:10.1631/jzus.C1400038

Your Location：

Home >

Browse articles >

Inertial measurement unit-camera calibration based on incomplete inertial sensor information

Computer & Automation | Updated：2022-05-19

- Inertial measurement unit-camera calibration based on incomplete inertial sensor information
- 基于部分惯性传感器信息的惯性传感器–摄像机标定方法
- Frontiers of Information Technology & Electronic Engineering Vol. 15, Issue 11, Pages: 999-1008(2014)
- Affiliations：
  
  Engineering Lab on Intelligent Perception for Internet of Things (ELIP), Shenzhen Graduate School, Peking University, Shenzhen 518055, China
- Author bio：
  
  [ "Hong LIU, E-mail: hongliu@pku.edu.cn" ]
  [ "Yu-long ZHOU, E-mail: 1101213442@sz.pku.edu.cn" ]
  Zhao-peng GU, E-mail: Guzp@pkusz.edu.cn
- Funds：
- DOI：10.1631/jzus.C1400038
  CLC： TP391
- Received：07 February 2014，
  
  Revised：2014-10-15，
  
  Published：2014-11
- Accepted：
Scan QR Code
Hong LIU, Yu-long ZHOU, Zhao-peng GU. Inertial measurement unit-camera calibration based on incomplete inertial sensor information[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(11): 999-1008.
DOI：

Hong LIU, Yu-long ZHOU, Zhao-peng GU. Inertial measurement unit-camera calibration based on incomplete inertial sensor information[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(11): 999-1008. DOI： 10.1631/jzus.C1400038.

摘要

研究目的

对于低端惯性传感器与摄像机组合设备惯性传感器Yaw角输出误差较大或无Yaw输出的情况，进行惯性传感器与摄像机之间的精确标定。

创新要点

分析经典手眼标定方法，利用标定方程中有关矩阵的相似性质，使得在缺少惯性传感器Yaw角测量信息的情况下，仍然能够精确标定惯性传感器与摄像机之间的相对姿态。

研究方法

首先，使惯性传感器和摄像机一起运动两次（图3）。根据摄像机拍摄标定板的视频图像求出摄像机两次相对位置姿态，同时记录两次惯性传感器的Roll角和Pitch角的输出。然后，通过分析经典手眼标定方程R

X=XR

（公式4），得出矩阵R

与矩阵R

为相似矩阵，且二者迹相等。据此，恢复两次惯性传感器运动的相对旋转矩阵R

（算法1）。最后，为获得精准标定结果，记录惯性传感器-摄像机设备的多次运动数据，并利用Levenberg-Marquardt（LM）算法进行优化，以减小噪声影响，获得惯性传感器和摄像机之间的相对旋转（图5）。

重要结论

利用相似矩阵的性质，可以恢复两次运动之间惯性传感器间的旋转矩阵，利用LM算法，可以优化标定结果。

Abstract

This paper is concerned with the problem of estimating the relative orientation between an inertial measurement unit (IMU) and a camera. Unlike most existing IMU-camera calibrations

the main challenge in this paper is that the information output from the IMU is incomplete. For example

only two tilt information can be read from the gravity sensor of a smart phone. Despite incomplete inertial information

there are strong restrictions between the IMU and camera coordinate systems. This paper addresses the incomplete information based IMU-camera calibration problem by exploiting the intrinsic restrictions among the coordinate transformations. First

the IMU transformation between two poses is formulated with the unknown IMU information. Then the defective IMU information is restored using the complementary visual information. Finally

the Levenberg-Marquardt (LM) algorithm is applied to estimate the optimal calibration result in noisy environments. Experiments on both synthetic and real data show the validity and robustness of our algorithm.

关键词

Keywords

references

T. Bajd , , , M. Mihelj , , , M. Munih . . Introduction to Robotics . . Springer, the Netherlands , , 2013 . . DOI: 10.1007/978-94-007-6101-8 http://doi.org/10.1007/978-94-007-6101-8 . .

H.H. Chen . . A screw motion approach to uniqueness analysis of head-eye geometry . . Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition , , 1991 . . p.145 - - 151 . . DOI: 10.1109/CVPR.1991.139677 http://doi.org/10.1109/CVPR.1991.139677 . .

J.C.K. Chow , , , M. Kamel . . Finding the position and orientation of a sensor on a robot manipulator using quaternions . . Int. J. Robot. Res. , , 1991 . . 10 ( ( 3 ): ): 240 - - 254 . . DOI: 10.1177/027836499101000305 http://doi.org/10.1177/027836499101000305 . .

I. Fassi , , , G. Legnani . . Hand to sensor calibration: a geometrical interpretation of the matrix equation AX=XB . . J. Robot. Syst. , , 2005 . . 22 ( ( 9 ): ): 497 - - 506 . . DOI: 10.1002/rob.20082 http://doi.org/10.1002/rob.20082 . .

Z.P. Gu . . A Study on Monocular Simultaneous Localization and Mapping , , : : Beijing, China PhD Thesis, Chinese Academy of Sciences , , 2011 . . .

Z.P. Gu , , , Q.L. Dong . . Monocular SLAM based on partial IMU information . . J. Comput. Aid. Des. Comput. Graph. , , 2012 . . 24 ( ( 2 ): ): 155 - - 160 . . . .

R. Hartley , , , A. Zisserman . . Multiple View Geometry in Computer Vision , , : : NY, USA Cambridge University Press , , 2003 . . .

J. Heller , , , M. Havlena , , , T. Pajdla . . A branch-andbound algorithm for globally optimal hand-eye calibration . . Proc. IEEE Conf. on Computer Vision and Pattern Recognition , , 2012 . . p.1608 - - 1615 . . DOI: 10.1109/CVPR.2012.6247853 http://doi.org/10.1109/CVPR.2012.6247853 . .

R. Horaud , , , F. Dornaika . . Hand-eye calibration . . Int. J. Robot. Res. , , 1995 . . 14 ( ( 3 ): ): 195 - - 210 . . DOI: 10.1177/027836499501400301 http://doi.org/10.1177/027836499501400301 . .

J. Lobo , , , J. Dias . . Relative pose calibration between visual and inertial sensors . . Int. J. Robot. Res. , , 2007 . . 26 ( ( 6 ): ): 561 - - 575 . . DOI: 10.1177/0278364907079276 http://doi.org/10.1177/0278364907079276 . .

Y. Lu , , , J.C.K. Chou . . Eight-space quaternion approach for robotic hand-eye calibration . . Proc. IEEE Int. Conf. on Systems, Man and Cybernetics , , 1995 . . p.3316 - - 3321 . . DOI: 10.1109/ICSMC.1995.538297 http://doi.org/10.1109/ICSMC.1995.538297 . .

S. Ma , , , Z. Zhang . . Computer Vision , , : : Beijing, China Science Press , , 1998 . . .

D.W. Marquardt . . An algorithm for least-squares estimation of nonlinear parameters . . J. Soc. Ind. Appl. Math. , , 1963 . . 11 ( ( 2 ): ): 431 - - 441 . . DOI: 10.1137/0111030 http://doi.org/10.1137/0111030 . .

F.M. Mirzaei , , , S.I. Roumeliotis . . A Kalman filter-based algorithm for IMU-camera calibration: observability analysis and performance evaluation . . IEEE Trans. Robot. , , 2008 . . 24 ( ( 5 ): ): 1143 - - 1156 . . DOI: 10.1109/TRO.2008.2004486 http://doi.org/10.1109/TRO.2008.2004486 . .

G. Nützi , , , S. Weiss , , , D. Scaramuzza , , , 等 . . Fusion of IMU and vision for absolute scale estimation in monocular SLAM . . J. Intell. Robot. Syst. , , 2011 . . 61 ( ( 1-4 ): ): 287 - - 299 . . DOI: 10.1007/s10846-010-9490-z http://doi.org/10.1007/s10846-010-9490-z . .

F.C. Park , , , B.J. Martin . . Robot sensor calibration: solving AX=XB on the Euclidean group . . IEEE Trans. Robot. Autom. , , 1994 . . 10 ( ( 5 ): ): 717 - - 721 . . DOI: 10.1109/70.326576 http://doi.org/10.1109/70.326576 . .

Y. Seo , , , Y.J. Choi , , , S.W. Lee . . A branch-and-bound algorithm for globally optimal calibration of a cameraand-rotation-sensor system . . Proc. 12th Int. Conf. on Computer Vision , , 2009 . . p.1173 - - 1178 . . DOI: 10.1109/ICCV.2009.5459343 http://doi.org/10.1109/ICCV.2009.5459343 . .

Y.C. Shiu , , , S. Ahmad . . Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX=XB . . IEEE Trans. Robot. Autom. , , 1989 . . 5 ( ( 1 ): ): 16 - - 29 . . DOI: 10.1109/70.88014 http://doi.org/10.1109/70.88014 . .

K.H. Strobl , , , G. Hirzinger . . Optimal hand-eye calibration . . Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems , , 2006 . . p.4647 - - 4653 . . DOI: 10.1109/IROS.2006.282250 http://doi.org/10.1109/IROS.2006.282250 . .

R.Y. Tsai , , , R.K. Lenz . . A new technique for fully autonomous and efficient 3D robotics hand/eye calibration . . IEEE Trans Robot. Autom. , , 1989 . . 5 ( ( 3 ): ): 345 - - 358 . . DOI: 10.1109/70.34770 http://doi.org/10.1109/70.34770 . .

F. Vicentini , , , N. Pedrocchi , , , M. Malosio , , , 等 . . Highaccuracy hand-eye calibration from motion on manifolds . . Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems , , 2011 . . p.3327 - - 3334 . . DOI: 10.1109/IROS.2011.6094587 http://doi.org/10.1109/IROS.2011.6094587 . .

Y. Wang , , , X. Ma , , , Y. Wang , , , 等 . . Location optimization of multiple distribution centers under fuzzy environment . . J. Zhejiang Univ.-Sci. A (Appl. Phys. & Eng.) , , 2012 . . 13 ( ( 10 ): ): 782 - - 798 . . DOI: 10.1631/jzus.A1200137 http://doi.org/10.1631/jzus.A1200137 . .

Z. Zhang . . A flexible new technique for camera calibration . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2000 . . 22 ( ( 11 ): ): 1330 - - 1334 . . DOI: 10.1109/34.888718 http://doi.org/10.1109/34.888718 . .

Z. Zhao . . Hand-eye calibration using convex optimization . . Proc. IEEE Int. Conf. on Robotics and Automation , , 2011 . . p.2947 - - 2952 . . DOI: 10.1109/ICRA.2011.5979569 http://doi.org/10.1109/ICRA.2011.5979569 . .

Z. Zhao , , , Y. Liu . . A hand-eye calibration algorithm based on screw motions . . Robotica , , 2009 . . 27 ( ( 2 ): ): 217 - - 223 . . DOI: 10.1017/S0263574708004608 http://doi.org/10.1017/S0263574708004608 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Light field imaging for computer vision: a survey

Auxiliary diagnostic system for ADHD in children based on AI technology

A quantitative attribute-based benchmark methodology for single-target visual tracking

Light field imaging; Plenoptic function; Imaging model; Calibration; Reconstruction

Performance analysis of visual markers for indoor navigation systems#

Related Author

Shengyong CHEN

Meng ZHAO

Fan SHI

Chen JIA

Ming KONG

Qiang ZHU

Rong LI

Chunmao WANG

Related Institution

Engineering Research Center of Learning-Based Intelligent System (Ministry of Education), Tianjin University of Technology

Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology

Department of Psychology, The Children's Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health

College of Computer Science and Technology, Zhejiang University

Hikvision Research Institute

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰