A survey on indoor 3D modeling and applications via RGB-D devices

Zhilu YUAN; You LI; Shengjun TANG; Ming LI; Renzhong GUO; Weixi WANG

doi:10.1631/FITEE.2000097

Your Location：

Home >

Browse articles >

A survey on indoor 3D modeling and applications via RGB-D devices

Regular Papers | Updated：2022-06-06

- A survey on indoor 3D modeling and applications via RGB-D devices
- 基于RGB-D传感器的室内三维建模及应用研究综述
- Frontiers of Information Technology & Electronic Engineering Vol. 22, Issue 6, Pages: 815-826(2021)
- Affiliations：
  
  School of Architecture and Urban Planning, Research Institute for Smart Cities, Shenzhen University & China Guangdong–Hong Kong–Macau Joint Laboratory for Smart Cities & Key Laboratory of Urban Land Resources Monitoring and Simulation, Ministry of Natural Resources, Shenzhen 518060, China
  State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
- Author bio：
  
  Shengjun TANG, E-mail: shengjuntang@szu.edu.cn
- Funds：
- DOI：10.1631/FITEE.2000097
  CLC： P232
- Received：07 March 2020，
  
  Revised：2021-;6-;1，
  
  Published：2021-06
- Accepted：
Scan QR Code
Zhilu YUAN, You LI, Shengjun TANG, et al. A survey on indoor 3D modeling and applications via RGB-D devices[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(6): 815-826.
DOI：

Zhilu YUAN, You LI, Shengjun TANG, et al. A survey on indoor 3D modeling and applications via RGB-D devices[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(6): 815-826. DOI： 10.1631/FITEE.2000097.

摘要

随着消费级RGB-D摄像机的快速发展，真实世界的室内三维场景建模和机器人应用越来越受到重视。然而，室内三维场景建模仍具有挑战性，因室内物体结构可能具有较高复杂性，在此情况下，消费者级传感器采集的RGB-D数据质量需进一步提升。近年来，在提高消费者级传感器采集的RGB-D数据质量方面，有很多值得关注的研究。本文介绍了室内场景建模方法的最新进展、室内公共数据集和库以及RGB-D设备的典型应用，包括室内定位和紧急疏散。

Abstract

With the fast development of consumer-level RGB-D cameras

real-world indoor three-dimensional (3D) scene modeling and robotic applications are gaining more attention. However

indoor 3D scene modeling is still challenging because the structure of interior objects may be complex and the RGB-D data acquired by consumer-level sensors may have poor quality. There is a lot of research in this area. In this survey

we provide an overview of recent advances in indoor scene modeling methods

public indoor datasets and libraries which can facilitate experiments and evaluations

and some typical applications using RGB-D devices including indoor localization and emergency evacuation.

关键词

Keywords

references

J Biswas , , , M Veloso . . WiFi localization and navigation for autonomous indoor mobile robots . . IEEE Int Conf on robotics and automation , , 2010 . . p.4379 - - 4384 . . DOI: 10.1109/ROBOT.2010.5509842 http://doi.org/10.1109/ROBOT.2010.5509842 . .

AY Chen , , , JC Chu . . TDVRP and BIM integrated approach for in-building emergency rescue routing . . J Comput Civil Eng , , 2016 . . 30 ( ( 5 ): ): C4015003 DOI: 10.1061/(ASCE)CP.1943-5487.0000522 http://doi.org/10.1061/(ASCE)CP.1943-5487.0000522 . .

AY Chen , , , T Huang . . Toward BIM-enabled decision making for in-building response missions . . IEEE Trans Intell Transp Syst , , 2015 . . 16 ( ( 5 ): ): 2765 - - 2773 . . DOI: 10.1109/TITS.2015.2422138 http://doi.org/10.1109/TITS.2015.2422138 . .

C Chen , , , BS Yang , , , S Song , , , 等 . . Calibrate multiple consumer RGB-D cameras for low-cost and efficient 3D indoor mapping . . Remot Sens , , 2018 . . 10 ( ( 2 ): ): 328 DOI: 10.3390/rs10020328 http://doi.org/10.3390/rs10020328 . .

JW Chen , , , D Bautembach , , , S Izadi . . Scalable real-time volumetric surface reconstruction . . ACM Trans Graph , , 2013 . . 32 ( ( 4 ): ): 113 DOI: 10.1145/2461912.2461940 http://doi.org/10.1145/2461912.2461940 . .

K Chen , , , YK Lai , , , YX Wu , , , 等 . . Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information . . ACM Trans Graph , , 2014 . . 33 ( ( 6 ): ): 208 DOI: 10.1145/2661229.2661239 http://doi.org/10.1145/2661229.2661239 . .

SN Chen , , , YL Zheng , , , PP Luo , , , 等 . . Visual search based indoor localization in low light via RGB-D camera . . Int Comput Inform Eng , , 2017 . . 11 ( ( 3 ): ): 403 - - 406 . . DOI: 10.5281/zenodo.1129950 http://doi.org/10.5281/zenodo.1129950 . .

JCP Cheng , , , Y Tan , , , YZ Song , , , 等 . . Developing an evacuation evaluation model for offshore oil and gas platforms using BIM and agent-based model . . Autom Constr , , 2018 . . 89 214 - - 224 . . DOI: 10.1016/j.autcon.2018.02.011 http://doi.org/10.1016/j.autcon.2018.02.011 . .

K Chintalapudi , , , IA Padmanabha , , , VN Padmanabhan . . Indoor localization without the pain . . Proc 16 th Annual Int Conf on Mobile Computing and Networking , , 2010 . . p. 173 - - 184 . . DOI: 10.1145/1859995.1860016 http://doi.org/10.1145/1859995.1860016 . .

J Choi , , , J Choi , , , I Kim . . Development of BIM-based evacuation regulation checking system for high-rise and complex buildings . . Autom Constr , , 2014 . . 46 38 - - 49 . . DOI: 10.1016/j.autcon.2013.12.005 http://doi.org/10.1016/j.autcon.2013.12.005 . .

A Dai , , , AX Chang , , , M Savva , , , 等 . . ScanNet: richly- annotated 3D reconstructions of indoor scenes . . IEEE Conf on Computer Vision and Pattern Recognition , , 2017 . . p. 2432 - - 2443 . . . .

DR dos Santos , , , MA Basso , , , K Khoshelham , , , 等 . . Mapping indoor spaces by adaptive coarse-to-fine registration of RGB-D data . . IEEE Geosci Remot Sens Lett , , 2016 . . 13 ( ( 2 ): ): 262 - - 266 . . DOI: 10.1109/LGRS.2015.2508880 http://doi.org/10.1109/LGRS.2015.2508880 . .

F Endres , , , J Hess , , , J Sturm , , , 等 . . 3-D mapping with an RGB-D camera . . IEEE Trans Robot , , 2014 . . 30 ( ( 1 ): ): 177 - - 187 . . DOI: 10.1109/TRO.2013.2279412 http://doi.org/10.1109/TRO.2013.2279412 . .

N Engelhard , , , F Endres , , , J Hess , , , 等 . . Real-time 3D visual SLAM with a hand-held RGB-D camera . . Proc RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum , , 2011 . . 180 1 - - 15 . . . .

M Fehr , , , F Furrer , , , I Dryanovski , , , 等 . . TSDF-based change detection for consistent long-term dense reconstruction and dynamic object discovery . . IEEE Int Conf on Robotics and Automation , , 2017 . . p.5237 - - 5244 . . DOI: 10.1109/ICRA.2017.7989614 http://doi.org/10.1109/ICRA.2017.7989614 . .

D Fox , , , W Burgard , , , S Thrun . . Markov localization for mobile robots in dynamic environments . . J Artif Intell Res , , 1999 . . 11 391 - - 427 . . DOI: 10.1613/jair.616 http://doi.org/10.1613/jair.616 . .

A Handa , , , T Whelan , , , J McDonald , , , 等 . . A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM . . IEEE Int Conf on Robotics and Automation , , 2014 . . p.1524 - - 1531 . . DOI: 10.1109/ICRA.2014.6907054 http://doi.org/10.1109/ICRA.2014.6907054 . .

FN He , , , A Habib . . Three-point-based solution for automated motion parameter estimation of a multi-camera indoor mapping system with planar motion constraint . . ISPRS J Photogr Remot Sens , , 2018 . . 142 278 - - 291 . . DOI: 10.1016/j.isprsjprs.2018.06.011 http://doi.org/10.1016/j.isprsjprs.2018.06.011 . .

D Helbing , , , I Farkas , , , T Vicsek . . Simulating dynamical features of escape panic . . Nature , , 2000 . . 407 ( ( 6803 ): ): 487 - - 490 . . DOI: 10.1038/35035023 http://doi.org/10.1038/35035023 . .

P Henry , , , M Krainin , , , E Herbst , , , 等 . . RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments . . Int J Robot Res , , 2012 . . 31 ( ( 5 ): ): 647 - - 663 . . DOI: 10.1177/0278364911434148 http://doi.org/10.1177/0278364911434148 . .

AS Huang , , , A Bachrach , , , P Henry , , , 等 . . Visual odometry and mapping for autonomous flight using an RGB-D camera . . In: Christensen HI, Khatib O (Eds.), Robotics Research. Springer, Cham, , , 2017 . . p.235 - - 252 . . DOI: 10.1007/978-3-319-29363-9_14 http://doi.org/10.1007/978-3-319-29363-9_14 . .

S Ikehata , , , H Yang , , , Y Furukawa . . Structured indoor modeling . . IEEE Int Conf on Computer Vision , , 2015 . . p.1323 - - 1331 . . DOI: 10.1109/ICCV.2015.156 http://doi.org/10.1109/ICCV.2015.156 . .

C Kerl , , , J Sturm , , , D Cremers . . Robust odometry estimation for RGB-D cameras . . IEEE Int Conf on Robotics and Automation , , 2013 . . p.3748 - - 3754 . . DOI: 10.1109/ICRA.2013.6631104 http://doi.org/10.1109/ICRA.2013.6631104 . .

K Konolige , , , M Agrawal . . FrameSLAM: from bundle adjustment to real-time visual mapping . . IEEE Trans Robot , , 2008 . . 24 ( ( 5 ): ): 1066 - - 1077 . . DOI: 10.1109/tro.2008.2004832 http://doi.org/10.1109/tro.2008.2004832 . .

K Lai , , , L Bo , , , D Fox . . Unsupervised feature learning for 3D scene labeling . . IEEE Int Conf on Robotics and Automation , , 2014 . . p.3050 - - 3057 . . DOI: 10.1109/ICRA.2014.6907298 http://doi.org/10.1109/ICRA.2014.6907298 . .

M Li , , , RZ Chen , , , X Liao , , , 等 . . A real-time indoor visual localization and navigation method based on Tango smartphone . . Ubiquitous Positioning Indoor Navigation and Location Based Service , , 2018 . . p.1 - - 6 . . DOI: 10.1109/UPINLBS.2018.8559720 http://doi.org/10.1109/UPINLBS.2018.8559720 . .

HM Liu , , , GF Zhang , , , HJ Bao . . Robust keyframe-based monocular SLAM for augmented reality . . IEEE Int Symp on Mixed and Augmented Reality , , 2016 . . p.1 - - 10 . . DOI: 10.1109/ISMAR-Adjunct.2016.0111 http://doi.org/10.1109/ISMAR-Adjunct.2016.0111 . .

R Liu , , , J Du , , , RRA Issa . . Human library for emergency evacuation in BIM-based serious game environment . . Int Conf on Computing in Civil and Building Engineering , , 2014 . . p.544 - - 551 . . DOI: 10.1061/9780784413616.068 http://doi.org/10.1061/9780784413616.068 . .

J Ma , , , W Jia , , , J Zhang . . Research of building evacuation path to guide based on BIM . . 29 th Chinese Control and Decision Conf , , 2017 . . p.1814 - - 1818 . . DOI: 10.1109/CCDC.2017.7978811 http://doi.org/10.1109/CCDC.2017.7978811 . .

R Mur-Artal , , , JD Tardós . . ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras . . IEEE Trans Robot , , 2017 . . 33 ( ( 5 ): ): 1255 - - 1262 . . DOI: 10.1109/TRO.2017.2705103 http://doi.org/10.1109/TRO.2017.2705103 . .

RA Newcombe , , , S Izadi , , , O Hilliges , , , 等 . . KinectFusion: real-time dense surface mapping and tracking . . 10 th IEEE Int Symp on Mixed and Augmented Reality , , 2011 . . p.127 - - 136 . . DOI: 10.1109/ISMAR.2011.6092378 http://doi.org/10.1109/ISMAR.2011.6092378 . .

M Nießner , , , M Zollhöfer , , , S Izadi , , , 等 . . Real-time 3D reconstruction at scale using voxel hashing . . ACM Trans Graph , , 2013 . . 32 ( ( 6 ): ): 169 DOI: 10.1145/2508363.2508374 http://doi.org/10.1145/2508363.2508374 . .

LE Ortiz , , , VE Cabrera , , , LMG Gonçalves . . Depth data error modeling of the ZED 3D vision sensor from stereolabs . . ELCVIA Electron Lett Comput Vis Imag Anal , , 2018 . . 17 ( ( 1 ): ): 1 - - 15 . . DOI: 10.5565/rev/elcvia.1084 http://doi.org/10.5565/rev/elcvia.1084 . .

U Rüppel , , , K Schatz . . Designing a BIM-based serious game for fire safety evacuation simulations . . Adv Eng Inform , , 2011 . . 25 ( ( 4 ): ): 600 - - 611 . . DOI: 10.1016/j.aei.2011.08.001 http://doi.org/10.1016/j.aei.2011.08.001 . .

R Santos , , , AA Costa , , , A Grilo . . Bibliometric analysis and review of building information modelling literature published between 2005 and 2015 . . Autom Constr , , 2017 . . 80 118 - - 136 . . DOI: 10.1016/j.autcon.2017.03.005 http://doi.org/10.1016/j.autcon.2017.03.005 . .

A Segal , , , D Haehnel , , , S Thrun . . Generalized-ICP . . Proc Robotics: Science and Systems , , 2009 . . 2 4 DOI: 10.15607/RSS.2009.V.021 http://doi.org/10.15607/RSS.2009.V.021 . .

Y Shi , , , K Xu , , , M Nießner , , , 等 . . PlaneMatch: patch coplanarity prediction for robust RGB-D reconstruction . . LNCS , , 2018 . . 11212 767 - - 784 . . DOI: 10.1007/978-3-030-01237-3_46 http://doi.org/10.1007/978-3-030-01237-3_46 . .

S Song , , , SP Lichtenberg , , , J Xiao . . SUN RGB-D: a RGB-D scene understanding benchmark suite . . IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.567 - - 576 . . DOI: 10.1109/CVPR.2015.7298655 http://doi.org/10.1109/CVPR.2015.7298655 . .

F Steinbrucker , , , J Sturm , , , D Cremers . . Real-time visual odometry from dense RGB-D images . . IEEE Int Conf on Computer Vision Workshops , , 2011 . . p.719 - - 722 . . DOI: 10.1109/ICCVW.2011.6130321 http://doi.org/10.1109/ICCVW.2011.6130321 . .

J Sturm , , , N Engelhard , , , F Endres , , , 等 . . A benchmark for the evaluation of RGB-D SLAM systems . . IEEE/RSJ Int Conf on Intelligent Robots and Systems , , 2012 . . p.573 - - 580 . . DOI: 10.1109/IROS.2012.6385773 http://doi.org/10.1109/IROS.2012.6385773 . .

SJ Tang , , , Y Li , , , ZL Yuan , , , 等 . . A vertex-to-edge weighted closed-form method for dense RGB-D indoor SLAM . . IEEE Access , , 2019a . . 7 32019 - - 32029 . . DOI: 10.1109/ACCESS.2019.2900990 http://doi.org/10.1109/ACCESS.2019.2900990 . .

SJ Tang , , , YJ Zhang , , , Y Li , , , 等 . . Fast and automatic reconstruction of semantically rich 3D indoor maps from low-quality RGB-D sequences . . Sensors , , 2019b . . 19 ( ( 3 ): ): 533 DOI: 10.3390/s19030533 http://doi.org/10.3390/s19030533 . .

D Thomas , , , A Sugimoto . . Modeling large-scale indoor scenes with rigid fragments using RGB-D cameras . . Comput Vis Imag Underst , , 2017 . . 157 103 - - 116 . . DOI: 10.1016/j.cviu.2016.11.008 http://doi.org/10.1016/j.cviu.2016.11.008 . .

KM Vestena , , , DR dos Santos , , , EMJr Oilveira , , , 等 . . A weighted closed-form solution for RGB-D data registration . . Int Arch Photogr Remote Sens Spat Inform Sci , , 2016 . . XLI-B3 403 - - 409 . . DOI: 10.5194/isprsarchives-XLI-B3-403-2016 http://doi.org/10.5194/isprsarchives-XLI-B3-403-2016 . .

KK Wang , , , GF Zhang , , , HJ Bao . . Robust 3D reconstruction with an RGB-D camera . . IEEE Trans Imag Process , , 2014 . . 23 ( ( 11 ): ): 4893 - - 4906 . . DOI: 10.1109/TIP.2014.2352851 http://doi.org/10.1109/TIP.2014.2352851 . .

MJ Westoby , , , J Brasington , , , NF Glasser , , , 等 . . 'Structure-from-motion' photogrammetry: a low-cost, effective tool for geoscience applications . . Geomorphology , , 2012 . . 179 300 - - 314 . . DOI: 10.1016/j.geomorph.2012.08.021 http://doi.org/10.1016/j.geomorph.2012.08.021 . .

T Whelan , , , K Michael , , , F Maurice , , , 等 . . Kintinuous: Spatially Extended Kinectfusion . . Technical Report No. MIT-CSAIL-TR-2012-020 , , 2013 . . https://doi.org/1721.1/71756 https://doi.org/1721.1/71756 , , . .

W Winterhalter , , , F Fleckenstein , , , B Steder , , , 等 . . Accurate indoor localization for RGB-D smartphones and tablets given 2D floor plans . . IEEE/RSJ Int Conf on Intelligent Robots and Systems , , 2015 . . p.3138 - - 3143 . . DOI: 10.1109/IROS.2015.7353811 http://doi.org/10.1109/IROS.2015.7353811 . .

ZL Yuan , , , HF Jia , , , MJ Liao , , , 等 . . Simulation model of self-organizing pedestrian movement considering following behavior . . Front Inform Technol Electron Eng , , 2017 . . 18 ( ( 8 ): ): 1142 - - 1150 . . DOI: 10.1631/FITEE.1601592 http://doi.org/10.1631/FITEE.1601592 . .

ZL Yuan , , , HF Jia , , , LF Zhang , , , 等 . . A social force evacuation model considering the effect of emergency signs . . Simulation , , 2018 . . 94 ( ( 8 ): ): 723 - - 737 . . DOI: 10.1177/0037549717741350 http://doi.org/10.1177/0037549717741350 . .

ZL Yuan , , , RZ Guo , , , SJ Tang , , , 等 . . Simulation of the separating crowd behavior in a T-shaped channel based on the social force model . . IEEE Access , , 2019 . . 7 13668 - - 13682 . . DOI: 10.1109/ACCESS.2019.2894345 http://doi.org/10.1109/ACCESS.2019.2894345 . .

M Zeng , , , FK Zhao , , , JX Zheng , , , 等 . . A memory-efficient KinectFusion using octree . . In: Hu SM, Martin RR (Eds.), Computational Visual Media. Springer, Berlin , , 2012 . . p.234 - - 241 . . DOI: 10.1007/978-3-642-34263-9_30 http://doi.org/10.1007/978-3-642-34263-9_30 . .

H Zou , , , N Li , , , L Cao . . Immersive virtual environments for investigating building emergency evacuation behaviors: a feasibility study . . 33 th Int Symp on Automation and Robotics in Construction , , 2016 . . p.1 - - 8 . . DOI: 10.22260/ISARC2016/0040 http://doi.org/10.22260/ISARC2016/0040 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Performance analysis of visual markers for indoor navigation systems#

Related Author

Gaetano C. LA DELFA

Salvatore MONTELEONE

Vincenzo CATANIA

Juan F. DE PAZ

Javier BAJO

Related Institution

Department of Electrical, Electronics and Computer Engineering (DIEEI), University of Catania

BISITE Research Group, Faculty of Science, University of Salamanca

Department of Artificial Intelligence, Polytechnic University of Madrid

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰