Applications of structure from motion: a survey

WEI Ying-mei; KANG Lai; YANG Bing; WU Ling-da

doi:10.1631/jzus.CIDE1302

Your Location：

Home >

Browse articles >

Applications of structure from motion: a survey

Special Issue on the Recent Advances of Data-Driven Digital Entertainment | Updated：2022-06-06

- Applications of structure from motion: a survey
- Applications of structure from motion: a survey
- Frontiers of Information Technology & Electronic Engineering Vol. 14, Issue 7, Pages: 486-494(2013)
- Affiliations：
  
  1.College of Information System and Management, National University of Defense Technology, Changsha 410073, China
  2.National Laboratory of Electronic Information Equipment System, Academy of Equipment Command and Technology, Beijing 101416, China
- Author bio：
  
  [ "Ying-mei WEI, E-mail: weiyingmei126@126.com" ]
  [ "Lai KANG, E-mail: lkang.vr@gmail.com" ]
  [ "Bing YANG, E-mail: mmlab@126.com" ]
  [ "Ling-da WU, E-mail: wld@nudt.edu.cn" ]
- Funds：
- DOI：10.1631/jzus.CIDE1302
  CLC： TP391
- Received：29 December 2012，
  
  Revised：2013-06-06，
  
  Published：2013-07
- Accepted：
Scan QR Code
WEI Ying-mei, KANG Lai, YANG Bing, et al. Applications of structure from motion: a survey[J]. Frontiers of Information Technology & Electronic Engineering, 2013, 14(7): 486-494.
DOI：

WEI Ying-mei, KANG Lai, YANG Bing, et al. Applications of structure from motion: a survey[J]. Frontiers of Information Technology & Electronic Engineering, 2013, 14(7): 486-494. DOI： 10.1631/jzus.CIDE1302.

摘要

Abstract

Structure from motion (SfM) has been an active research area in computer vision for decades and numerous practical applications are benefiting from this research. While no previous work has tried to summarize the applications appearing in the literature

this paper deals with a comprehensive overview of recent applications of SfM by classifying them into 10 categories

namely augmented reality

autonomous navigation/guidance

motion capture

hand-eye calibration

image/video processing

image-based 3D modeling

remote sensing

image organization/browsing

segmentation and recognition

and military applications. The goal is to provide insights for researchers to position their work more appropriately in the context of existing techniques

and to perceive both new applications and relevant research problems.

关键词

Keywords

references

S. Agarwal , N. Snavely , I. Simon , S.M. Seitz , R. Szeliski . Building Rome in a Day . Proc. IEEE Int. Conf. on Computer Vision , 2009 . p.72 - 79 . DOI: 10.1109/ICCV.2009.5459148 http://doi.org/10.1109/ICCV.2009.5459148 .

S. Agarwal , N. Snavely , S.M. Seitz , R. Szeliski . Bundle Adjustment in the Large . Proc. European Conf. on Computer Vision:Part Ⅱ , 2010 . p.29 - 42 . DOI: 10.1007/978-3-642-15552-9_3 http://doi.org/10.1007/978-3-642-15552-9_3 .

N. Andreff , R. Horaud , B. Espiau . Robot hand-eye calibration using structure-from-motion . Int. J. Robot. Res. , 2001 . 20 ( 3 ): 228 - 248 . DOI: 10.1177/02783640122067372 http://doi.org/10.1177/02783640122067372 .

S.Y. Bao , M. Bagra , Y.W. Chao , S. Savarese . Semantic Structure from Motion with Points, Regions, and Objects . CVPR , 2012 . p.2703 - 2710 . DOI: 10.1109/CVPR.2012.6247992 http://doi.org/10.1109/CVPR.2012.6247992 .

P. Bhat , C.L. Zitnick , N. Snavely , A. Agarwala , M. Agrawala , M. Cohen , B. Curless , S.B. Kang . Using Photographs to Enhance Videos of a Static Scene . Eurographics Symp. on Rendering , 2007 . p.327 - 338 . DOI: 10.2312/EGWR/EGSR07/327-338 http://doi.org/10.2312/EGWR/EGSR07/327-338 .

G.J. Brostow , J. Shotton , J. Fauqueur , R. Cipolla . Segmentation and Recognition Using Structure from Motion Point Clouds . Proc. 10th European Conf. on Computer Vision:Part Ⅰ , 2008 . p.44 - 57 . DOI: 10.1007/978-3-540-88682-2_5 http://doi.org/10.1007/978-3-540-88682-2_5 .

G.J. Brostow , J. Fauqueur , R. Cipolla . Semantic object classes in video:a high definition ground truth database . Pattern Recogn. Lett. , 2009 . 30 ( 2 ): 88 - 97 . DOI: 10.1016/j.patrec.2008.04.005 http://doi.org/10.1016/j.patrec.2008.04.005 .

K. Cornelis , M. Pollefeys , L.V. Gool . Tracking Based Structure and Motion Recovery for Augmented Video Productions . Proc. ACM Symp. on Virtual Reality Software and Technology , 2001 . p.17 - 24 . DOI: 10.1145/505008.505012 http://doi.org/10.1145/505008.505012 .

J.M. Frahm , P. Fite-Georgel , D. Gallup , T. Johnson , R. Raguram , C. Wu , Y.H. Jen , E. Dunn , B. Clipp , S. Lazebnik , 等 . Building Rome on a Cloudless Day . Proc. 11th European Conf. on Computer Vision:Part Ⅳ , 2010 . p.368 - 381 . DOI: 10.1007/978-3-642-15561-1_27 http://doi.org/10.1007/978-3-642-15561-1_27 .

R. Hartley , A. Zisserman . Multiple View Geometry in Computer Vision , : Cambridge, UK Cambridge University Press , 2004 . DOI: 10.1017/CBO9780511811685 http://doi.org/10.1017/CBO9780511811685 .

N. Hasler , B. Rosenhahn , T. Thormahlen , M. Wand , J. Gall , H.P. Seidel . Markerless Motion Capture with Unsynchronized Moving Cameras . CVPR , 2009 . p.224 - 231 . DOI: 10.1109/CVPR.2009.5206859 http://doi.org/10.1109/CVPR.2009.5206859 .

M.A. Helala , L.A. Zarrabeitia , F.Z. Qureshi . Mosaic of Near Ground UAV Videos under Parallax Effects . Proc. 6th ACM/IEEE Int. Conf. on Distributed Smart Cameras , 2012 . p.1 - 6 . .

J. Heller , M. Havlena , A. Sugimoto , T. Pajdla . Structure-from-Motion Based Hand-Eye Calibration Using l ∞ Minimization . CVPR , 2011 . p.3497 - 3503 . DOI: 10.1109/CVPR.2011.5995629 http://doi.org/10.1109/CVPR.2011.5995629 .

A. Irschara , C. Zach , J.M. Frahm , H. Bischof . From Structure-from-Motion Point Clouds to Fast Location Recognition . CVPR , 2009 . p.2599 - 2606 . DOI: 10.1109/CVPR.2009.5206587 http://doi.org/10.1109/CVPR.2009.5206587 .

N.L. Jackson . Precision Reconstruction Based Tracking for Autonomous Synthetic Battlefield Displays Acquired from Unmanned Aerial Vehicle Video Streams . Dissertation, Morgan State University, Baltimore, Maryland, United States , 2008 .

C. Kurz , T. Thormahlen , H.P. Seidel . SceneAware Video Stabilization by Visual Fixation . Proc. Conf. for Visual Media Production , 2009 . p.1 - 6 . DOI: 10.1109/CVMP.2009.9 http://doi.org/10.1109/CVMP.2009.9 .

Y. Li , N. Snavely , D. Huttenlocher . Location Recognition Using Prioritized Feature Matching . Proc. 11th European Conf. on Computer Vision:Part Ⅱ , 2010 . p.791 - 804 . DOI: 10.1007/978-3-642-15552-9_57 http://doi.org/10.1007/978-3-642-15552-9_57 .

Y. Li , N. Snavely , D. Huttenlocher , P. Fua . Worldwide Pose Estimation Using 3D Point Clouds . Proc. 12th European Conf. on Computer Vision:Part Ⅰ , 2012 . p.15 - 29 . DOI: 10.1007/978-3-642-33718-5_2 http://doi.org/10.1007/978-3-642-33718-5_2 .

F. Liu , M. Gleicher , H. Jin , A. Agarwala . Contentpreserving warps for 3D video stabilization . ACM Trans. Graph. , 2009 . 28 ( 3 ): Article 44, p.1 - 9 . DOI: 10.1145/1531326.1531350 http://doi.org/10.1145/1531326.1531350 .

H.C. Longuet-Higgins . A computer algorithm for reconstructing a scene from two projections . Nature , 1981 . 293 ( 5828 ): 133 - 135 . DOI: 10.1038/293133a0 http://doi.org/10.1038/293133a0 .

M.I.A. Lourakis , A.A. Argyros . SBA:a software package for generic sparse bundle adjustment . ACM Trans. Math. Software , 2009 . 36 ( 1 ): 1 - 30 . DOI: 10.1145/1486525.1486527 http://doi.org/10.1145/1486525.1486527 .

D.G. Lowe . Distinctive image features from scaleinvariant keypoints . Int. J. Comput. Vis. , 2004 . 60 ( 2 ): 91 - 110 . DOI: 10.1023/B:VISI.0000029664.99615.94 http://doi.org/10.1023/B:VISI.0000029664.99615.94 .

A.M. Manferdini . A Methodology for the Promotion of Cultural Heritage Sites Through the Use of LowCost Technologies and Procedures . Proc. 17th Int. Conf. on 3D Web Technology , 2012 . p.180 DOI: 10.1145/2338714.2338747 http://doi.org/10.1145/2338714.2338747 .

J. Manweiler , P. Jain , R.R. Choudhury . Satellites in Our Pockets:an Object Positioning System Using Smartphones . Proc. 10th Int. Conf. on Mobile Systems, Applications, and Services , 2012 . p.211 - 224 . DOI: 10.1145/2307636.2307656 http://doi.org/10.1145/2307636.2307656 .

K. Mikolajczyk , C. Schmid . A performance evaluation of local descriptors . IEEE Trans. Pattern Anal. Mach. Intell. , 2005 . 27 ( 10 ): 1615 - 1630 . DOI: 10.1109/TPAMI.2005.188 http://doi.org/10.1109/TPAMI.2005.188 .

J. Mooser , S. You , U. Neumann , Q. Wang . Applying Robust Structure from Motion to Markerless Augmented Reality . Workshop on Applications of Computer Vision , 2009 . p.1 - 8 . DOI: 10.1109/WACV.2009.5403038 http://doi.org/10.1109/WACV.2009.5403038 .

O. Moslah , V. Guitteny , S. Couvet . Geo-referencing Uncalibrated Photographs Using Aerial Images and 3D Urban Models . CORESA , 2009 . p.1 - 5 . .

M. Muja , D.G. Lowe . Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration . Proc. 4th Int. Conf. on Computer Vision Theory and Applications , 2009 . p.331 - 340 . .

P. Musialski , P. Wonka , D.G. Aliaga , M. Wimmer , L. van Gool , W. Purgathofer . A Survey of Urban Reconstruction . Eurographics State of the Art Reports , 2012 . p.1 - 28 . DOI: 10.1111/cgf.12077 http://doi.org/10.1111/cgf.12077 .

K. Nassar , E.A. Aly , Y. Jung . Structure-from-Motion for Earthwork Planning . Proc. 28th ISARC , 2011 . p.310 - 316 . .

T. Nicosevici , R. Garcia . Online Robust 3D Mapping Using Structure from Motion Cues . OCEANS , 2008 . p.1 - 7 . DOI: 10.1109/OCEANSKOBE.2008.4531022 http://doi.org/10.1109/OCEANSKOBE.2008.4531022 .

U. Niethammer , S. Rothmund , U. Schwaderer , J. Zeman , M. Joswig . Open Source Image-Processing Tools for Low-Cost UAV-Based Landslide Investigations . Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences , 2011 . p.1 - 6 . .

D. Nilosek , K. Walli . Aerial Scene Synthesis from Images . SIGGRAPH Posters, Article 65. , 2009 . DOI: 10.1145/1599301.1599366 http://doi.org/10.1145/1599301.1599366 .

J. Oliensis . A multi-frame structure-from-motion algorithm under perspective projection . Int. J. Comput. Vis. , 1999 . 34 ( 2-3 ): 163 - 192 . DOI: 10.1023/A:1008139920864 http://doi.org/10.1023/A:1008139920864 .

M. Pollefeys , L.V. Gool , M. Vergauwen , K. Cornelis , F. Verbiest , J. Tops . Image-Based 3D Acquisition of Archaeological Heritage and Applications . Proc. Conf. on Virtual Reality, Archeology, and Cultural Heritage , 2001a . p.255 - 262 . DOI: 10.1145/584993.585033 http://doi.org/10.1145/584993.585033 .

M. Pollefeys , M. Vergauwen , K. Cornelis , F. Verbiest , J. Schouteden , J. Tops , L.V. Gool . 3D Acquisition of Archaeological Heritage from Images . CIPA Conf., Int. Archive of Photogrammetry and Remote Sensing , 2001b . p.1 - 8 . .

M. Pupilli , A. Calway . Real-Time Structure from Motion for Augmented Reality . University of Bristol, Bristol, UK. , 2002 .

T. Pylvanainen , J. Berclaz , T. Korah , V. Hedau , M. Aanjaneya , R. Grzeszczuk . 3D City Modeling from Street-Level Data for Augmented Reality Applications . 2nd Int. Conf. on 3D Imaging, Modeling, Processing, Visualization and Transmission , 2012 . p.238 - 245 . DOI: 10.1109/3DIMPVT.2012.19 http://doi.org/10.1109/3DIMPVT.2012.19 .

F. Remondino , S. El-Hakim . Image-based 3D modelling:a review . Photogrammetr. Rec. , 2006 . 21 ( 115 ): 269 - 291 . DOI: 10.1111/j.1477-9730.2006.00383.x http://doi.org/10.1111/j.1477-9730.2006.00383.x .

E. Royer , M. Lhuillier , M. Dhome , J.M. Lavest . Monocular vision for mobile robot localization and autonomous navigation . Int. J. Comput. Vis. , 2007 . 74 ( 3 ): 237 - 260 . DOI: 10.1007/s11263-006-0023-y http://doi.org/10.1007/s11263-006-0023-y .

T. Sato , A. Iketani , S. Ikeda , M. Kanbara , N. Nakajima , N. Yokoya . Video Mosaicing for Curved Documents by Structure from Motion . ACM SIGGRAPH Sketches , 2006 . DOI: 10.1145/1179849.1180007 http://doi.org/10.1145/1179849.1180007 .

F. Schaffalitzky , A. Zisserman . Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps" . Proc. 7th European Conf. on Computer Vision:Part Ⅰ , 2002 . p.414 - 431 . DOI: 10.1007/3-540-47969-4_28 http://doi.org/10.1007/3-540-47969-4_28 .

G. Schindler , P. Krishnamurthy , F. Dellaert . LineBased Structure from Motion for Urban Environments . Proc. 3rd Int. Symp. on 3D Data Processing, Visualization, and Transmission , 2006 . p.846 - 853 . DOI: 10.1109/3DPVT.2006.90 http://doi.org/10.1109/3DPVT.2006.90 .

J. Schmidt , F. Vogt , H. Niemann . Calibration Free HandEye Calibration:a Structure from Motion Approach . Proc. 27th DAGM Conf. on Pattern Recognition , 2005 . p.67 - 74 . DOI: 10.1007/11550518_9 http://doi.org/10.1007/11550518_9 .

G. Schweighofer , S. Segvic , A. Pinz . Online/Realtime Structure and Motion for General Camera Models . IEEE Workshop on Applications of Computer Vision , 2008 . p.1 - 6 . DOI: 10.1109/WACV.2008.4544016 http://doi.org/10.1109/WACV.2008.4544016 .

M. Shim , S. Yilma , K. Bonner . A robust realtime structure from motion for situational awareness and RSTA . SPIE , 2008 . 6962 1 - 11 . DOI: 10.1117/12.778074 http://doi.org/10.1117/12.778074 .

T. Shiratori , H.S. Park , L. Sigal , Y. Sheikh , J.K. Hodgins . Motion capture from body-mounted cameras . ACM Trans. Graph. , 2011 . 30 ( 4 ): Article 31, p.1 - 10 . DOI: 10.1145/2010324.1964926 http://doi.org/10.1145/2010324.1964926 .

S.N. Sinha , D. Steedly , R. Szeliski , M. Agrawala , M. Pollefeys . Interactive 3D architectural modeling from unordered photo collections . ACM Trans. Graph. , 2008 . 27 ( 5 ): Article 159, p.1 - 10 . DOI: 10.1145/1409060.1409112 http://doi.org/10.1145/1409060.1409112 .

N. Snavely , S.M. Seitz , R. Szeliski . Photo tourism:exploring photo collections in 3D . ACM Trans. Graph. , 2006 . 25 ( 3 ): 835 - 846 . DOI: 10.1145/1141911.1141964 http://doi.org/10.1145/1141911.1141964 .

N. Snavely , I. Simon , M. Goesele , R. Szeliski , S.M. Seitz . Scene reconstruction and visualization from community photo collections . Proc. IEEE , 2010 . 98 ( 8 ): 1370 - 1390 . DOI: 10.1109/JPROC.2010.2049330 http://doi.org/10.1109/JPROC.2010.2049330 .

M. Spetsakis , J. Aloimonos . A multi-frame approach to visual motion perception . Int. J. Comput. Vis. , 1991 . 6 ( 3 ): 245 - 255 . DOI: 10.1007/BF00115698 http://doi.org/10.1007/BF00115698 .

S. Srinivasan , R. Chellappa . Fast Structure from Motion Recovery Applied to 3D Image Stabilization . Proc. IEEE Int. Conf. on the Acoustics, Speech, and Signal , 1999 . p.3357 - 3360 . DOI: 10.1109/ICASSP.1999.757561 http://doi.org/10.1109/ICASSP.1999.757561 .

B. Streckel , J.F. Evers-Senne , R. Koch . Lens Model Selection for a Markerless AR Tracking System . Proc. 4th IEEE and ACM Int. Symp. on Mixed and Augmented Reality , 2005 . p.130 - 133 . DOI: 10.1109/ISMAR.2005.38 http://doi.org/10.1109/ISMAR.2005.38 .

D.W. Strucl , M. Quartisch . A Structure Based Mosaicking Approach for Aerial Images from Low Altitude of Non-planar Scenes . Proc. 16th Computer Vision Winter Workshop , 2001 . p.51 - 58 . .

P. Sturgess , K. Alahari , L. Ladicky , P.H.S. Torr . Combining Appearance and Structure from Motion Features for Road Scene Understanding . Proc. British Machine Vision Conf. , 2009 . p.1 - 11 . DOI: 10.5244/C.23.62 http://doi.org/10.5244/C.23.62 .

R. Szeliski . Computer Vision:Algorithms and Applications . Springer, New York , 2010 .

R. Szeliski , S.B. Kang . Recovering 3D shape and motion from image streams using nonlinear least squares . J. Vis. Commun. Image Represent. , 1994 . 5 ( 1 ): 10 - 28 . DOI: 10.1006/jvci.1994.1002 http://doi.org/10.1006/jvci.1994.1002 .

C. Tomasi . Shape and motion from image streams under orthography:a factorization method . Int. J. Comput. Vis. , 1992 . 9 ( 2 ): 137 - 154 . DOI: 10.1007/BF00129684 http://doi.org/10.1007/BF00129684 .

B. Triggs , P. Mclauchlan , R. Hartley , A. Fitzgibbon . Bundle adjustment:a modern synthesis . LNCS , 2000 . 1883 298 - 375 . DOI: 10.1007/3-540-44480-7_21 http://doi.org/10.1007/3-540-44480-7_21 .

D. Turner , A. Lucieer , C. Watson . An automated technique for generating georectified mosaics from ultrahigh resolution unmanned aerial vehicle (UAV) imagery, structure from motion (SfM) point clouds . Remote Sens. , 2012 . 4 ( 12 ): 1392 - 1410 . DOI: 10.3390/rs4051392 http://doi.org/10.3390/rs4051392 .

T. Tuytelaars , K. Mikolajczyk . Local invariant feature detectors:a survey . Found. Trends Comput. Graph. Vis. , 2007 . 3 ( 3 ): 177 - 280 . DOI: 10.1561/0600000017 http://doi.org/10.1561/0600000017 .

C. Wang . Extrinsic calibration of a vision sensor mounted on a robot . IEEE Trans. Robot. Autom. , 1992 . 8 ( 2 ): 161 - 175 . DOI: 10.1109/70.134271 http://doi.org/10.1109/70.134271 .

Y. Wang , M. Olano . A Framework for GPU 3D Model Reconstruction Using Structure-from-Motion . ACM SIGGRAPH Posters , 2011 . p.27 DOI: 10.1145/2037715.2037748 http://doi.org/10.1145/2037715.2037748 .

C. Wu , S. Agarwal , B. Curless , S.M. Seitz . Multicore Bundle Adjustment . CVPR , 2011 . p.3057 - 3064 . DOI: 10.1109/CVPR.2011.5995552 http://doi.org/10.1109/CVPR.2011.5995552 .

J. Xiao , T. Fang , P. Tan , P. Zhao , E. Ofek , L. Quan . Image-based facade modeling . ACM Trans. Graph. , 2008 . 27 ( 5 ): Article 161, p.1 - 10 . DOI: 10.1145/1409060.1409114 http://doi.org/10.1145/1409060.1409114 .

M.D. Yang , C.F. Chao , K.S. Huang , L.Y. Lu , Y.P. Chen . Image-based 3D scene reconstruction and exploration in augmented reality . Autom. Construct. , 2013 . 33 48 - 60 . DOI: 10.1016/j.autcon.2012.09.017 http://doi.org/10.1016/j.autcon.2012.09.017 .

A. Yao , A. Calway . Robust Estimation of 3-D Camera Motion for Uncalibrated Augmented Reality . University of Bristol, Bristol, UK , 2002 .

J.S. Zelek , E. Fazl-Ersi , D.C. Asmar , A.H. Fakih . Computer Vision Geo-location, Awareness & Detail . Proc. 1st Int. Conf. and Exhibition on Computing for Geospatial Research & Application , 2010 . DOI: 10.1145/1823854.1823906 http://doi.org/10.1145/1823854.1823906 .

G. Zhang , W. Hua , X. Qin , Y. Shao , H. Bao . Video stabilization based on a 3D perspective camera model . Vis. Comput. , 2009 . 25 ( 11 ): 997 - 1008 . DOI: 10.1007/s00371-009-0310-z http://doi.org/10.1007/s00371-009-0310-z .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰