基于多尺度轮廓插值生成准密集时变点云模型序列

Chu-hua HUANG; Dong-ming LU; Chang-yu DIAO

doi:10.1631/FITEE.1500316

Your Location：

Home >

Browse articles >

基于多尺度轮廓插值生成准密集时变点云模型序列

常规文章 | Updated：2022-05-19

- 基于多尺度轮廓插值生成准密集时变点云模型序列
- A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence
- 信息与电子工程前沿（英文） 2016年17卷第5期页码：422-434
- Affiliations：
  
  1College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
  2College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
  3Academy of Cultural Heritage, Zhejiang University, Hangzhou 310058, China
- Author bio：
  
  E-mail: diaochangyu@gmail.com
- Funds：
  
  Project supported by the National Basic Research Program (973) of China (No. 2012CB725305), the Natural Science Foundation of Guizhou Province, China (No. 20132094), the National Social Science Fund, China (No. 12&ZD32), and the Introducing Talents Foundation of Guizhou University, China (No. 2012028)
- DOI：10.1631/FITEE.1500316
  中图分类号： TP391.4
- 收稿：2015-09-28，
  
  录用：2016-2-21，
  
  纸质出版：2016-05
- Accepted：
Scan QR Code
基于多尺度轮廓插值生成准密集时变点云模型序列[J]. 信息与电子工程前沿（英文）, 2016,17(5):422-434.

Chu-hua HUANG, Dong-ming LU, Chang-yu DIAO. A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(5): 422-434.
基于多尺度轮廓插值生成准密集时变点云模型序列[J]. 信息与电子工程前沿（英文）, 2016,17(5):422-434. DOI： 10.1631/FITEE.1500316.

Chu-hua HUANG, Dong-ming LU, Chang-yu DIAO. A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(5): 422-434. DOI： 10.1631/FITEE.1500316.

摘要

目的

提高采用多视角视频序列生成时变三维模型序列的速度，以更好地跟踪三维动态物体的形变过程；同时保证时变三维模型序列的外观质量。

创新点

提出基于多尺度轮廓插值生成时变点云模型序列的方法。利用重建对象轮廓的多尺度时空连续性，完成时变三维模型序列的时空维度插值。

方法

首先，采用基于剪影轮廓原理重建物体关键帧的稀疏三维模型。接着，分析三维模型的轮廓点在点级别、轮廓级别、模型级别的连续性，并在该过程中采用距离图插值来增强轮廓的连续性。然后，采用最近点查找方法获得匹配点对，在三个尺度上对匹配点对进行线性密集化。最后，生成具有良好时空一致性的准密集时变三维模型序列。

结论

利用轮廓多尺度时空连续性能够提高重建对象的形变跟踪速度，且时变三维模型序列具有良好的外观质量。

Abstract

To speed up the reconstruction of 3D dynamic scenes in an ordinary hardware platform

we propose an efficient framework to reconstruct 3D dynamic objects using a multiscale-contour-based interpolation from multi-view videos. Our framework takes full advantage of spatio-temporal-contour consistency. It exploits the property to interpolate single contours

two neighboring contours which belong to the same model

and two contours which belong to the same view at different times

corresponding to point-

contour-

and model-level interpolations

respectively. The framework formulates the interpolation of two models as point cloud transport rather than non-rigid surface deformation. Our framework speeds up the reconstruction of a dynamic scene while improving the accuracy of point-pairing which is used to perform the interpolation. We obtain a higher frame rate

spatio-temporal-coherence

and a quasi-dense point cloud sequence with color information. Experiments with real data were conducted to test the efficiency of the framework.

关键词

Keywords

references

N Ahmed , , , I.N Junejo . . A system for 3D video acquisition and spatio-temporally coherent 3D animation reconstruction using multiple RGB-D cameras . . Int. J. Signal Process. Image Process. Patt. Recogn. , , 2013 . . 6 ( ( 2 ): ): 113 - - 128 . . . .

B Allain , , , J.S Franco , , , R.E Boye . . An efficient volumetric framework for shape tracking . . 2015 . . IEEE Conf. on Computer Vision and Pattern Recognition . . 268 - - 276 . . DOI: 10.1109/CVPR.2015.7298623 http://doi.org/10.1109/CVPR.2015.7298623 . .

D Arita , , , R Taniguchi . . RPV-II: a stream-based realtime parallel vision system and its application to real-time volume reconstruction . . 2001 . . 2nd Int. Workshop on Computer Vision Systems . . 174 - - 189 . . DOI: 10.1007/3-540-48222-9_12 http://doi.org/10.1007/3-540-48222-9_12 . .

B.G Baumgart . . Geometric Modeling for Computer Vision , , 1974 . . : : Stanford University Stanford, USA , , PhD Thesis . .

S.C Bilir , , , Y Yemez . . Non-rigid 3D shape tracking from multiview video . . Comput. Vis. Image Understand. , , 2012 . . 116 ( ( 11 ): ): 1121 - - 1134 . . DOI: 10.1016/j.cviu.2012.07.001 http://doi.org/10.1016/j.cviu.2012.07.001 . .

E Borovikov , , , A Sussman , , , L Davis . . A high performance multi-perspective vision studio . . 2003 . . 17th Annual Int. Conf. on Supercomputing . . 348 - - 357 . . DOI: 10.1145/782814.782862 http://doi.org/10.1145/782814.782862 . .

G.K.M Cheung , , , T Kanade , , , J.Y Bouguet . . A real time system for robust 3D voxel reconstruction of human motions . . 2000 . . IEEE Conf. on Computer Vision and Pattern Recognition . . 714 - - 720 . . DOI: 10.1109/CVPR.2000.854944 http://doi.org/10.1109/CVPR.2000.854944 . .

G.K.M Cheung , , , S Baker , , , T Kanade . . Visual hull alignment and refinement across time: a 3D reconstruction algorithm combining shape-from-silhouette with stereo . . 2003 . . IEEE Computer Society Conf. on Computer Vision and Pattern Recognition . . 375 - - 382 . . DOI: 10.1109/CVPR.2003.1211493 http://doi.org/10.1109/CVPR.2003.1211493 . .

L Daz-Ms , , , R Muoz-Salinas , , , F.J Madrid-Cuevas , , , 等 . . Shape from silhouette using Dempster-Shafer theory . . Patt. Recog. , , 2010 . . 43 ( ( 6 ): ): 2119 - - 2131 . . DOI: 10.1016/j.patcog.2010.01.001 http://doi.org/10.1016/j.patcog.2010.01.001 . .

T Duckworth , , , D.J Roberts . . Accelerated polyhedral visual hulls using OpenCL . . 2011 . . IEEE Virtual Reality Conf. . . 203 - - 204 . . DOI: 10.1109/VR.2011.5759469 http://doi.org/10.1109/VR.2011.5759469 . .

J.S Franco , , , E Boyer . . Efficient polyhedral modeling from silhouettes . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2009 . . 31 ( ( 3 ): ): 414 - - 427 . . DOI: 10.1109/TPAMI.2008.104 http://doi.org/10.1109/TPAMI.2008.104 . .

Y Furukawa , , , J Ponce . . Carved visual hulls for imagebased modeling . . Int. J. Comput. Vis. , , 2009 . . 81 ( ( 1 ): ): 53 - - 67 . . DOI: 10.1007/s11263-008-0134-8 http://doi.org/10.1007/s11263-008-0134-8 . .

Y Furukawa , , , J Ponce . . Accurate, dense, and robust multiview stereopsis . . IEEE Trans. Patt. Anal. Mach. Intell. , , 2010 . . 32 ( ( 8 ): ): 1362 - - 1376 . . DOI: 10.1109/TPAMI.2009.161 http://doi.org/10.1109/TPAMI.2009.161 . .

G Haro . . Shape from silhouette consensus . . Patt. Recogn. , , 2012 . . 45 ( ( 9 ): ): 3231 - - 3244 . . DOI: 10.1016/j.patcog.2012.02.029 http://doi.org/10.1016/j.patcog.2012.02.029 . .

N Hasler , , , B Rosenhahn , , , T Thormahlen , , , 等 . . Markerless motion capture with unsynchronized moving cameras . . 2009 . . IEEE Conf. on Computer Vision and Pattern Recognition . . 224 - - 231 . . DOI: 10.1109/CVPR.2009.5206859 http://doi.org/10.1109/CVPR.2009.5206859 . .

S Hauswiesner , , , R Khlebnikov , , , M Steinberger , , , 等 . . Multi-GPU image-based visual hull rendering . . 2012 . . 12th Eurographics Symp. on Parallel Graphics and Visualization . . 119 - - 128 . . . .

M.H Hofmann , , , M.G Davrila . . Multi-view 3D human pose estimation combining single-frame recovery, temporal integration and model adaptation . . 2009 . . IEEE Conf. on Computer Vision and Pattern Recognition . . 2214 - - 2221 . . DOI: 10.1109/CVPR.2009.5206508 http://doi.org/10.1109/CVPR.2009.5206508 . .

C.H Huang , , , D.M Lu , , , C.Y Diao . . Accelerated visual hulls of complex objects using contribution weights . . 2013 . . Proc. 7th Int. Conf. on Image and Graphics . . 685 - - 689 . . DOI: 10.1109/ICIG.2013.139 http://doi.org/10.1109/ICIG.2013.139 . .

C.H Huang , , , D.M Lu , , , C.Y Diao . . A point cloud representation using plane-space-local-area-colorconsistency . . J. Comput.-Aided Des. Comput. Graph. , , 2014a . . 26 ( ( 8 ): ): 1297 - - 1303 . . . .

C.H Huang , , , E Boyer , , , N Navab , , , 等 . . Human shape and pose tracking using keyframes . . 2014b . . IEEE Conf. on Computer Vision and Pattern Recognition . . 3446 - - 3453 . . DOI: 10.1109/CVPR.2014.440 http://doi.org/10.1109/CVPR.2014.440 . .

A Kanaujia , , , N Haering , , , G Taylor , , , 等 . . 3D human pose and shape estimation from multi-view imagery. . . 2011 . . IEEE Computer Vision and Pattern Recognition Workshops . . 49 - - 56 . . DOI: 10.1109/CVPRW.2011.5981821 http://doi.org/10.1109/CVPRW.2011.5981821 . .

D Kim , , , R Dahyot . . Bayesian shape from silhouettes . . 2012 . . Int. Workshop on Multimedia Understanding Through Semantics, Computation, and Learning . . 78 - - 89 . . DOI: 10.1007/978-3-642-32436-9_7 http://doi.org/10.1007/978-3-642-32436-9_7 . .

A Laurentini . . The visual hull concept for silhouette based image understanding . . IEEE Trans. Patt. Anal. Mach. Intell. , , 1994 . . 16 ( ( 2 ): ): 150 - - 162 . . DOI: 10.1109/34.273735 http://doi.org/10.1109/34.273735 . .

K Li , , , Q.H Dai , , , W.L Xu . . Markerless shape and motion capture from multiview video sequences . . IEEE Trans. Circ. Syst. Video Technol. , , 2011 . . 21 ( ( 3 ): ): 320 - - 334 . . DOI: 10.1109/TCSVT.2011.2106251 http://doi.org/10.1109/TCSVT.2011.2106251 . .

Y.B Liu , , , Q.H Dai , , , W.L Xu . . A point-cloud-based multiview stereo algorithm for free-view-point video . . IEEE Trans. Vis. Comput. Graph. , , 2010 . . 16 ( ( 3 ): ): 407 - - 418 . . DOI: 10.1109/TVCG.2009.88 http://doi.org/10.1109/TVCG.2009.88 . .

T Matsuyama , , , X.J Wu , , , T Takai , , , 等 . . Real-time dynamic 3-D object shape reconstruction and high-fidelity texture mapping for 3-D video . . IEEE Trans. Circ. Syst. Video Technol. , , 2004 . . 14 ( ( 3 ): ): 357 - - 369 . . DOI: 10.1109/TCSVT.2004.823396 http://doi.org/10.1109/TCSVT.2004.823396 . .

W Matusik , , , C Buehler , , , R Raskar , , , 等 . . Image-based visual hulls . . 2000 . . ACM Special Interest Group on Computer Graphics . . 369 - - 374 . . DOI: 10.1145/344779.344951 http://doi.org/10.1145/344779.344951 . .

H Nakajima , , , Y Makihara , , , H Hsu , , , 等 . . Point cloud transport . . 2012 . . 21st Int. Conf. on Pattern Recognition . . 3803 - - 3806 . . . .

M Nakazawa , , , I Mitsugami , , , Y Makihara , , , 等 . . Dynamic scene reconstruction using asynchronous multiple Kinects . . 2012 . . 21st Int. Conf. on Pattern Recognition . . 469 - - 472 . . . .

J.M Perez , , , P.G Aledo , , , P.P Sanchez . . Real-time voxel-based visual hull reconstruction . . Microprocess. Microsyst. , , 2012 . . 36 ( ( 5 ): ): 439 - - 447 . . DOI: 10.1016/j.micpro.2012.05.003 http://doi.org/10.1016/j.micpro.2012.05.003 . .

N., M.R Raeesi , , , Q.M.J Wu . . A complete visual hull representation using bounding edges . . 2010 . . 11th Pacific-Rim Conf. on Multimedia . . 171 - - 182 . . DOI: 10.1007/978-3-642-15702-8_16 http://doi.org/10.1007/978-3-642-15702-8_16 . .

A Taneja , , , L Ballan , , , M Pollefeys . . Modeling dynamic scenes recorded with freely moving cameras . . 2011 . . 10th Asian Conf. on Computer Vision . . 613 - - 626 . . . .

D Vlasic , , , P Peers , , , I Baran , , , 等 . . Dynamic shape capture using multi-view photometric stereo . . ACM Trans. Graph. , , 2009 . . 28 ( ( 5 ): ): 174 DOI: 10.1145/1618452.1618520 http://doi.org/10.1145/1618452.1618520 . .

S.Y Wang , , , H.M Yu . . Convex relaxation for a 3D spatiotemporal segmentation model using the primal-dual method . . J. Zhejiang Univ.-Sci. C (Comput. & Electron.) , , 2012 . . 13 ( ( 6 ): ): 428 - - 439 . . DOI: 10.1631/jzus.C1100331 http://doi.org/10.1631/jzus.C1100331 . .

X.J Wu , , , O Takizawa , , , T Matsuyama . . Parallel pipeline volume intersection for real-time 3D shape reconstruction on a PC cluster . . 2006 . . IEEE Int. Conf. on Computer Vision Systems . . 1 - - 4 . . DOI: 10.1109/ICVS.2006.49 http://doi.org/10.1109/ICVS.2006.49 . .

D Xia , , , D.H Li , , , Q.G Li . . A novel approach for computing exact visual hull from silhouettes . . Optik , , 2011 . . 122 ( ( 24 ): ): 2220 - - 2226 . . DOI: 10.1016/j.ijleo.2011.02.013 http://doi.org/10.1016/j.ijleo.2011.02.013 . .

Z Zhang , , , H.S Seah , , , C.K Quah , , , 等 . . A multiple camera system with real-time volume reconstruction for articulated skeleton pose tracking . . 2011 . . 17th Int. Multimedia Modeling Conf. . . 182 - - 192 . . DOI: 10.1007/978-3-642-17832-0_18 http://doi.org/10.1007/978-3-642-17832-0_18 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

DDUC: an erasure-coded system with decoupled data updating and coding