FOLLOWUS
1.College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China
2.Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking (IPCAN), Hangzhou 310027, China
E-mail: zhuzhiwei21@zju.edu.cn
xiangg@zju.edu.cn
‡ Corresponding authors
Received:30 April 2024,
Revised:18 September 2024,
Published:2025-05
Scan QR Code
Zhiwei ZHU, Xiang GAO, Lu YU, et al. Neural mesh refinement[J]. Frontiers of information technology & electronic engineering, 2025, 26(5): 695-712.
Zhiwei ZHU, Xiang GAO, Lu YU, et al. Neural mesh refinement[J]. Frontiers of information technology & electronic engineering, 2025, 26(5): 695-712. DOI: 10.1631/FITEE.2400344.
细分是一种广泛使用的网格细化技术。经典方法依赖于固定的手工定义的加权规则,难以生成具有适当细节的更精细网格,而先进的神经细分方法虽然实现了数据驱动的非线性细分,但缺乏鲁棒性,细分级别有限,而且在新形状上会出现伪像。为解决这些问题,提出一种神经网格细化(NMR)方法,该方法从精细形状中学习几何先验,再通过细分自适应地细化粗糙网格,并展示了鲁棒的可泛化性。我们的关键见解是,有必要将网络从非结构信息(如尺度、旋转和平移)中解耦出来,使其能够专注于学习和应用局部补丁的结构先验来进行自适应细化。为此,引入内在结构描述符和局部自适应神经滤波器。内在结构描述符排除非结构信息以对齐局部补丁,从而稳定了输入特征空间,使网络能够鲁棒地提取结构先验。神经滤波器采用图注意机制,提取局部结构特征,并将学习到的先验知识应用于局部补丁。此外,我们观察到,与L2损失相比,Charbonnier损失可以减轻过度平滑。结合这些设计选择,所提方法获得了鲁棒的几何学习和局部自适应能力,增强了对未知形状和任意细化级别的泛化能力。在一组复杂的三维形状上评估了该方法,结果表明它在几何质量方面优于现有细分方法。项目页面见https://zhuzhiwei99.github.io/NeuralMeshRefinement.
Subdivision is a widely used technique for mesh refinement. Classic methods rely on fixed manually defined weighting rules and struggle to generate a finer mesh with appropriate details
while advanced neural subdivision methods achieve data-driven nonlinear subdivision but lack robustness
suffering from limited subdivision levels and artifacts on novel shapes. To address these issues
this paper introduces a neural mesh refinement (NMR) method that uses the geometric structural priors learned from fine meshes to adaptively refine coarse meshes through subdivision
demonstrating robust generalization. Our key insight is that it is necessary to disentangle the network from non-structural information such as scale
rotation
and translation
enabling the network to focus on learning and applying the structural priors of local patches for adaptive refinement. For this purpose
we introduce an intrinsic structure descriptor and a locally adaptive neural filter. The intrinsic structure descriptor excludes the non-structural information to align local patches
thereby stabilizing the input feature space and enabling the network to robustly extract structural priors. The proposed neural filter
using a graph attention mechanism
extracts local structural features and adapts learned priors to local patches. Additionally
we observe that Charbonnier loss can alleviate over-smoothing compared to L2 loss. By combining these design choices
our method gains robust geometric learning and locally adaptive capabilities
enhancing generalization to various situations such as unseen shapes and arbitrary refinement levels. We evaluate our method on a diverse set of complex three-dimensional (3D) shapes
and experimental results show that it outperforms existing subdivision methods in terms of geometry quality. See https://zhuzhiwei99.github.io/NeuralMeshRefinement for the project page.
Abutbul A , Elidan G , Katzir L , et al. , 2020 . DNF-Net: a neural architecture for tabular data . https://doi.org/10.48550/arXiv.2006.06465 https://doi.org/10.48550/arXiv.2006.06465
Atwood J , Towsley D , 2016 . Diffusion-convolutional neural networks . Proc 30 th Int Conf on Neural Information Processing Systems , p. 2001 - 2009 .
Beatty JC , Dyn N , Levine D , et al. , 1990 . A butterfly subdivision scheme for surface interpolation with tension control . ACM Trans Graph , 9 ( 2 ): 160 - 169 . https://doi.org/10.1145/78956.78958 https://doi.org/10.1145/78956.78958
Bronstein AM , Bronstein MM , Kimmel R , 2009 . Numerical Geometry of Non-rigid Shapes . Springer , New York, USA . https://doi.org/10.1007/978-0-387-73301-2 https://doi.org/10.1007/978-0-387-73301-2
Catmull E , Clark J , 1998 . Recursively generated B-spline surfaces on arbitrary topological meshes . Proc Seminal Graphics: Pioneering Efforts That Shaped the Field , p. 183 - 188 . https://doi.org/10.1145/280811.280992 https://doi.org/10.1145/280811.280992
Charbonnier P , Blanc-Féraud L , Aubert G , et al. , 1994 . Two deterministic half-quadratic regularization algorithms for computed imaging . Proc 1 st Int Conf on Image Processing , p. 168 - 172 . https://doi.org/10.1109/ICIP.1994.413553 https://doi.org/10.1109/ICIP.1994.413553
Chen HG , He XH , Qing LB , et al. , 2022 . Real-world single image super-resolution: a brief review . Inform Fus , 79 : 124 - 145 . https://doi.org/10.1016/j.inffus.2021.09.005 https://doi.org/10.1016/j.inffus.2021.09.005
Chen Y , Zhao JY , Shi CW , et al. , 2020 . Mesh convolution: a novel feature extraction method for 3D nonrigid object classification . IEEE Trans Multim , 23 : 3098 - 3111 . https://doi.org/10.1109/TMM.2020.3020693 https://doi.org/10.1109/TMM.2020.3020693
Chen YC , Kim V , Aigerman N , et al. , 2023 . Neural progressive meshes . Proc ACM SIGGRAPH , Article 84 . https://doi.org/10.1145/3588432.3591531 https://doi.org/10.1145/3588432.3591531
Choi Y , Jeong JB , Lee S , et al. , 2022 . Overview of the video-based dynamic mesh coding (V-DMC) standard work . Proc 13 th Int Conf on Information and Communication Technology Convergence , p. 578 - 581 . https://doi.org/10.1109/ICTC55196.2022.9952734 https://doi.org/10.1109/ICTC55196.2022.9952734
Dai A , Nießner M , 2019 . Scan2Mesh: from unstructured range scans to 3D meshes . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 5569 - 5578 . https://doi.org/10.1109/CVPR.2019.00572 https://doi.org/10.1109/CVPR.2019.00572
Dai T , Cai JR , Zhang YB , et al. , 2019 . Second-order attention network for single image super-resolution . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 11057 - 11066 . https://doi.org/10.1109/CVPR.2019.01132 https://doi.org/10.1109/CVPR.2019.01132
DeRose T , Kass M , Truong T , 1998 . Subdivision surfaces in character animation . Proc 25 th Annual Conf on Computer Graphics and Interactive Techniques , p. 85 - 94 . https://doi.org/10.1145/280814.280826 https://doi.org/10.1145/280814.280826
Dong C , Loy CC , Tang XO , 2016a . Accelerating the super-resolution convolutional neural network . Proc 14 th European Conf on Computer Vision , p. 391 - 407 . https://doi.org/10.1007/978-3-319-46475-6_25 https://doi.org/10.1007/978-3-319-46475-6_25
Dong C , Loy CC , He KM , et al. , 2016b . Image super-resolution using deep convolutional networks . IEEE Trans Patt Anal Mach Intell , 38 ( 2 ): 295 - 307 . https://doi.org/10.1109/TPAMI.2015.2439281 https://doi.org/10.1109/TPAMI.2015.2439281
Doo D , Sabin M , 1998 . Behaviour of recursive division surfaces near extraordinary points . Proc Seminal Graphics: Pioneering Efforts That Shaped the Field , p. 177 - 181 . https://doi.org/10.1145/280811.280991 https://doi.org/10.1145/280811.280991
Dyn N , 2006 . Three families of nonlinear subdivision schemes . Stud Comput Math , 12 : 23 - 38 . https://doi.org/10.1016/S1570-579X(06)80003-0 https://doi.org/10.1016/S1570-579X(06)80003-0
Feng YT , Feng YF , You HX , et al. , 2019 . MeshNet: mesh neural network for 3D shape representation . Proc 33 rd AAAI Conf on Artificial Intelligence , p. 8279 - 8286 . https://doi.org/10.1609/aaai.v33i01.33018279 https://doi.org/10.1609/aaai.v33i01.33018279
Garland M , Heckbert PS , 1997 . Surface simplification using quadric error metrics . Proc 24 th Annual Conf on Computer Graphics and Interactive Techniques , p. 209 - 216 . https://doi.org/10.1145/258734.258849 https://doi.org/10.1145/258734.258849
Haim N , Segol N , Ben-Hamu H , et al. , 2019 . Surface networks via general covers . Proc IEEE/CVF Int Conf on Computer Vision , p. 632 - 641 . https://doi.org/10.1109/ICCV.2019.00072 https://doi.org/10.1109/ICCV.2019.00072
Han B , Zhang XY , Ren S , 2022 . PU-GACNet: graph attention convolution network for point cloud upsampling . Image Vis Comput , 118 : 104371 . https://doi.org/10.1016/j.imavis.2021.104371 https://doi.org/10.1016/j.imavis.2021.104371
Hanocka R , Hertz A , Fish N , et al. , 2019 . MeshCNN: a network with an edge . ACM Trans Graph , 38 ( 4 ): 90 . https://doi.org/10.1145/3306346.3322959 https://doi.org/10.1145/3306346.3322959
Haris M , Shakhnarovich G , Ukita N , 2018 . Deep back-projection networks for super-resolution . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 1664 - 1673 . https://doi.org/10.1109/CVPR.2018.00179 https://doi.org/10.1109/CVPR.2018.00179
Hertz A , Hanocka R , Giryes R , et al. , 2020 . Deep geometric texture synthesis . ACM Trans Graph , 39 ( 4 ): 108 . https://doi.org/10.1145/3386569.3392471 https://doi.org/10.1145/3386569.3392471
Hoppe H , DeRose T , Duchamp T , et al. , 1994 . Piecewise smooth surface reconstruction . Proc 21 st Annual Conf on Computer Graphics and Interactive Techniques , p. 295 - 302 . https://doi.org/10.1145/192161.192233 https://doi.org/10.1145/192161.192233
Hu SM , Liu ZN , Guo MH , et al. , 2022 . Subdivision-based mesh convolution networks . ACM Trans Graph , 41 ( 3 ): 25 . https://doi.org/10.1145/3506694 https://doi.org/10.1145/3506694
Hu YX , Schneider T , Wang BL , et al. , 2020 . Fast tetrahedral meshing in the wild . ACM Trans Graph , 39 ( 4 ): 117 . https://doi.org/10.1145/3386569.3392385 https://doi.org/10.1145/3386569.3392385
Huang JW , Zhang HT , Yi L , et al. , 2019 . TextureNet: consistent local parametrizations for learning from high-resolution signals on meshes . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 4435 - 4444 . https://doi.org/10.1109/CVPR.2019.00457 https://doi.org/10.1109/CVPR.2019.00457
Jiao JB , Tu WC , He SF , et al. , 2017 . FormResNet: formatted residual learning for image restoration . Proc IEEE Conf on Computer Vision and Pattern Recognition Workshops , p. 1034 - 1042 . https://doi.org/10.1109/CVPRW.2017.140 https://doi.org/10.1109/CVPRW.2017.140
Karčiauskas K , Peters J , 2007 . Concentric tessellation maps and curvature continuous guided surfaces . Comput Aided Geom Des , 24 ( 2 ): 99 - 111 . https://doi.org/10.1016/j.cagd.2006.10.006 https://doi.org/10.1016/j.cagd.2006.10.006
Kim D , Shin M , Paik J , 2023 . PU-Edgeformer: edge transformer for dense prediction in point cloud upsampling . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , p. 1 - 5 . https://doi.org/10.1109/ICASSP49357.2023.10095541 https://doi.org/10.1109/ICASSP49357.2023.10095541
Kim J , Lee JK , Lee KM , 2016 . Deeply-recursive convolutional network for image super-resolution . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 1637 - 1645 . https://doi.org/10.1109/CVPR.2016.181 https://doi.org/10.1109/CVPR.2016.181
Kingma DP , Ba J , 2015 . Adam: a method for stochastic optimization . Proc 3 rd Int Conf on Learning Representations .
Lahav A , Tal A , 2020 . MeshWalker: deep mesh understanding by random walks . ACM Trans Graph , 39 ( 6 ): 263 . https://doi.org/10.1145/3414685.3417806 https://doi.org/10.1145/3414685.3417806
Lai WS , Huang JB , Ahuja N , et al. , 2017 . Deep Laplacian pyramid networks for fast and accurate super-resolution . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 5835 - 5843 . https://doi.org/10.1109/CVPR.2017.618 https://doi.org/10.1109/CVPR.2017.618
Levie R , Monti F , Bresson X , et al. , 2019 . CayleyNets: graph convolutional neural networks with complex rational spectral filters . IEEE Trans Signal Process , 67 ( 1 ): 97 - 109 . https://doi.org/10.1109/TSP.2018.2879624 https://doi.org/10.1109/TSP.2018.2879624
Levin A , 2006 . Modified subdivision surfaces with continuous curvature . ACM Trans Graph , 25 ( 3 ): 1035 - 1040 . https://doi.org/10.1145/1141911.1141990 https://doi.org/10.1145/1141911.1141990
Li RH , Li XZ , Fu CW , et al. , 2019 . PU-GAN: a point cloud upsampling adversarial network . Proc IEEE/CVF Int Conf on Computer Vision , p. 7202 - 7211 . https://doi.org/10.1109/ICCV.2019.00730 https://doi.org/10.1109/ICCV.2019.00730
Lim B , Son S , Kim H , et al. , 2017 . Enhanced deep residual networks for single image super-resolution . Proc IEEE Conf on Computer Vision and Pattern Recognition Workshops , p. 1132 - 1140 . https://doi.org/10.1109/CVPRW.2017.151 https://doi.org/10.1109/CVPRW.2017.151
Lim I , Dielen A , Campen M , et al. , 2019 . A simple approach to intrinsic correspondence learning on unstructured 3D meshes . Proc European Conf on Computer Vision , p. 349 - 362 . https://doi.org/10.1007/978-3-030-11015-4_26 https://doi.org/10.1007/978-3-030-11015-4_26
Liu HTD , Kim VG , Chaudhuri S , et al. , 2020 . Neural subdivision . ACM Trans Graph , 39 ( 4 ): 124 . https://doi.org/10.1145/3386569.3392418 https://doi.org/10.1145/3386569.3392418
Liu K , Ma N , Wang ZH , et al. , 2023 . Implicit neural distance optimization for mesh neural subdivision . Proc IEEE Int Conf on Multimedia and Expo , p. 2039 - 2044 . https://doi.org/10.1109/ICME55011.2023.00349 https://doi.org/10.1109/ICME55011.2023.00349
Loop C , 1987 . Smooth Subdivision Surfaces Based on Triangles . MS Thesis, University of Utah , Salt Lake City, USA .
Maron H , Galun M , Aigerman N , et al. , 2017 . Convolutional neural networks on surfaces via seamless toric covers . ACM Trans Graph , 36 ( 4 ): 71 . https://doi.org/10.1145/3072959.3073616 https://doi.org/10.1145/3072959.3073616
Milano F , Loquercio A , Rosinol A , et al. , 2020 . Primal-dual mesh convolutional neural networks . Proc 34 th Int Conf on Neural Information Processing Systems , Article 81 .
Monti F , Shchur O , Bojchevski A , et al. , 2018 . Dual-primal graph convolutional networks . https://doi.org/10.48550/arXiv.1806.00770 https://doi.org/10.48550/arXiv.1806.00770
Nair V , Hinton GE , 2010 . Rectified linear units improve restricted Boltzmann machines . Proc 27 th Int Conf on Machine Learning , p. 807 - 814 .
Oliensis J , 1993 . Local reproducible smoothing without shrinkage . IEEE Trans Patt Anal Mach Intell , 15 ( 3 ): 307 - 312 . https://doi.org/10.1109/34.204914 https://doi.org/10.1109/34.204914
Paszke A , Gross S , Massa F , et al. , 2019 . PyTorch: an imperative style, high-performance deep learning library . Proc 33 rd Int Conf on Neural Information Processing Systems , Article 721 .
Perez D , Shen YZ , Li J , 2023 . Mesh convolutional networks with face and vertex feature operators . IEEE Trans Vis Comput Graph , 29 ( 3 ): 1678 - 1690 . https://doi.org/10.1109/TVCG.2021.3129156 https://doi.org/10.1109/TVCG.2021.3129156
Poulenard A , Ovsjanikov M , 2018 . Multi-directional geodesic neural networks via equivariant convolution . ACM Trans Graph , 37 ( 6 ): 236 . https://doi.org/10.1145/3272127.3275102 https://doi.org/10.1145/3272127.3275102
Qi CR , Su H , Mo KC , et al. , 2017 . PointNet: deep learning on point sets for 3D classification and segmentation . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 77 - 85 . https://doi.org/10.1109/CVPR.2017.16 https://doi.org/10.1109/CVPR.2017.16
Qian GC , Abualshour A , Li GH , et al. , 2021 . PU-GCN: point cloud upsampling using graph convolutional networks . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 11678 - 11687 . https://doi.org/10.1109/CVPR46437.2021.01151 https://doi.org/10.1109/CVPR46437.2021.01151
Schaefer S , Vouga E , Goldman R , 2008 . Nonlinear subdivision through nonlinear averaging . Comput Aided Geom Des , 25 ( 3 ): 162 - 180 . https://doi.org/10.1016/j.cagd.2007.07.003 https://doi.org/10.1016/j.cagd.2007.07.003
Sharp N , Attaiki S , Crane K , et al. , 2022 . DiffusionNet: discretization agnostic learning on surfaces . ACM Trans Graph , 41 ( 3 ): 27 . https://doi.org/10.1145/3507905 https://doi.org/10.1145/3507905
Smirnov D , Solomon J , 2021 . HodgeNet: learning spectral geometry on triangle meshes . ACM Trans Graph , 40 ( 4 ): 166 . https://doi.org/10.1145/3450626.3459797 https://doi.org/10.1145/3450626.3459797
Sorkine O , Cohen-Or D , Lipman Y , et al. , 2004 . Laplacian surface editing . Proc Eurographics/ACM SIGGRAPH Symp on Geometry Processing , p. 175 - 184 . https://doi.org/10.1145/1057432.1057456 https://doi.org/10.1145/1057432.1057456
Stam J , 1998 . Exact evaluation of Catmull-Clark subdivision surfaces at arbitrary parameter values . Proc 25 th Annual Conf on Computer Graphics and Interactive Techniques , p. 395 - 404 . https://doi.org/10.1145/280814.280945 https://doi.org/10.1145/280814.280945
Tai Y , Yang J , Liu XM , 2017 . Image super-resolution via deep recursive residual network . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 2790 - 2798 . https://doi.org/10.1109/CVPR.2017.298 https://doi.org/10.1109/CVPR.2017.298
Tatarchenko M , Park J , Koltun V , et al. , 2018 . Tangent convolutions for dense prediction in 3D . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 3887 - 3896 . https://doi.org/10.1109/CVPR.2018.00409 https://doi.org/10.1109/CVPR.2018.00409
Taubin G , 1995 . A signal processing approach to fair surface design . Proc 22 nd Annual Conf on Computer Graphics and Interactive Techniques , p. 351 - 358 . https://doi.org/10.1145/218380.218473 https://doi.org/10.1145/218380.218473
Vaxman A , Müller C , Weber O , 2018 . Canonical Möbius subdivision . ACM Trans Graph , 37 ( 6 ): 227 . https://doi.org/10.1145/3272127.3275007 https://doi.org/10.1145/3272127.3275007
Veličković P , Cucurull G , Casanova A , et al. , 2018 . Graph attention networks . https://doi.org/10.48550/arXiv.1710.10903 https://doi.org/10.48550/arXiv.1710.10903
Wang NY , Zhang YD , Li ZW , et al. , 2018 . Pixel2Mesh: generating 3D mesh models from single RGB images . Proc 15 th European Conf on Computer Vision , p. 55 - 71 . https://doi.org/10.1007/978-3-030-01252-6_4 https://doi.org/10.1007/978-3-030-01252-6_4
Wang XT , Yu K , Wu SX , et al. , 2018 . ESRGAN: enhanced super-resolution generative adversarial networks . Proc European Conf on Computer Vision , p. 63 - 79 . https://doi.org/10.1007/978-3-030-11021-5_5 https://doi.org/10.1007/978-3-030-11021-5_5
Wang Y , Sun YB , Liu ZW , et al. , 2019 . Dynamic graph CNN for learning on point clouds . ACM Trans Graph , 38 ( 5 ): 146 . https://doi.org/10.1145/3326362 https://doi.org/10.1145/3326362
Wang YF , Perazzi F , McWilliams B , et al. , 2018 . A fully progressive approach to single-image super-resolution . IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops . https://doi.org/10.1109/CVPRW.2018.00131 https://doi.org/10.1109/CVPRW.2018.00131
Wang YF , Wu SH , Huang H , et al. , 2019 . Patch-based progressive 3D point set upsampling . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 5951 - 5960 . https://doi.org/10.1109/CVPR.2019.00611 https://doi.org/10.1109/CVPR.2019.00611
Wang ZH , Chen J , Hoi SCH , 2021 . Deep learning for image super-resolution: a survey . IEEE Trans Patt Anal Mach Intell , 43 ( 10 ): 3365 - 3387 .
Yang WM , Zhang XC , Tian YP , et al. , 2019 . Deep learning for single image super-resolution: a brief review . IEEE Trans Multim , 21 ( 12 ): 3106 - 3121 . https://doi.org/10.1109/TMM.2019.2919431 https://doi.org/10.1109/TMM.2019.2919431
Yu LQ , Li XZ , Fu CW , et al. , 2018 . PU-Net: point cloud upsampling network . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 2790 - 2799 . https://doi.org/10.1109/CVPR.2018.00295 https://doi.org/10.1109/CVPR.2018.00295
Zhou QN , Jacobson A , 2016 . Thingi10K: a dataset of 10,000 3D-printing models . https://doi.org/10.48550/arXiv.1605.04797 https://doi.org/10.48550/arXiv.1605.04797
Zorin D , 2007 . Subdivision on arbitrary meshes: algorithms and theory . Proc Mathematics and Computation in Imaging Science and Information Processing , p. 1 - 46 . https://doi.org/10.1142/9789812709066_0001 https://doi.org/10.1142/9789812709066_0001
Zorin D , Schröder P , Sweldens W , 1996 . Interpolating subdivision for meshes with arbitrary topology . Proc 23 rd Annual Conf on Computer Graphics and Interactive Techniques , p. 189 - 192 . https://doi.org/10.1145/237170.237254 https://doi.org/10.1145/237170.237254
Publicity Resources
Related Articles
Related Author
Related Institution