A non-Lambertian photometric stereo under perspective projection

Min LI; Chang-yu DIAO; Duan-qing XU; Wei XING; Dong-ming LU

doi:10.1631/FITEE.1900156

Your Location：

Home >

Browse articles >

A non-Lambertian photometric stereo under perspective projection

Regular Papers | Updated：2022-05-19

- A non-Lambertian photometric stereo under perspective projection
- 透视投影下的非朗伯光度立体技术
- Frontiers of Information Technology & Electronic Engineering Vol. 21, Issue 8, Pages: 1191-1205(2020)
- Affiliations：
  
  College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
  School of Art and Archeology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  [ "Min LI, E-mail: liminlim@126.com" ]
  Chang-yu DIAO, E-mail:dcy@zju.edu.cn
- Funds：
- DOI：10.1631/FITEE.1900156
  CLC： TP391.4
- Received：19 March 2019，
  
  Revised：2019-11-12，
  
  Published Online：21 December 2019，
  
  Published：2020-08
- Accepted：
Scan QR Code
Min LI, Chang-yu DIAO, Duan-qing XU, et al. A non-Lambertian photometric stereo under perspective projection[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1191-1205.
DOI：

Min LI, Chang-yu DIAO, Duan-qing XU, et al. A non-Lambertian photometric stereo under perspective projection[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1191-1205. DOI： 10.1631/FITEE.1900156.

摘要

在透视投影假设下，非朗伯光度立体技术是一个高度非线性问题。本文提出一种透视投影下基于非朗伯反射模型的表面法线和深度重构优化框架。将图像分解为漫反射分量和镜面分量，可同时计算表面法向量和反射率。此外提出一个变分公式，其在表面重构中表现出鲁棒性与有益性。实验结果表明，所提方法能准确重构非朗伯曲面彩色物体表面形状并计算出物体表面的反射率。

Abstract

Under the perspective projection assumption

non-Lambertian photometric stereo is a highly non-linear problem. In this study

we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection. By decomposing the images into diffuse and specular components

we compute the surface normal and reflectance simultaneously. We also propose a variational formulation that is robust and useful for surface reconstruction. The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.

关键词

Keywords

references

A Agrawal , , , R Raskar , , , SK Nayar , , , 等 . . Removing photography artifacts using gradient projection and flash-exposure sampling . . ACM Trans Graph , , 2005 . . 24 ( ( 3 ): ): 828 - - 835 . . DOI: 10.1145/1073204.1073269 http://doi.org/10.1145/1073204.1073269 . .

A Agrawal , , , R Raskar , , , R Chellappa . . Edge suppression by gradient field transformation using cross-projection tensors . . IEEE Computer Society Conf on Computer Vision and Pattern Recognition , , 2006a . . p.2301 - - 2308 . . DOI: 10.1109/CVPR.2006.106 http://doi.org/10.1109/CVPR.2006.106 . .

A Agrawal , , , R Raskar , , , R Chellappa . . What is the range of surface reconstructions from a gradient field? . . 9 th European Conf on Computer Vision , , 2006b . . p.578 - - 591 . . DOI: 10.1007/11744023_45 http://doi.org/10.1007/11744023_45 . .

S Barsky , , , M Petrou . . The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows . . IEEE Trans Patt Anal Mach Intell , , 2003 . . 25 ( ( 10 ): ): 1239 - - 1252 . . DOI: 10.1109/TPAMI.2003.1233898 http://doi.org/10.1109/TPAMI.2003.1233898 . .

M Chandraker , , , S Agarwal , , , D Kriegman . . Shadowcuts: photometric stereo with shadows . . IEEE Conf on Computer Vision and Pattern Recognition , , 2007 . . p.1 - - 8 . . DOI: 10.1109/CVPR.2007.383288 http://doi.org/10.1109/CVPR.2007.383288 . .

HS Chung , , , JY Jia . . Efficient photometric stereo on glossy surfaces with wide specular lobes . . IEEE Conf on Computer Vision and Pattern Recognition , , 2008 . . p.1 - - 8 . . DOI: 10.1109/CVPR.2008.4587771 http://doi.org/10.1109/CVPR.2008.4587771 . .

ENJr Coleman , , , R Jain . . Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry . . Comput Graph Image Process , , 1982 . . 18 ( ( 4 ): ): 309 - - 328 . . DOI: 10.1016/0146-664X(82)90001-6 http://doi.org/10.1016/0146-664X(82)90001-6 . .

DL Donoho . . Compressed sensing . . IEEE Trans Inform Theory , , 2006 . . 52 ( ( 4 ): ): 1289 - - 1306 . . DOI: 10.1109/TIT.2006.871582 http://doi.org/10.1109/TIT.2006.871582 . .

CH Esteban , , , G Vogiatzis , , , R Cipolla . . Multiview photometric stereo . . IEEE Trans Patt Anal Mach Intell , , 2008 . . 30 ( ( 3 ): ): 548 - - 554 . . DOI: 10.1109/TPAMI.2007.70820 http://doi.org/10.1109/TPAMI.2007.70820 . .

GD Finlayson , , , G Schaefer . . Convex and non-convex illuminant constraints for dichromatic colour constancy . . Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition , , 2001a . . p.598 - - 604 . . DOI: 10.1109/CVPR.2001.990528 http://doi.org/10.1109/CVPR.2001.990528 . .

GD Finlayson , , , G Schaefer . . Solving for colour constancy using a constrained dichromatic reflection model . . Int J Comput Vis , , 2001b . . 42 ( ( 3 ): ): 127 - - 144 . . DOI: 10.1023/A:1011120214885 http://doi.org/10.1023/A:1011120214885 . .

D Gabay , , , B Mercier . . A dual algorithm for the solution of nonlinear variational problems via finite element approximation . . Comput Math Appl , , 1976 . . 2 ( ( 1 ): ): 17 - - 40 . . DOI: 10.1016/0898-1221(76)90003-1 http://doi.org/10.1016/0898-1221(76)90003-1 . .

S Galliani , , , K Lasinger , , , K Schindler . . Massively parallel multiview stereopsis by surface normal diffusion . . Proc IEEE Int Conf on Computer Vision , , 2015 . . p.873 - - 881 . . DOI: 10.1109/ICCV.2015.106 http://doi.org/10.1109/ICCV.2015.106 . .

AS Georghiades . . Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo . . Proc 9 th IEEE Int Conf on Computer Vision, Article 816 , , 2003 . . DOI: 10.1109/ICCV.2003.1238432 http://doi.org/10.1109/ICCV.2003.1238432 . .

A Ghosh , , , TB Chen , , , P Peers , , , 等 . . Estimating specular roughness and anisotropy from second order spherical gradient illumination . . Comput Graph For , , 2009 . . 28 ( ( 4 ): ): 1161 - - 1170 . . DOI: 10.1111/j.1467-8659.2009.01493.x http://doi.org/10.1111/j.1467-8659.2009.01493.x . .

DB Goldman , , , B Curless , , , A Hertzmann , , , 等 . . Shape and spatially-varying BRDFs from photometric stereo . . IEEE Trans Patt Anal Mach Intell , , 2010 . . 32 ( ( 6 ): ): 1060 - - 1071 . . DOI: 10.1109/TPAMI.2009.102 http://doi.org/10.1109/TPAMI.2009.102 . .

XJ Guo , , , XC Cao , , , Y Ma . . Robust separation of reflection from multiple images . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2014 . . p.2187 - - 2194 . . DOI: 10.1109/CVPR.2014.281 http://doi.org/10.1109/CVPR.2014.281 . .

BJ Han , , , JY Sim . . Glass reflection removal using co-saliency-based image alignment and low-rank matrix completion in gradient domain . . IEEE Trans Image Process , , 2018 . . 27 ( ( 10 ): ): 4873 - - 4888 . . DOI: 10.1109/TIP.2018.2849880 http://doi.org/10.1109/TIP.2018.2849880 . .

TQ Han , , , HL Shen . . Photometric stereo for general BRDFs via reflection sparsity modeling . . IEEE Trans Image Process , , 2015 . . 24 ( ( 12 ): ): 4888 - - 4903 . . DOI: 10.1109/TIP.2015.2471081 http://doi.org/10.1109/TIP.2015.2471081 . .

XT He , , , YX Peng . . Fine-grained visual-textual representation learning . . IEEE Trans Circ Syst Video Technol, online , , 2019 . . DOI: 10.1109/TCSVT.2019.2892802 http://doi.org/10.1109/TCSVT.2019.2892802 . .

XT He , , , YX Peng , , , JJ Zhao . . Fast fine-grained image classification via weakly supervised discriminative localization . . IEEE Trans Circ Syst Video Technol , , 2019a . . 29 ( ( 5 ): ): 1394 - - 1407 . . DOI: 10.1109/TCSVT.2018.2834480 http://doi.org/10.1109/TCSVT.2018.2834480 . .

XT He , , , YX Peng , , , JJ Zhao . . Which and how many regions to gaze: focus discriminative regions for finegrained visual categorization . . Int J Comput Vis , , 2019b . . 127 ( ( 9 ): ): 1235 - - 1255 . . DOI: 10.1007/s11263-019-01176-2 http://doi.org/10.1007/s11263-019-01176-2 . .

CP Huynh , , , A Robles-Kelly . . A solution of the dichromatic model for multispectral photometric invariance . . Int J Comput Vis , , 2010 . . 90 ( ( 1 ): ): 1 - - 27 . . DOI: 10.1007/s11263-010-0333-y http://doi.org/10.1007/s11263-010-0333-y . .

S Ikehata . . CNN-PS: CNN-based photometric stereo for general non-convex surfaces . . European Conf on Computer Vision , , 2018 . . p.3 - - 19 . . DOI: 10.1007/978-3-030-01267-0_1 http://doi.org/10.1007/978-3-030-01267-0_1 . .

S Ikehata , , , K Aizawa . . Photometric stereo using constrained bivariate regression for general isotropic surfaces . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2014 . . p.2179 - - 2186 . . DOI: 10.1109/CVPR.2014.280 http://doi.org/10.1109/CVPR.2014.280 . .

S Ikehata , , , D Wipf , , , Y Matsushita , , , 等 . . Robust photometric stereo using sparse regression . . IEEE Conf on Computer Vision and Pattern Recognition , , 2012 . . p.318 - - 325 . . DOI: 10.1109/CVPR.2012.6247691 http://doi.org/10.1109/CVPR.2012.6247691 . .

N Kong , , , YW Tai , , , SY Shin . . A physically-based approach to reflection separation . . IEEE Conf on Computer Vision and Pattern Recognition , , 2012 . . p.9 - - 16 . . DOI: 10.1109/CVPR.2012.6247652 http://doi.org/10.1109/CVPR.2012.6247652 . .

A Levin , , , Y Weiss . . User assisted separation of reflections from a single image using a sparsity prior . . IEEE Trans Patt Anal Mach Intell , , 2007 . . 29 ( ( 9 ): ): 1647 - - 1654 . . DOI: 10.1109/TPAMI.2007.1106 http://doi.org/10.1109/TPAMI.2007.1106 . .

Y Li , , , MS Brown . . Exploiting reflection change for automatic reflection removal . . Proc IEEE Int Conf on Computer Vision , , 2013 . . p.2432 - - 2439 . . DOI: 10.1109/ICCV.2013.302 http://doi.org/10.1109/ICCV.2013.302 . .

SP Mallick , , , TE Zickler , , , DJ Kriegman , , , 等 . . Beyond Lambert: reconstructing specular surfaces using color . . IEEE Computer Society Conf on Computer Vision and Pattern Recognition , , 2005 . . p.619 - - 626 . . DOI: 10.1109/CVPR.2005.88 http://doi.org/10.1109/CVPR.2005.88 . .

SK Nayar , , , XS Fang , , , T Boult . . Separation of reflection components using color and polarization . . Int J Comput Vis , , 1997 . . 21 ( ( 3 ): ): 163 - - 186 . . DOI: 10.1023/A:1007937815113 http://doi.org/10.1023/A:1007937815113 . .

T Papadhimitri , , , P Favaro . . A new perspective on uncalibrated photometric stereo . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2013 . . p.1474 - - 1481 . . DOI: 10.1109/CVPR.2013.194 http://doi.org/10.1109/CVPR.2013.194 . .

YX Peng , , , XT He , , , JJ Zhao . . Object-part attention model for fine-grained image classification . . IEEE Trans Image Process , , 2018 . . 27 ( ( 3 ): ): 1487 - - 1500 . . DOI: 10.1109/TIP.2017.2774041 http://doi.org/10.1109/TIP.2017.2774041 . .

Y Quéau , , , T Wu , , , F Lauze , , , 等 . . A non-convex variational approach to photometric stereo under inaccurate lighting . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2017 . . p.350 - - 359 . . DOI: 10.1109/CVPR.2017.45 http://doi.org/10.1109/CVPR.2017.45 . .

Y Quéau , , , B Durix , , , T Wu , , , 等 . . LED-based photometric stereo: modeling, calibration and numerical solution . . J Math Imag Vis , , 2018 . . 60 ( ( 3 ): ): 313 - - 340 . . DOI: 10.1007/s10851-017-0761-1 http://doi.org/10.1007/s10851-017-0761-1 . .

H Santo , , , M Samejima , , , Y Sugano , , , 等 . . Deep photometric stereo network . . Proc IEEE Int Conf on Computer Vision , , 2017 . . p.501 - - 509 . . DOI: 10.1109/ICCVW.2017.66 http://doi.org/10.1109/ICCVW.2017.66 . .

YY Schechner , , , N Kiryati , , , R Basri . . Separation of transparent layers using focus . . Int J Comput Vis , , 2000 . . 39 ( ( 1 ): ): 25 - - 39 . . DOI: 10.1023/A:1008166017466 http://doi.org/10.1023/A:1008166017466 . .

SA Shafer . . Using color to separate reflection components . . Color Res Appl , , 1985 . . 10 ( ( 4 ): ): 210 - - 218 . . DOI: 10.1002/col.5080100409 http://doi.org/10.1002/col.5080100409 . .

HL Shen , , , TQ Han , , , CG Li . . Efficient photometric stereo using kernel regression . . IEEE Trans Image Process , , 2017 . . 26 ( ( 1 ): ): 439 - - 451 . . DOI: 10.1109/TIP.2016.2627805 http://doi.org/10.1109/TIP.2016.2627805 . .

BX Shi , , , P Tan , , , Y Matsushita , , , 等 . . Bi-polynomial modeling of low-frequency reflectances . . IEEE Trans Patt Anal Mach Intell , , 2014 . . 36 ( ( 6 ): ): 1078 - - 1091 . . DOI: 10.1109/TPAMI.2013.196 http://doi.org/10.1109/TPAMI.2013.196 . .

BX Shi , , , ZP Mo , , , Z Wu , , , 等 . . A benchmark dataset and evaluation for non-Lambertian and uncalibrated photometric stereo . . IEEE Trans Patt Anal Mach Intell , , 2019 . . 41 ( ( 2 ): ): 271 - - 284 . . DOI: 10.1109/TPAMI.2018.2799222 http://doi.org/10.1109/TPAMI.2018.2799222 . .

Y Shih , , , D Krishnan , , , F Durand , , , 等 . . Reflection removal using ghosting cues . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.3193 - - 3201 . . DOI: 10.1109/CVPR.2015.7298939 http://doi.org/10.1109/CVPR.2015.7298939 . .

F Solomon , , , K Ikeuchi . . Extracting the shape and roughness of specular lobe objects using four light photometric stereo . . IEEE Trans Patt Anal Mach Intell , , 1996 . . 18 ( ( 4 ): ): 449 - - 454 . . DOI: 10.1109/34.491627 http://doi.org/10.1109/34.491627 . .

C Sun , , , SC Liu , , , TT Yang , , , 等 . . Automatic reflection removal using gradient intensity and motion cues . . Proc 24 th ACM Int Conf on Multimedia , , 2016 . . p.466 - - 470 . . DOI: 10.1145/2964284.2967264 http://doi.org/10.1145/2964284.2967264 . .

K Sunkavalli , , , T Zickler , , , H Pfister . . Visibility subspaces: uncalibrated photometric stereo with shadows . . 11 th European Conf on Computer Vision , , 2010 . . p.251 - - 264 . . DOI: 10.1007/978-3-642-15552-9_19 http://doi.org/10.1007/978-3-642-15552-9_19 . .

RT Tan , , , K Ikeuchi . . Separating reflection components of textured surfaces using a single image . . IEEE Trans Patt Anal Mach Intell , , 2008 . . 27 ( ( 2 ): ): 353 - - 384 . . DOI: 10.1109/TPAMI.2005.36 http://doi.org/10.1109/TPAMI.2005.36 . .

A Tankus , , , N Kiryati . . Photometric stereo under perspective projection . . 10 th IEEE Int Conf on Computer Vision , , 2005 . . p.611 - - 616 . . DOI: 10.1109/ICCV.2005.190 http://doi.org/10.1109/ICCV.2005.190 . .

RJ Wan , , , BX Shi , , , TA Hwee , , , 等 . . Depth of field guided reflection removal . . IEEE Int Conf on Image Processing , , 2016 . . p.21 - - 25 . . DOI: 10.1109/ICIP.2016.7532311 http://doi.org/10.1109/ICIP.2016.7532311 . .

RJ Wan , , , BX Shi , , , LY Duan , , , 等 . . CRRN: multi-scale guided concurrent reflection removal network . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2018 . . p.4777 - - 4785 . . http://arxiv.org/abs/1805.11802 http://arxiv.org/abs/1805.11802 , , . .

F Wang , , , S Ainouz , , , C Petitjean , , , 等 . . Specularity removal: a global energy minimization approach based on polarization imaging . . Comput Vis Image Underst , , 2017 . . 158 31 - - 39 . . DOI: 10.1016/j.cviu.2017.03.003 http://doi.org/10.1016/j.cviu.2017.03.003 . .

RJ Woodham . . Photometric stereo: a reflectance map technique for determining surface orientation from image intensity . . 22 nd Annual Technical Symp on Image Understanding Systems and Industrial Applications I , , 1979 . . p.136 - - 143 . . DOI: 10.1117/12.956740 http://doi.org/10.1117/12.956740 . .

L Wu , , , A Ganesh , , , B Shi , , , 等 . . Robust photometric stereo via low-rank matrix completion and recovery . . 10 th Asian Conf on Computer Vision , , 2010 . . p.703 - - 717 . . DOI: 10.1007/978-3-642-19318-7_55 http://doi.org/10.1007/978-3-642-19318-7_55 . .

J Yang , , , D Gong , , , LQ Liu , , , 等 . . Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal . . Proc European Conf on Computer Vision , , 2018 . . p.654 - - 669 . . DOI: 10.1007/978-3-030-01219-9_40 http://doi.org/10.1007/978-3-030-01219-9_40 . .

QX Yang , , , JH Tang , , , N Ahuja . . Efficient and robust specular highlight removal . . IEEE Trans Patt Anal Mach Intell , , 2015 . . 37 ( ( 6 ): ): 1304 - - 1311 . . DOI: 10.1109/TPAMI.2014.2360402 http://doi.org/10.1109/TPAMI.2014.2360402 . .

SK Yeung , , , TP Wu , , , CK Tang , , , 等 . . Normal estimation of a transparent object using a video . . IEEE Trans Patt Anal Mach Intell , , 2015 . . 37 ( ( 4 ): ): 890 - - 897 . . DOI: 10.1109/TPAMI.2014.2346195 http://doi.org/10.1109/TPAMI.2014.2346195 . .

C Yu , , , Y Seo , , , SW Lee . . Photometric stereo from maximum feasible Lambertian reflections . . 11 th European Conf on Computer Vision , , 2010 . . p.115 - - 126 . . DOI: 10.1007/978-3-642-15561-1_9 http://doi.org/10.1007/978-3-642-15561-1_9 . .

XE Zhang , , , R Ng , , , QF Chen . . Single image reflection separation with perceptual losses . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2018 . . p.4786 - - 4794 . . DOI: 10.1109/CVPR.2018.00503 http://doi.org/10.1109/CVPR.2018.00503 . .

ZL Zhou , , , Z Wu , , , P Tan . . Multi-view photometric stereo with spatially varying isotropic materials . . IEEE Conf on Computer Vision and Pattern Recognition , , 2013 . . p.1482 - - 1489 . . DOI: 10.1109/CVPR.2013.195 http://doi.org/10.1109/CVPR.2013.195 . .

Views

308

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰