A graph-based two-stage classification network for mobile screen defect inspection

Chaofan ZHOU; Meiqin LIU; Senlin ZHANG; Ping WEI; Badong CHEN

doi:10.1631/FITEE.2200524

Your Location：

Home >

Browse articles >

A graph-based two-stage classification network for mobile screen defect inspection

Regular Papers | Updated：2023-02-27

- A graph-based two-stage classification network for mobile screen defect inspection
- 用于手机屏缺陷检测的基于图的两阶段分类网络
- Frontiers of Information Technology & Electronic Engineering Vol. 24, Issue 2, Pages: 203-216(2023)
- Affiliations：
  
  1.State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China
  2.College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China
  3.Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an 710049, China
- Author bio：
  
  E-mail: zhouchaofan@zju.edu.cn;
  ‡Corresponding author
  slzhang@zju.edu.cn;
  pingwei@mail.xjtu.edu.cn;
  chenbd@mail.xjtu.edu.cn
- Funds：
- DOI：10.1631/FITEE.2200524
  CLC： TP391.4
- Received：31 October 2022，
  
  Accepted：28 November 2022，
  
  Published：0 February 2023
- Accepted：
Scan QR Code
Chaofan ZHOU, Meiqin LIU, Senlin ZHANG, et al. A graph-based two-stage classification network for mobile screen defect inspection[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(2): 203-216.
DOI：

Chaofan ZHOU, Meiqin LIU, Senlin ZHANG, et al. A graph-based two-stage classification network for mobile screen defect inspection[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(2): 203-216. DOI： 10.1631/FITEE.2200524.

摘要

缺陷检测是手机屏质量控制的重要环节。手机屏缺陷的特性带来了一些具有挑战性的问题，包括：（1）类间相似性和类内差异性；（2）低对比度、微小尺寸或不完整缺陷的识别带来的困难；（3）针对多标签图像的类别相关性建模。为了解决这些问题，本文提出一种图推理模块，它可以堆放在常规的分类模块上。该推理模块利用类别间的依赖性、图像间的关系以及类别图像之间的相互作用来扩展特征维度，并且达到改进低质量图像特征的目的。为了进一步提高分类性能，分类模块的分类器被设计为一个余弦相似度函数。在对比学习的帮助下，分类模块可以更好地初始化推理模块的类别图。在手机屏缺陷数据集上的实验表明，所提出的两阶段网络取得了最佳性能：准确率为97.7%，

-measure为97.3%。这证明了本文所提出的方法在工业应用中是有效的。

Abstract

Defect inspection

also known as defect detection

is significant in mobile screen quality control. There are some challenging issues brought by the characteristics of screen defects

including the following: (1) the problem of interclass similarity and intraclass variation

(2) the difficulty in distinguishing low contrast

tiny-sized

or incomplete defects

and (3) the modeling of category dependencies for multi-label images. To solve these problems

a graph reasoning module

stacked on a classification module

is proposed to expand the feature dimension and improve low-quality image features by exploiting category-wise dependency

image-wise relations

and interactions between them. To further improve the classification performance

the classifier of the classification module is redesigned as a cosine similarity function. With the help of contrastive learning

the classification module can better initialize the category-wise graph of the reasoning module. Experiments on the mobile screen defect dataset show that our two-stage network achieves the following best performances: 97.7% accuracy and 97.3%

-measure. This proves that the proposed approach is effective in industrial applications.

关键词

Keywords

references

Bottou L , 2010 . Large-scale machine learning with stochastic gradient descent . Proc 19 th Int Conf on Computational Statistics , p. 177 - 186 . doi: 10.1007/978-3-7908-2604-3_16 http://doi.org/10.1007/978-3-7908-2604-3_16

Chen T , Kornblith S , Norouzi M , et al. , 2020 . A simple framework for contrastive learning of visual representations . Proc 37 th Int Conf on Machine Learning , p. 1597 - 1607 .

Chen ZM , Wei XS , Wang P , et al. , 2019 . Multi-label image recognition with graph convolutional networks . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 5177 - 5186 . doi: 10.1109/CVPR.2019.00532 http://doi.org/10.1109/CVPR.2019.00532

Gidaris S , Komodakis N , 2018 . Dynamic few-shot visual learning without forgetting . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 4367 - 4375 . doi: 10.1109/CVPR.2018.00459 http://doi.org/10.1109/CVPR.2018.00459

Haurum JB , Moeslund TB , 2021 . Sewer-ML: a multi-label sewer defect classification dataset and benchmark . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 13451 - 13462 . doi: 10.1109/CVPR46437.2021.01325 http://doi.org/10.1109/CVPR46437.2021.01325

He KM , Zhang XY , Ren SQ , et al. , 2016 . Deep residual learning for image recognition . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 770 - 778 . doi: 10.1109/CVPR.2016.90 http://doi.org/10.1109/CVPR.2016.90

He KM , Fan HQ , Wu YX , et al. , 2020 . Momentum contrast for unsupervised visual representation learning . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 9726 - 9735 . doi: 10.1109/CVPR42600.2020.00975 http://doi.org/10.1109/CVPR42600.2020.00975

Hjelm RD , Fedorov A , Lavoie-Marchildon S , et al. , 2019 . Learning deep representations by mutual information estimation and maximization . Proc 7 th Int Conf on Learning Representations .

Khosla P , Teterwak P , Wang C , et al. , 2020 . Supervised contrastive learning . Proc 34 th Conf on Neural Information Processing Systems , p. 18661 - 18673 .

Kong YH , Liu X , Zhao ZB , et al. , 2022 . Bolt defect classification algorithm based on knowledge graph and feature fusion . Energy Rep , 8 ( Suppl 1 ): 856 - 863 . doi: 10.1016/j.egyr.2021.11.127 http://doi.org/10.1016/j.egyr.2021.11.127

Lei J , Gao X , Feng ZL , et al. , 2018 . Scale insensitive and focus driven mobile screen defect detection in industry . Neurocomputing , 294 : 72 - 81 . doi: 10.1016/j.neucom.2018.03.013 http://doi.org/10.1016/j.neucom.2018.03.013

Li CS , Zhang XM , Huang YJ , et al. , 2020 . A novel algorithm for defect extraction and classification of mobile phone screen based on machine vision . Comput Ind Eng , 146 : 106530 . doi: 10.1016/j.cie.2020.106530 http://doi.org/10.1016/j.cie.2020.106530

Lu Y , Ma L , Jiang HQ , 2020 . A light CNN model for defect detection of LCD . In: Hung JC , Yen NY , Chang JW (Eds.), Frontier Computing. Springer , Singapore , p. 10 - 19 . doi: 10.1007/978-981-15-3250-4_2 http://doi.org/10.1007/978-981-15-3250-4_2

Park JY , Hwang Y , Lee D , et al. , 2020 . MarsNet: multi-label classification network for images of various sizes . IEEE Access , 8 : 21832 - 21846 . doi: 10.1109/ACCESS.2020.2969217 http://doi.org/10.1109/ACCESS.2020.2969217

Paszke A , Gross S , Chintala S , et al. , 2017 . Automatic differentiation in PyTorch . Proc 31 st Conf on Neural Information Processing Systems .

Simonyan K , Zisserman A , 2015 . Very deep convolutional networks for large-scale image recognition . Proc 3 rd Int Conf on Learning Representations .

Szegedy C , Vanhoucke V , Ioffe S , et al. , 2016 . Rethinking the inception architecture for computer vision . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 2818 - 2826 . doi: 10.1109/CVPR.2016.308 http://doi.org/10.1109/CVPR.2016.308

Wang T , Zhang C , Ding RW , et al. , 2021 . Mobile phone surface defect detection based on improved faster R-CNN . Proc 25 th Int Conf on Pattern Recognition , p. 9371 - 9377 . doi: 10.1109/ICPR48806.2021.9412119 http://doi.org/10.1109/ICPR48806.2021.9412119

Wang Y , He DL , Li F , et al. , 2020 . Multi-label classification with label graph superimposing . Proc AAAI Conf Artif Intell , 34 ( 7 ): 12265 - 12272 . doi: https://doi.org/10.1609/aaai.v34i07.6909 https://doi.org/10.1609/aaai.v34i07.6909

Wang YC , Gao L , Li XY , et al. , 2021a . A new graph-based method for class imbalance in surface defect recognition . IEEE Trans Instrum Meas , 70 : 5007816 . doi: 10.1109/TIM.2021.3063755 http://doi.org/10.1109/TIM.2021.3063755

Wang YC , Gao L , Gao YP , et al. , 2021b . A new graph-based semi-supervised method for surface defect classification . Rob Comput Integr Manuf , 68 : 102083 . doi: 10.1016/j.rcim.2020.102083 http://doi.org/10.1016/j.rcim.2020.102083

Wei B , Hao KR , Gao L , et al. , 2021 . Bioinspired visual-integrated model for multilabel classification of textile defect images . IEEE Trans Cognit Dev Syst , 13 ( 3 ): 503 - 513 . doi: 10.1109/TCDS.2020.2977974 http://doi.org/10.1109/TCDS.2020.2977974

Wu ZR , Xiong YJ , Yu SX , et al. , 2018 . Unsupervised feature learning via non-parametric instance discrimination . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 3733 - 3742 . doi: 10.1109/CVPR.2018.00393 http://doi.org/10.1109/CVPR.2018.00393

Xiao WW , Song KC , Liu J , et al. , 2022 . Graph embedding and optimal transport for few-shot classification of metal surface defect . IEEE Trans Instrum Meas , 71 : 5010310 . doi: 10.1109/TIM.2022.3169547 http://doi.org/10.1109/TIM.2022.3169547

Xu H , Jiang CH , Liang XD , et al. , 2019 . Reasoning-RCNN: unifying adaptive global reasoning into large-scale object detection . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 6412 - 6421 . doi: 10.1109/CVPR.2019.00658 http://doi.org/10.1109/CVPR.2019.00658

Yuan ZC , Zhang ZT , Su H , et al. , 2018 . Vision-based defect detection for mobile phone cover glass using deep neural networks . Int J Precis Eng Manuf , 19 ( 6 ): 801 - 810 . doi: 10.1007/s12541-018-0096-x http://doi.org/10.1007/s12541-018-0096-x

Zhang JB , Su H , Zou W , et al. , 2021 . CADN: a weakly supervised learning-based category-aware object detection network for surface defect detection . Patt Recogn , 109 : 107571 . doi: 10.1016/j.patcog.2020.107571 http://doi.org/10.1016/j.patcog.2020.107571

Zhao JW , Yan K , Zhao YF , et al. , 2021 . Transformer-based dual relation graph for multi-label image recognition . Proc IEEE/CVF Int Conf on Computer Vision , p. 163 - 172 . doi: 10.1109/ICCV48922.2021.00023 http://doi.org/10.1109/ICCV48922.2021.00023

Zhou BL , Khosla A , Lapedriza A , et al. , 2016 . Learning deep features for discriminative localization . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 2921 - 2929 . doi: 10.1109/CVPR.2016.319 http://doi.org/10.1109/CVPR.2016.319

Zhu Y , Ding RW , Huang WB , et al. , 2022 . HMFCA-Net: hierarchical multi-frequency based channel attention net for mobile phone surface defect detection . Patt Recogn Lett , 153 : 118 - 125 . doi: 10.1016/j.patrec.2021.11.029 http://doi.org/10.1016/j.patrec.2021.11.029

Views

168

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Detecting compromised accounts caused by phone number recycling on e-commerce platforms: taking Meituan as an example

Hybrid embedding and joint training of stacked encoder for opinion question machine reading comprehension

A novel color image encryption algorithm based on a fractional-order discrete chaotic neural network and DNA sequence operations

Adaptive robust neural control of a two-manipulator system holding a rigid object with inaccurate base frame parameters

Supervised topic models with weighted words: multi-label document classification

Related Author

Min GAO

Shutong CHEN

Yangbo GAO

Zhenhua ZHANG

Yu CHEN

Yupeng LI

Yang CHEN

Xin WANG

Related Institution

School of Computer Science, Fudan University

Department of Interactive Media, Hong Kong Baptist University

Meituan

Shanghai Key Lab of Intelligent Information Processing, Fudan University

College of Computer Science and Technology, Zhejiang University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰