体育视频摘要中的镜头分类和回放检测

Ali JAVED; Amen ALI KHAN

doi:10.1631/FITEE.2000414

Your Location：

Home >

Browse articles >

体育视频摘要中的镜头分类和回放检测

常规文章 | Updated：2022-05-23

- 体育视频摘要中的镜头分类和回放检测
  Enhanced Publication
- Shot classification and replay detection for sports video summarization
- 信息与电子工程前沿（英文） 2022年23卷第5期页码：790-800
- Affiliations：
  
  Department of Software Engineering, University of Engineering and Technology, Taxila47050, Pakistan
- Author bio：
  
  ‡Corresponding author
- Funds：
  
  Directorate of Advanced Studies, Research & Technological Development, University of Engineering and Technology Taxila(UET/ASRTD/RG-1002-3)
- DOI：10.1631/FITEE.2000414
  中图分类号： TP391
- 纸质出版日期：2022-05-0 ，
  
  收稿日期：2020-08-16，
  
  录用日期：2021-03-25
- Accepted：
Scan QR Code
Ali JAVED, Amen ALI KHAN. 体育视频摘要中的镜头分类和回放检测[J]. 信息与电子工程前沿（英文）, 2022,23(5):790-800.

ALI JAVED, AMEN ALI KHAN. Shot classification and replay detection for sports video summarization. [J]. Frontiers of information technology & electronic engineering, 2022, 23(5): 790-800.
Ali JAVED, Amen ALI KHAN. 体育视频摘要中的镜头分类和回放检测[J]. 信息与电子工程前沿（英文）, 2022,23(5):790-800. DOI： 10.1631/FITEE.2000414.

ALI JAVED, AMEN ALI KHAN. Shot classification and replay detection for sports video summarization. [J]. Frontiers of information technology & electronic engineering, 2022, 23(5): 790-800. DOI： 10.1631/FITEE.2000414.

摘要

由于摄像机、回放速度、光照条件、剪辑效果、比赛结构和类型等方面的差异，体育视频摘要的自动分析具有挑战性。为了解决这些问题，针对户外运动视频，本文提出一种基于镜头分类和回放检测的有效视频摘要框架。准确的镜头分类对于更好地安排输入视频从而进行进一步处理（关键事件或回放检测）是必要的。因此，提出一种基于轻量级卷积神经网络的镜头分类方法。该方法对每一个镜头进行回放检测；特别地，检测出从体育视频中识别出回放片段的连续标识转换帧。为此，提出局部八元模式特征来表示视频帧，并训练极限学习机分为回放或非回放两类。所提框架对于摄像机、回放速度、镜头速度、光照条件、比赛结构、运动类型、广播公司、标识设计和位置、帧转换和剪辑效果具有鲁棒性。基于YouTube体育视频集中的足球、棒球和板球运动对所提框架进行性能评估。实验结果证明所提框架能够可靠地用于户外运动视频摘要的镜头分类和回放检测。

Abstract

Automated analysis of sports video summarization is challenging due to variations in cameras

replay speed

illumination conditions

editing effects

game structure

genre

etc. To address these challenges

we propose an effective video summarization framework based on shot classification and replay detection for field sports videos. Accurate shot classification is mandatory to better structure the input video for further processing

i.e.

key events or replay detection. Therefore

we present a lightweight convolutional neural network based method for shot classification. Then we analyze each shot for replay detection and specifically detect the successive batch of logo transition frames that identify the replay segments from the sports videos. For this purpose

we propose local octa-pattern features to represent video frames and train the extreme learning machine for classification as replay or non-replay frames. The proposed framework is robust to variations in cameras

replay speed

shot speed

illumination conditions

game structure

sports genre

broadcasters

logo designs and placement

frame transitions

and editing effects. The performance of our framework is evaluated on a dataset containing diverse YouTube sports videos of soccer

baseball

and cricket. Experimental results demonstrate that the proposed framework can reliably be used for shot classification and replay detection to summarize field sports videos.

关键词

极限学习机轻量级卷积神经网络局部八元模式镜头分类回放检测视频摘要

Keywords

Extreme learning machineLightweight convolutional neural networkLocal octa-patternsShot classificationReplay detectionVideo summarization

references

Bagheri-Khaligh A, Raziperchikolaei R, Moghaddam ME, 2012. A new method for shot classification in soccer sports video based on SVM classifier. Proc IEEE Southwest Symp on Image Analysis and Interpretation, p.109-112. doi: 10.1109/SSIAI.2012.6202465http://doi.org/10.1109/SSIAI.2012.6202465

Chen CM, Chen LH, 2014. Novel framework for sports video analysis: a basketball case study. Proc Int Conf on Image Processing, p.961-965. doi: 10.1109/ICIP.2014.7025193http://doi.org/10.1109/ICIP.2014.7025193

Chen CM, Chen LH, 2015. A novel method for slow motion replay detection in broadcast basketball video. Multimed Tools Appl, 74(21):9573-9593. doi: 10.1007/s11042-014-2137-5http://doi.org/10.1007/s11042-014-2137-5

Choroś K, Gogol A, 2016. Improved method of detecting replay logo in sports videos based on contrast feature and histogram difference. Proc 8th Int Conf on Computational Collective Intelligence, p.542-552. doi: 10.1007/978-3-319-45243-2_50http://doi.org/10.1007/978-3-319-45243-2_50

Dang ZH, Du J, Huang QM, et al., 2007. Replay detection based on semi-automatic logo template sequence extraction in sports video. Proc 4th Int Conf on Image and Graphics, p.839-844. doi: 10.1109/ICIG.2007.73http://doi.org/10.1109/ICIG.2007.73

Duan LY, Xu M, Tian Q, et al., 2004. Mean shift-based video segment representation and applications to replay detection. Proc 29th IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.709-712. doi: 10.1109/ICASSP.2004.1327209http://doi.org/10.1109/ICASSP.2004.1327209

Ekin A, Tekalp AM, Mehrotra R, 2003. Automatic soccer video analysis and summarization. IEEE Trans Image Process, 12(7):796-807. doi: 10.1109/TIP.2003.812758http://doi.org/10.1109/TIP.2003.812758

Eldib MY, Zaid BSA, Zawbaa HM, et al., 2009. Soccer video summarization using enhanced logo detection. Proc 16th IEEE Int Conf on Image Processing, p.4345-4348. doi: 10.1109/ICIP.2009.5413649http://doi.org/10.1109/ICIP.2009.5413649

Fani M, Yazdi M, Clausi DA, et al., 2017. Soccer video structure analysis by parallel feature fusion network and hidden-to-observable transferring Markov model. IEEE Access, 5:27322-27336. doi: 10.1109/ACCESS.2017.2769140http://doi.org/10.1109/ACCESS.2017.2769140

Javed A, Bajwa KB, Malik H, et al., 2016. An efficient framework for automatic highlights generation from sports videos. IEEE Signal Process Lett, 23(7):954-958. doi: 10.1109/LSP.2016.2573042http://doi.org/10.1109/LSP.2016.2573042

Javed A, Irtaza A, Khaliq Y, et al., 2019. Replay and keyevents detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine. Appl Intell, 49(8):2899-2917. doi: 10.1007/s10489-019-01410-xhttp://doi.org/10.1007/s10489-019-01410-x

Javed A, Malik KM, Irtaza A, et al., 2020. A decision tree framework for shot classification of field sports videos. J Supercomput, 76(9):7242-7267. doi: 10.1007/s11227-020-03155-8http://doi.org/10.1007/s11227-020-03155-8

Jiang H, Zhang M, 2011. Tennis video shot classification based on support vector machine. Proc IEEE Int Conf on Computer Science and Automation Engineering, p.757-761. doi: 10.1109/CSAE.2011.5952612http://doi.org/10.1109/CSAE.2011.5952612

Kapela R, McGuinness K, O’Connor NE, 2017. Real-time field sports scene classification using colour and frequency space decompositions. J Real-Time Image Process, 13(4):725-737. doi: 10.1007/s11554-014-0437-7http://doi.org/10.1007/s11554-014-0437-7

Li W, Chen SJ, Wang HB, 2009. A rule-based sports video event detection method. Proc Int Conf on Computational Intelligence and Software Engineering, p.1-4. doi: 10.1109/CISE.2009.5366226http://doi.org/10.1109/CISE.2009.5366226

Minhas RA, Javed A, Irtaza A, et al., 2019. Shot classification of field sports videos using AlexNet convolutional neural network. Appl Sci, 9(3):483. doi: 10.3390/app9030483http://doi.org/10.3390/app9030483

Murala S, Maheshwari RP, Balasubramanian R, 2012. Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process, 21(5):2874-2886. doi: 10.1109/TIP.2012.2188809http://doi.org/10.1109/TIP.2012.2188809

Nguyen N, Yoshitaka A, 2012. Shot type and replay detection for soccer video parsing. Proc IEEE Int Symp on Multimedia, p.344-347. doi: 10.1109/ISM.2012.69http://doi.org/10.1109/ISM.2012.69

Pan H, Van Beek P, Sezan MI, 2001. Detection of slow-motion replay segments in sports video for highlights generation. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.1649-1652. doi: 10.1109/ICASSP.2001.941253http://doi.org/10.1109/ICASSP.2001.941253

Pan H, Li BX, Sezan MI, 2002. Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.IV-3385-IV-3388. doi: 10.1109/ICASSP.2002.5745380http://doi.org/10.1109/ICASSP.2002.5745380

Raventós A, Quijada R, Torres L, et al., 2015. Automatic summarization of soccer highlights using audio-visual descriptors. SpringerPlus, 4(1):301. doi: 10.1186/s40064-015-1065-9http://doi.org/10.1186/s40064-015-1065-9

Su PC, Lan CH, Wu CS, et al., 2013. Transition effect detection for extracting highlights in baseball videos. EURASIP J Image Video Process, 2013(1):27. doi: 10.1186/1687-5281-2013-27http://doi.org/10.1186/1687-5281-2013-27

Tavassolipour M, Karimian M, Kasaei S, 2014. Event detection and summarization in soccer videos using Bayesian network and copula. IEEE Trans Circ Syst Video Technol, 24(2):291-304. doi: 10.1109/TCSVT.2013.2243640http://doi.org/10.1109/TCSVT.2013.2243640

Tien MC, Chen HT, Chen YW, et al., 2007. Shot classification of basketball videos and its application in shooting position extraction. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1085-1088. doi: 10.1109/ICASSP.2007.366100http://doi.org/10.1109/ICASSP.2007.366100

Wang DH, Tian Q, Gao S, et al., 2004. News sports video shot classification with sports play field and motion features. Proc IEEE Conf on Image Processing, p.2247-2250. doi: 10.1109/ICIP.2004.1421545http://doi.org/10.1109/ICIP.2004.1421545

Wang JJ, Chang E, Xu CS, 2005. Soccer replay detection using scene transition structure analysis. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.433-436. doi: 10.1109/ICASSP.2005.1415434http://doi.org/10.1109/ICASSP.2005.1415434

Wang L, Liu X, Lin S, et al., 2004. Generic slow-motion replay detection in sports video. Proc Int Conf on Image Processing, p.1585-1588. doi: 10.1109/ICIP.2004.1421370http://doi.org/10.1109/ICIP.2004.1421370

Wu X, He R, Sun ZN, et al., 2018. A light CNN for deep face representation with noisy labels. IEEE Trans Inform Forens Secur, 13(11):2884-2896. doi: 10.1109/TIFS.2018.2833032http://doi.org/10.1109/TIFS.2018.2833032

Xu W, Yi Y, 2011. A robust replay detection algorithm for soccer video. IEEE Signal Process Lett, 18(9):509-512. doi: 10.1109/LSP.2011.2161287http://doi.org/10.1109/LSP.2011.2161287

Zhao F, Dong Y, Wei Z, et al., 2012. Matching logos for slow motion replay detection in broadcast sports video. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1409-1412. doi: 10.1109/ICASSP.2012.6288154http://doi.org/10.1109/ICASSP.2012.6288154

Zhao Z, Jiang SQ, Huang QM, et al., 2006. Highlight summarization in sports video based on replay detection. Proc Int Conf on Multimedia and Expo, p.1613-1616. doi: 10.1109/ICME.2006.262855http://doi.org/10.1109/ICME.2006.262855

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Video summarization with a graph convolutional attention network

An effective fault prediction model developed using an extreme learning machine with various kernel methods