Shot classification and replay detection for sports video summarization

Ali JAVED; Amen ALI KHAN

doi:10.1631/FITEE.2000414

Your Location：

Home >

Browse articles >

Shot classification and replay detection for sports video summarization

Regular Papers | Updated：2022-05-23

- Shot classification and replay detection for sports video summarization
- 体育视频摘要中的镜头分类和回放检测
- Frontiers of Information Technology & Electronic Engineering Vol. 23, Issue 5, Pages: 790-800(2022)
- Affiliations：
  
  Department of Software Engineering, University of Engineering and Technology, Taxila47050, Pakistan
- Author bio：
  
  ‡Corresponding author
- Funds：
- DOI：10.1631/FITEE.2000414
  CLC： TP391
- Received：16 August 2020，
  
  Accepted：25 March 2021，
  
  Published：0 May 2022
- Accepted：
Scan QR Code
Ali JAVED, Amen ALI KHAN. Shot classification and replay detection for sports video summarization[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(5): 790-800.
DOI：

Ali JAVED, Amen ALI KHAN. Shot classification and replay detection for sports video summarization[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(5): 790-800. DOI： 10.1631/FITEE.2000414.

摘要

由于摄像机、回放速度、光照条件、剪辑效果、比赛结构和类型等方面的差异，体育视频摘要的自动分析具有挑战性。为了解决这些问题，针对户外运动视频，本文提出一种基于镜头分类和回放检测的有效视频摘要框架。准确的镜头分类对于更好地安排输入视频从而进行进一步处理（关键事件或回放检测）是必要的。因此，提出一种基于轻量级卷积神经网络的镜头分类方法。该方法对每一个镜头进行回放检测；特别地，检测出从体育视频中识别出回放片段的连续标识转换帧。为此，提出局部八元模式特征来表示视频帧，并训练极限学习机分为回放或非回放两类。所提框架对于摄像机、回放速度、镜头速度、光照条件、比赛结构、运动类型、广播公司、标识设计和位置、帧转换和剪辑效果具有鲁棒性。基于YouTube体育视频集中的足球、棒球和板球运动对所提框架进行性能评估。实验结果证明所提框架能够可靠地用于户外运动视频摘要的镜头分类和回放检测。

Abstract

Automated analysis of sports video summarization is challenging due to variations in cameras

replay speed

illumination conditions

editing effects

game structure

genre

etc. To address these challenges

we propose an effective video summarization framework based on shot classification and replay detection for field sports videos. Accurate shot classification is mandatory to better structure the input video for further processing

i.e.

key events or replay detection. Therefore

we present a lightweight convolutional neural network based method for shot classification. Then we analyze each shot for replay detection and specifically detect the successive batch of logo transition frames that identify the replay segments from the sports videos. For this purpose

we propose local octa-pattern features to represent video frames and train the extreme learning machine for classification as replay or non-replay frames. The proposed framework is robust to variations in cameras

replay speed

shot speed

illumination conditions

game structure

sports genre

broadcasters

logo designs and placement

frame transitions

and editing effects. The performance of our framework is evaluated on a dataset containing diverse YouTube sports videos of soccer

baseball

and cricket. Experimental results demonstrate that the proposed framework can reliably be used for shot classification and replay detection to summarize field sports videos.

关键词

Keywords

references

Bagheri-Khaligh A , Raziperchikolaei R , Moghaddam ME , 2012 . A new method for shot classification in soccer sports video based on SVM classifier . Proc IEEE Southwest Symp on Image Analysis and Interpretation , p. 109 - 112 . doi: 10.1109/SSIAI.2012.6202465 http://doi.org/10.1109/SSIAI.2012.6202465

Chen CM , Chen LH , 2014 . Novel framework for sports video analysis: a basketball case study . Proc Int Conf on Image Processing , p. 961 - 965 . doi: 10.1109/ICIP.2014.7025193 http://doi.org/10.1109/ICIP.2014.7025193

Chen CM , Chen LH , 2015 . A novel method for slow motion replay detection in broadcast basketball video . Multimed Tools Appl , 74 ( 21 ): 9573 - 9593 . doi: 10.1007/s11042-014-2137-5 http://doi.org/10.1007/s11042-014-2137-5

Choroś K , Gogol A , 2016 . Improved method of detecting replay logo in sports videos based on contrast feature and histogram difference . Proc 8 th Int Conf on Computational Collective Intelligence , p. 542 - 552 . doi: 10.1007/978-3-319-45243-2_50 http://doi.org/10.1007/978-3-319-45243-2_50

Dang ZH , Du J , Huang QM , et al. , 2007 . Replay detection based on semi-automatic logo template sequence extraction in sports video . Proc 4 th Int Conf on Image and Graphics , p. 839 - 844 . doi: 10.1109/ICIG.2007.73 http://doi.org/10.1109/ICIG.2007.73

Duan LY , Xu M , Tian Q , et al. , 2004 . Mean shift-based video segment representation and applications to replay detection . Proc 29 th IEEE Int Conf on Acoustics, Speech, and Signal Processing , p. 709 - 712 . doi: 10.1109/ICASSP.2004.1327209 http://doi.org/10.1109/ICASSP.2004.1327209

Ekin A , Tekalp AM , Mehrotra R , 2003 . Automatic soccer video analysis and summarization . IEEE Trans Image Process , 12 ( 7 ): 796 - 807 . doi: 10.1109/TIP.2003.812758 http://doi.org/10.1109/TIP.2003.812758

Eldib MY , Zaid BSA , Zawbaa HM , et al. , 2009 . Soccer video summarization using enhanced logo detection . Proc 16 th IEEE Int Conf on Image Processing , p. 4345 - 4348 . doi: 10.1109/ICIP.2009.5413649 http://doi.org/10.1109/ICIP.2009.5413649

Fani M , Yazdi M , Clausi DA , et al. , 2017 . Soccer video structure analysis by parallel feature fusion network and hidden-to-observable transferring Markov model . IEEE Access , 5 : 27322 - 27336 . doi: 10.1109/ACCESS.2017.2769140 http://doi.org/10.1109/ACCESS.2017.2769140

Javed A , Bajwa KB , Malik H , et al. , 2016 . An efficient framework for automatic highlights generation from sports videos . IEEE Signal Process Lett , 23 ( 7 ): 954 - 958 . doi: 10.1109/LSP.2016.2573042 http://doi.org/10.1109/LSP.2016.2573042

Javed A , Irtaza A , Khaliq Y , et al. , 2019 . Replay and keyevents detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine . Appl Intell , 49 ( 8 ): 2899 - 2917 . doi: 10.1007/s10489-019-01410-x http://doi.org/10.1007/s10489-019-01410-x

Javed A , Malik KM , Irtaza A , et al. , 2020 . A decision tree framework for shot classification of field sports videos . J Supercomput , 76 ( 9 ): 7242 - 7267 . doi: 10.1007/s11227-020-03155-8 http://doi.org/10.1007/s11227-020-03155-8

Jiang H , Zhang M , 2011 . Tennis video shot classification based on support vector machine . Proc IEEE Int Conf on Computer Science and Automation Engineering , p. 757 - 761 . doi: 10.1109/CSAE.2011.5952612 http://doi.org/10.1109/CSAE.2011.5952612

Kapela R , McGuinness K , O’Connor NE , 2017 . Real-time field sports scene classification using colour and frequency space decompositions . J Real-Time Image Process , 13 ( 4 ): 725 - 737 . doi: 10.1007/s11554-014-0437-7 http://doi.org/10.1007/s11554-014-0437-7

Li W , Chen SJ , Wang HB , 2009 . A rule-based sports video event detection method . Proc Int Conf on Computational Intelligence and Software Engineering , p. 1 - 4 . doi: 10.1109/CISE.2009.5366226 http://doi.org/10.1109/CISE.2009.5366226

Minhas RA , Javed A , Irtaza A , et al. , 2019 . Shot classification of field sports videos using AlexNet convolutional neural network . Appl Sci , 9 ( 3 ): 483 . doi: 10.3390/app9030483 http://doi.org/10.3390/app9030483

Murala S , Maheshwari RP , Balasubramanian R , 2012 . Local tetra patterns: a new feature descriptor for content-based image retrieval . IEEE Trans Image Process , 21 ( 5 ): 2874 - 2886 . doi: 10.1109/TIP.2012.2188809 http://doi.org/10.1109/TIP.2012.2188809

Nguyen N , Yoshitaka A , 2012 . Shot type and replay detection for soccer video parsing . Proc IEEE Int Symp on Multimedia , p. 344 - 347 . doi: 10.1109/ISM.2012.69 http://doi.org/10.1109/ISM.2012.69

Pan H , Van Beek P , Sezan MI , 2001 . Detection of slow-motion replay segments in sports video for highlights generation . Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing , p. 1649 - 1652 . doi: 10.1109/ICASSP.2001.941253 http://doi.org/10.1109/ICASSP.2001.941253

Pan H , Li BX , Sezan MI , 2002 . Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions . Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing , p. IV-3385 - IV-3388 . doi: 10.1109/ICASSP.2002.5745380 http://doi.org/10.1109/ICASSP.2002.5745380

Raventós A , Quijada R , Torres L , et al. , 2015 . Automatic summarization of soccer highlights using audio-visual descriptors . SpringerPlus , 4 ( 1 ): 301 . doi: 10.1186/s40064-015-1065-9 http://doi.org/10.1186/s40064-015-1065-9

Su PC , Lan CH , Wu CS , et al. , 2013 . Transition effect detection for extracting highlights in baseball videos . EURASIP J Image Video Process , 2013 ( 1 ): 27 . doi: 10.1186/1687-5281-2013-27 http://doi.org/10.1186/1687-5281-2013-27

Tavassolipour M , Karimian M , Kasaei S , 2014 . Event detection and summarization in soccer videos using Bayesian network and copula . IEEE Trans Circ Syst Video Technol , 24 ( 2 ): 291 - 304 . doi: 10.1109/TCSVT.2013.2243640 http://doi.org/10.1109/TCSVT.2013.2243640

Tien MC , Chen HT , Chen YW , et al. , 2007 . Shot classification of basketball videos and its application in shooting position extraction . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , p. 1085 - 1088 . doi: 10.1109/ICASSP.2007.366100 http://doi.org/10.1109/ICASSP.2007.366100

Wang DH , Tian Q , Gao S , et al. , 2004 . News sports video shot classification with sports play field and motion features . Proc IEEE Conf on Image Processing , p. 2247 - 2250 . doi: 10.1109/ICIP.2004.1421545 http://doi.org/10.1109/ICIP.2004.1421545

Wang JJ , Chang E , Xu CS , 2005 . Soccer replay detection using scene transition structure analysis . Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing , p. 433 - 436 . doi: 10.1109/ICASSP.2005.1415434 http://doi.org/10.1109/ICASSP.2005.1415434

Wang L , Liu X , Lin S , et al. , 2004 . Generic slow-motion replay detection in sports video . Proc Int Conf on Image Processing , p. 1585 - 1588 . doi: 10.1109/ICIP.2004.1421370 http://doi.org/10.1109/ICIP.2004.1421370

Wu X , He R , Sun ZN , et al. , 2018 . A light CNN for deep face representation with noisy labels . IEEE Trans Inform Forens Secur , 13 ( 11 ): 2884 - 2896 . doi: 10.1109/TIFS.2018.2833032 http://doi.org/10.1109/TIFS.2018.2833032

Xu W , Yi Y , 2011 . A robust replay detection algorithm for soccer video . IEEE Signal Process Lett , 18 ( 9 ): 509 - 512 . doi: 10.1109/LSP.2011.2161287 http://doi.org/10.1109/LSP.2011.2161287

Zhao F , Dong Y , Wei Z , et al. , 2012 . Matching logos for slow motion replay detection in broadcast sports video . Proc IEEE Int Conf on Acoustics, Speech and Signal Processing , p. 1409 - 1412 . doi: 10.1109/ICASSP.2012.6288154 http://doi.org/10.1109/ICASSP.2012.6288154

Zhao Z , Jiang SQ , Huang QM , et al. , 2006 . Highlight summarization in sports video based on replay detection . Proc Int Conf on Multimedia and Expo , p. 1613 - 1616 . doi: 10.1109/ICME.2006.262855 http://doi.org/10.1109/ICME.2006.262855

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Video summarization with a graph convolutional attention network

An effective fault prediction model developed using an extreme learning machine with various kernel methods

Related Author

Ping LI

Chao TANG

Xianghua XU

Lov KUMAR

Anand TIRKEY

Santanu-Ku. RATH

Related Institution

School of Computer Science and Technology, Hangzhou Dianzi University

The State Key Laboratory for Novel Software Technology, Nanjing University

Department of Computer Science and Engineering, National Institute of Technology Rourkela, Rourkela

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰