Long-term prediction for hierarchical-B-picture-based coding of video with repeated shots

Xu-guang ZUO; Lu YU

doi:10.1631/FITEE.1601552

Your Location：

Home >

Browse articles >

Long-term prediction for hierarchical-B-picture-based coding of video with repeated shots

Regular Papers | Updated：2022-05-19

- Long-term prediction for hierarchical-B-picture-based coding of video with repeated shots
- 重复镜头视频基于分级B帧结构的长期预测编码方案
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 3, Pages: 459-470(2018)
- Affiliations：
  
  Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking (IPCAN), Institute of Information and Communication Engineering, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  Lu YU, E-mail: yul@zju.edu.cn
- Funds：
- DOI：10.1631/FITEE.1601552
  CLC： TN919.8
- Received：13 September 2016，
  
  Revised：2018-;3-15，
  
  Published：2018-03
- Accepted：
Scan QR Code
Xu-guang ZUO, Lu YU. Long-term prediction for hierarchical-B-picture-based coding of video with repeated shots[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(3): 459-470.
DOI：

Xu-guang ZUO, Lu YU. Long-term prediction for hierarchical-B-picture-based coding of video with repeated shots[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(3): 459-470. DOI： 10.1631/FITEE.1601552.

摘要

最新视频编码标准--高效视频编码（HEVC）—获得了远高于之前视频编码标准的编码效率。特别是，在使用分级B帧预测结构时，HEVC可显著去除临近图像间的时间冗余。实际中，消费者使用的视频--例如电视剧、电影和脱口秀等 —通常包含了很多重复镜头。据我们观察，这些视频在分级B帧结构下使用HEVC编码时，每个镜头的时域相关性可以得到很好利用。然而，重复镜头之间的长期相关性并未被使用。我们提出一种长期预测方案，以利用视频中重复镜头之间的长期时域相关性。首先，聚类源视频中的相似镜头并从中抽取代表性图像作为长期参考帧。然后，使用长期参考帧对分级B帧的编码结构进行调整，以实现长期时域预测。最后，为长期参考帧设计一种自适应量化方法，以提高整体的视频编码效率。实验表明，提出的新编码方案可获得高达22.86%的编码性能增益。

Abstract

The latest video coding standard High Efficiency Video Coding (HEVC) can achieve much higher coding efficiency than previous video coding standards. Particularly

by exploiting the hierarchical B-picture prediction structure

temporal redundancy among neighbor frames is eliminated remarkably well. In practice

videos available to consumers usually contain many repeated shots

such as TV series

movies

and talk shows. According to our observations

when these videos are encoded by HEVC with the hierarchical B-picture structure

the temporal correlation in each shot is well exploited. However

the long-term correlation between repeated shots has not been used. We propose a long-term prediction (LTP) scheme to use the long-term temporal correlation between correlated shots in a video. The long-term reference (LTR) frames of a source video are chosen by clustering similar shots and extracting the representative frames

and a modified hierarchical B-picture coding structure based on an LTR frame is introduced to support long-term temporal prediction. An adaptive quantization method is further designed for LTR frames to improve the overall video coding efficiency. Experimental results show that up to 22.86% coding gain can be achieved using the new coding scheme.

关键词

Keywords

references

D Alfonso , , , B Biffi , , , L Pezzoni . . Adaptive GOP size control in H.264/AVC encoding based on scene change detection . . 7 th Nordic Signal Processing Symp , , 2006 . . p.86 - - 89 . . DOI: 10.1109/NORSIG.2006.275283 http://doi.org/10.1109/NORSIG.2006.275283 . .

G Bjontegaard . . Calculation of average PSNR differences between RD curves . . Document VCEG-M33. Austin, TX, USA , , 2001 . . .

F Bossen . . Common HM test conditions and software reference configurations . . Document JCT-VC L1100. Geneva, Switzerland , , 2013 . . .

M Cendrowski . . The Hofstadter Insufficiency . . The Big Bang Theory. DVD. Season 7. Episode 1. CBS , , 2013 . . .

J Dahl . . Chapter 33. House of Cards. DVD . . DVD. Season 3, Episode 7. Netflix , , 2015 . . .

YB Gao , , , C Zhu , , , S Li . . Hierarchical temporal dependent rate-distortion optimization for low-delay coding . . Proc IEEE Int Symp on Circuits and Systems , , 2016 . . p.570 - - 573 . . DOI: 10.1109/ISCAS.2016.7527304 http://doi.org/10.1109/ISCAS.2016.7527304 . .

JA Hartigan , , , MA Wong . . Algorithm AS 136: a K -means clustering algorithm . . J R Stat Soc , , 1979 . . 28 ( ( 1 ): ): 100 - - 108 . . DOI: 10.2307/2346830 http://doi.org/10.2307/2346830 . .

N Hu , , , EH Yang . . Fast mode selection for HEVC intra- frame coding with entropy coding refinement based on a transparent composite model . . IEEE Trans Circ Syst Video Technol , , 2015 . . 25 ( ( 9 ): ): 1521 - - 1532 . . DOI: 10.1109/TCSVT.2015.2395772 http://doi.org/10.1109/TCSVT.2015.2395772 . .

J Lee , , , S Kim , , , K Lim , , , 等 . . A fast CU size decision algorithm for HEVC . . IEEE Trans Circ Syst Video Technol , , 2015 . . 25 ( ( 3 ): ): 411 - - 421 . . DOI: 10.1109/TCSVT.2014.2339612 http://doi.org/10.1109/TCSVT.2014.2339612 . .

K Lenka , , , P Jaroslav , , , M Michal . . Adaptive group of pictures structure based on the positions of video cuts . . Proc World Academy of Science, Engineering and Technology , , 2018 . . p.377 - - 380 . . . .

S Li , , , C Zhu , , , YB Gao , , , 等 . . Lagrangian multiplier adaptation for rate-distortion optimization with inter-frame dependency . . IEEE Trans Circ Syst Video Technol , , 2016 . . 26 ( ( 1 ): ): 117 - - 129 . . DOI: 10.1109/TCSVT.2015.2450131 http://doi.org/10.1109/TCSVT.2015.2450131 . .

D Liu , , , DB Zhao , , , XY Ji , , , 等 . . Dual frame motion compensation with optimal long-term reference frame selection and bit allocation . . IEEE Trans Circ Syst Video Technol , , 2010 . . 20 ( ( 3 ): ): 325 - - 339 . . DOI: 10.1109/TCSVT.2009.2031442 http://doi.org/10.1109/TCSVT.2009.2031442 . .

C McCarthy . . The Sign of Three . . Sherlock. DVD. Season 3, Episode 2. BBC , , 2014 . . .

CW Ngo , , , TC Pong , , , HJ Zhang . . On clustering and retrieval of video shots . . Proc 9 th ACM Int Conf on Multimedia , , 2001 . . p.51 - - 60 . . DOI: 10.1145/500141.500151 http://doi.org/10.1145/500141.500151 . .

D Nutter . . A Man Without Honor. Game of Thrones . . DVD. Season 2, Episode 7. HBO , , 2012 . . .

ZQ Pan , , , S Kwong , , , MT Sun , , , 等 . . Early MERGE mode decision based on motion estimation and hierarchical depth correlation for HEVC . . IEEE Trans Broadcast , , 2014 . . 60 ( ( 2 ): ): 405 - - 412 . . DOI: 10.1109/TBC.2014.2321682 http://doi.org/10.1109/TBC.2014.2321682 . .

ZQ Pan , , , Y Zhang , , , JJ Lei , , , 等 . . Early DIRECT mode decision based on all-zero block and rate distortion cost for multiview video coding . . IET Image Process , , 2016a . . 10 ( ( 1 ): ): 9 - - 15 . . DOI: 10.1049/iet-ipr.2014.1018 http://doi.org/10.1049/iet-ipr.2014.1018 . .

ZQ Pan , , , Y Zhang , , , S Kwong . . Fast mode decision based on texture–depth correlation and motion prediction for multiview depth video coding . . J Real-Time Image Process , , 2016b . . 11 ( ( 1 ): ): 27 - - 36 . . DOI: 10.1007/s11554-013-0328-3 http://doi.org/10.1007/s11554-013-0328-3 . .

ZQ Pan , , , JJ Lei , , , Y Zhang , , , 等 . . Fast motion estimation based on content property for low-complexity H . . 265/ HEVC encoder. IEEE Trans Broadcast , , 2016c . . 62 ( ( 3 ): ): 675 - - 684 . . DOI: 10.1109/TBC.2016.2580920 http://doi.org/10.1109/TBC.2016.2580920 . .

ZQ Pan , , , P Jin , , , JJ Lei , , , 等 . . Fast reference frame selection based on content similarity for low complexity HEVC encoder . . J Vis Commun Image Represent , , 2016d . . 40 516 - - 524 . . DOI: 10.1016/j.jvcir.2016.07.018 http://doi.org/10.1016/j.jvcir.2016.07.018 . .

M Paul , , , WS Lin , , , CT Lau , , , 等 . . Explore and model better Ⅰ-frames for video coding . . IEEE Trans Circ Syst Video Technol , , 2011 . . 21 ( ( 9 ): ): 1242 - - 1254 . . DOI: 10.1109/TCSVT.2011.2138750 http://doi.org/10.1109/TCSVT.2011.2138750 . .

M Paul , , , WS Lin , , , CT Lau , , , 等 . . A long-term reference frame for hierarchical B-picture-based video coding . . IEEE Trans Circ Syst Video Technol , , 2014 . . 24 ( ( 10 ): ): 1729 - - 1742 . . DOI: 10.1109/TCSVT.2014.2302555 http://doi.org/10.1109/TCSVT.2014.2302555 . .

C Rosewarne , , , B Bross , , , M Naccari , , , 等 . . High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) . . Document JCTVC-X1002. Geneva, Switzerland , , 2016 . . .

D Scardino . . And the Show and Don't Tell. 2 Broke Girls . . DVD. Season 5, Episode 17. CBS , , 2015 . . .

H Schwarz , , , D Marpe , , , T Wiegand . . Overview of the scalable video coding extension of the H . . 264/AVC standard. IEEE Trans Circ Syst Video Technol , , 2007 . . 17 ( ( 9 ): ): 1103 - - 1120 . . DOI: 10.1109/TCSVT.2007.905532 http://doi.org/10.1109/TCSVT.2007.905532 . .

GJ Sullivan , , , JR Ohm , , , WJ Han , , , 等 . . Overview of the High Efficiency Video Coding (HEVC) standard . . IEEE Trans Circ Syst Video Technol , , 2012 . . 22 ( ( 12 ): ): 1649 - - 1668 . . DOI: 10.1109/TCSVT.2012.2221191 http://doi.org/10.1109/TCSVT.2012.2221191 . .

XL Tang , , , SK Dai , , , CH Cai . . An analysis of TZSearch algorithm in JMVC . . Proc IEEE Int Conf on Green Circuits and Systems , , 2010 . . p.516 - - 520 . . DOI: 10.1109/ICGCS.2010.5543008 http://doi.org/10.1109/ICGCS.2010.5543008 . .

R Tirone . . The Price. Once Upon a Time . . DVD. Season 5, Episode 2. ABC , , 2015 . . .

M Tiwari , , , PC Cosman . . Selection of long-term reference frames in dual-frame video coding using simulated annealing . . IEEE Signal Process Lett , , 2008 . . 15 249 - - 252 . . DOI: 10.1109/LSP.2007.914928 http://doi.org/10.1109/LSP.2007.914928 . .

J Vendrig , , , M Worring . . Systematic evaluation of logical story unit segmentation . . IEEE Trans Multim , , 2002 . . 4 ( ( 4 ): ): 492 - - 499 . . DOI: 10.1109/TMM.2002.802021 http://doi.org/10.1109/TMM.2002.802021 . .

XY Wang , , , ZK Weng . . Scene abrupt change detection . . Proc IEEE Conf on Electrical and Computer Engineering , , 2000 . . p.880- - - 883 . . DOI: 10.1109/CCECE.2000.849592 http://doi.org/10.1109/CCECE.2000.849592 . .

T Wiegand , , , GJ Sullivan , , , G Bjontegaard , , , 等 . . Overview of the H.264/AVC video coding standard . . IEEE Trans Circ Syst Video Technol , , 2003 . . 13 ( ( 7 ): ): 560 - - 576 . . DOI: 10.1109/TCSVT.2003.815165 http://doi.org/10.1109/TCSVT.2003.815165 . .

S Youm , , , W Kim . . Dynamic threshold method for scene change detection . . Proc IEEE Int Conf on Multimedia and Expo , , 2003 . . p.337 - - 340 . . DOI: 10.1109/ICME.2003.1221622 http://doi.org/10.1109/ICME.2003.1221622 . .

XG Zhang , , , LH Liang , , , H Huang , , , 等 . . An efficient coding scheme for surveillance videos captured by stationary cameras . . Proc SPIE Visual Communications and Image Processing , , 2010 . . p.1 - - 10 . . DOI: 10.1117/12.863522 http://doi.org/10.1117/12.863522 . .

XG Zhang , , , YH Tian , , , TJ Huang , , , 等 . . Low-complexity and high-efficiency background modeling for surveillance video coding . . Proc IEEE Visual Communications and Image Processing , , 2012 . . p.769 - - 784 . . DOI: 10.1109/VCIP.2012.6410796 http://doi.org/10.1109/VCIP.2012.6410796 . .

XG Zhang , , , TJ Huang , , , YH Tian , , , 等 . . Background- modeling-based adaptive prediction for surveillance video coding . . IEEE Trans Image Process , , 2014 . . 23 ( ( 2 ): ): 769 - - 784 . . DOI: 10.1109/TIP.2013.2294549 http://doi.org/10.1109/TIP.2013.2294549 . .

XL Zheng . . Empresses in the Palace. DVD . . Beijing Television Arts Centre, Beijing, China (in Chinese) , , 2012 . . .

XG Zuo , , , L Yu . . A novel interpolation-free scheme for fractional pixel motion estimation . . Proc Picture Coding Symp , , 2015 . . p.80 - - 84 . . DOI: 10.1109/PCS.2015.7170051 http://doi.org/10.1109/PCS.2015.7170051 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰