深度學習應用於HEVC畫面間解碼之後處理機制

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：24

、訪客IP：52.14.84.29

姓名

謝宗凱(Chung-Kai Hsieh) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

深度學習應用於HEVC畫面間解碼之後處理機制
(CNN-Based Post-Processing for HEVC Inter Prediction)

相關論文

★ 10Gb/s MM XFP光收發模組設計與實現	★ 資訊產品自動化測試之研究
★ 高電流密度鰭式氮化鎵高電子遷移率電晶體研究	★ 電子郵件及壓縮檔案解碼之研究
★ 渦輪碼在光學記錄系統上之應用	★ 離散餘弦轉換硬體架構之研究
★ 動態影像之錯誤隱藏研究	★ 即時性無失真壓縮編碼之研究
★ 類神經網路在手寫數字辨識之研究	★ 事後機率演算法則在資料儲存系統之研究
★ 紅外線傳輸協定及通道之研究	★ 低密度同位元檢查碼在數位資料儲存系統之研究
★ 一種新型的JPEG2000竄改偵測與還原技術	★ 即時性無失真壓縮之研究
★ 混合快速模式決策演算法之研究	★ 光學記錄MEPR2通道系統之時序恢復探討與研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

隨著影音娛樂蓬勃發展，不只電視、電影甚至當紅的影音串流平台
Youtube、Twitch 皆追求越來越高的畫質，近期直播更是流行，不只有追求畫質更要在即時傳輸達到一定的水準，硬體方面各家電視廠商螢幕也是越做越大，而人們為了有效的壓縮高解析度影像的巨大資料量，HEVC(High Efficiency Video Coding)運用的許多方式有效的降低位元傳輸，在此篇論文中我們在HEVC 畫面間預測的編碼端應用了 SVM(Support Vector Machine)模型，對編碼單元深度做分類，利用畫面間預測的編碼單元之運動向量變異數資訊、合併模式的 CBF 資訊，以及相鄰區塊的深度資訊作為訓練 SVM 模型的特徵(Features)將一個 CTU 做分類，分類為 Subgroup0、1、2、3 共四種類別，其中 Subgroup0包含 CTU 深度 0，Subgroup1 包含深度 0、1，Subgroup2 包含深度 0、1、2 而Subgroup3 包含深度 0、1、2、3 最後會在經過 RDO 過程選出 CTU 最佳深度，此演算法可以在編碼端節省 23.5%的編碼時間，但增加了 0.07%的 BDBR，所以我們決定使用後處理技術，在解碼端將節省編碼時間所造成的編碼效能損失補償回來，我們運用日漸流行的卷積神經網路CNN(Convolutional Neural Network)於 HEVC 後處理，來提高影像品質。在實驗裡結合了消息理論中提及的側面消息概念，越多的側面消息可以降低越多的未定量，所以在 CNN 模型中除了輸入經過壓縮過後的失真影像也會加入編碼端 SVM 模型所使用的特徵做為第二輸入，幫助 CNN 模型訓練的更精準，最後更會加入編碼中量化(Quantization)所造成的誤差做為 CNN 模型的第三輸入，於是在我們結合整體架構後，最終在 HEVC 畫面間預測與參考程式 HM16.0 相比，可以達到 BDBR 減少6.59%，在 BDPSNR 增加0.237dB。

Network)於 HEVC 後處理，來提高影像品質。在實驗裡結合了消息理論中提及
的側面消息概念，越多的側面消息可以降低越多的未定量，所以在 CNN 模
型中除了輸入經過壓縮過後的失真影像也會加入編碼端 SVM 模型所使用的特
徵做為第二輸入，幫助 CNN 模型訓練的更精準，最後更會加入編碼中量化(Quantization)所造成的誤差做為 CNN 模型的第三輸入，於是在我們結合整體架構後，最終在 HEVC 畫面間預測與參考程式 HM16.0 相比，可以達到 BDBR 減少-6.59%，在 BDPSNR 增加 0.237dB。

摘要(英)

With the development of video and audio entertainment, not only TVs, movies, but also popular video streaming platforms Youtube and Twitch are pursuing higher
and higher image quality. Recently, live broadcast is becoming more popular. In terms of hardware, the screens of TV are getting larger and larger. In order to effectively compress the huge data volume of high-resolution images, HEVC (High Efficiency Video Coding) uses many methods to effectively reduce bit transmission. In this paper, we apply the SVM (Support Vector Machine) model to the encoding side of HEVC inter prediction classify the depth of the coding unit, and use the motion vector variation information of the inter prediction coding unit and the CBF information of the merge mode. And the depth information of the adjacent blocks is used as the features of the training SVM model to classify a CTU, which is divided into four categories: Subgroup0, 1, 2, and 3. Subgroup0 contains CTU depth 0, and Subgroup1
contains depth 0, 1. , Subgroup2 contains depth 0, 1, 2 and Subgroup3 contains depth 0, 1, 2, 3. Finally, the best depth of CTU will be selected after the RDO process. This algorithm can save 23.5% of the encoding time at the encoder, but it increases by 0.07 % BDBR, so we decided to use post-processing technology to compensate for
the coding performance caused by saving coding time at the decoder. We use the convolutional neural network CNN (Convolutional Neural Network) model in HEVC
post-processing to improve Image quality. In the experiment, the side information concept mentioned in the information theory is combined. The more side information
can reduce the more uncertainty, so in addition to the input of the distorted image after compression, the features used in the SVM model at the encoder will be added as the second input. It can help CNN model training more accurately. Finally, the error caused by quantization in the encoding will be added as the third input of the CNN model. So after we combined the overall architecture, compared with the reference program HM16.0, our algorithm achieves up to 6.59% BDBR reduction and 0.237dB BDPSNR increase.

關鍵字(中)

★ HEVC
★ 畫面間預測
★ 支持向量機
★ 運動向量
★ 卷積神經網路
★ 影像後處理

關鍵字(英)

★ HEVC
★ Inter prediction
★ SVM
★ motion vector
★ CNN
★ Image post-processing

論文目次

論文摘要................................................ V
Abstract............................................. VII
誌謝.................................................. IX
章節目錄................................................ X
附圖索引............................................. XIII
附表索引............................................ XVIII
第一章緒論............................................. 1
1.1 高效率視訊編碼(HEVC) ............................... 2
1.2 高效率視訊編碼架構介紹 .............................. 3
1.2.1 編碼單元(Coding Unit) ............................ 4
1.2.2 預測單元(Prediction Unit) ........................ 5
1.2.3 轉換單元(Transform Unit).......................... 6
1.2.4 碼率失真函數(RD cost) ............................ 6
1.2.5 HEVC 編碼結構(Configuration) ..................... 8
1.2.6 量化(Quantization) ............................. 10
1.3 研究動機及目的 .................................... 11
1.4 論文架構 .......................................... 11
第二章先備知識與相關文獻回顧 ........................... 12
2.1 畫面間預測介紹(Inter Prediction) .................. 12
2.1.1 合併模式決策介紹(Merge Mode Decision) ........... 12
2.1.2 畫面間模式決策介紹(Inter Mode Decision) .......... 16
2.2 支持向量機(Support Vector Machine) ................ 21
2.3 機械學習 .......................................... 26
2.3.1 類神經網路 ...................................... 27 1.非監督式學習..........................................27 2.監督式學習............................................27 2.3.2 深度學習......................................... 28 1.深度神經網路(DNN).....................................28 2.卷積神經網路(CNN).....................................30
2.4 SVM 應用於 HEVC 畫面間編碼單元快速決策演算法......... 32
2.4.1 支持向量機編碼單元特徵選取 ....................... 35 1.移動向量變異數(Motion Vector Variance)...............35 2.Coded Block Flag(CBF)...............................39 3.鄰近編碼單元深度資訊(Neighboring CU)..................40 2.4.2 系統流程圖 ..................................... 42
2.4.3 實驗數據 ....................................... 43
2.5 相關文獻 ......................................... 44
2.5.1 CNN Based Post-Processing to Improve HEVC ...... 44
2.5.2 Enhancing HEVC Compressed Videos with A Partition
Masked Convolutional Neural Network .................. 46
2.5.3An In-loop Filter Based on Low-Complexity CNN Using Residuals in Intra Video Coding ...................... 48
第三章結合 SVM 及 CNN 用於 HEVC 解碼端後處理之消息理論背景 50
3.1 動機說明 .......................................... 50
3.2 SVM 分類訓練資料前處理理論基礎 ...................... 53
第四章系統架構與模型製作 ............................... 57
4.1 系統架構 .......................................... 57
4.2 模型製作 .......................................... 61
4.2.1 訓練環境配置 .................................... 61
4.2.2 訓練資料製作與前處理 ............................. 63
1.HEVC+CNN.............................................63 2.CNN_1................................................64 3.CNN_1+CNN_2..........................................66 4.CNN_1+CNN_3..........................................68 5.CNN_1+CNN_2+CNN_3....................................69 4.2.3 訓練階段 ........................................ 70
1.HEVC+CNN.............................................70 2.CNN_1................................................72 3.CNN_1+CNN_2..........................................73 4.CNN_1+CNN_3..........................................75 5.CNN_1+CNN_2+CNN_3....................................76 4.2.4 驗證階段 ........................................ 79
第五章個架構實驗性能分析 ............................... 83
5.1 Random access 結構下性能分析 ...................... 83
5.1.1 碼率失真曲線說明 ................................. 91
5.1.2 架構間圖片差異 .................................. 93
5.1.3 編解碼時間分析 ................................. 102
5.2 Low-Delay 結構下性能分析 ......................... 110
第六章結論與未來展望 ................................. 112
參考文獻.............................................. 114

參考文獻

[1] JCT-VC, “High efficiency video coding (HEVC) test model 15(HM15) encoder description,” JCTVC-Q1002, JCT-VC Meeting, Valencia, ES, Apr. 2014.
[2] I. E. G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia. Aberdeen, U.K.: John Wiley & Sons, 2003.
[3] “Generic coding of moving pictures and associated audio information,” ISO/IEC 13818-2: Video (MPEG-2), May 1996.
[4] P. Helle, S. Oudin, B. Bross, D. Marpe, M. O. Bici, K. Ugur, J. Jung, G. Clare, and T. Wiegand, “Block merging for quadtree-based partitioning in HEVC,” in
Proc. IEEE Transactions on circuits and systems for video technology, vol. 22, no.12, pp. 1720-1731, Dec. 2012.
[5] L. Zhao, X. Guo, S. Lei, S. Ma and D. Zhao, “Simplified AMVP for high efficiency video coding,” in Proc. IEEE ICIP, pp. 1-4, 27-30 Nov. 2012.
[6] J. L. Lin, Y. W. Chen, Y. W. Huang, and S. M. Lei, “Motion vector coding in the HEVC standard,” in Proc. IEEE Journal of Selected Topics in Signal Processing,
vol. 7, no. 6, pp. 957-968, 3 July 2013.
[7] Y. Ismail and S. El-etriby, “Fast diamond search algorithm for real time video coding,” in Proc. IEEE ICNC, pp. 729-733, Feb. 2012.
[8] LIBSVM—A Library for Support Vector, Machines
http://www.csie.ntu.edu.tw/~cjlin/libsvm/index.html
[9] K. Alex, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in Advances in Neural Information Processing Systems, pp.1097-1105, 2012.
[10] Y. Lecun, et al., “Gradient-based learning applied to document recognition”, Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[11] I. Mrazova, M. Kukacka, “Hybrid convolutional neural networks”, Industrial Informatics INDIN 2008. 6th IEEE International Conference, 2008.
[12] S. Lawrence, et al., “Face recognition: A convolutional neural-network approach”, IEEE Transactions on Neural Networks, vol.8, no. 1, pp. 98-113, 1997.
[13] J.K. Liu, “Efficient HEVC inter prediction using SVM,” Department of Communication Engineering National Central University, Taiwan 32054, R.O.C., Jan 2019.
[14] S.J Cai, “Reduction of computation complexity for HEVC intra prediction with support vector machine,” National Central University, Master Thesis, Jun 2017.
[15] C. Li, L. Song, R. Xie, W. Zhang, "Cnn Based Post-Processing To Improve Hevc", International Conference on Image Processing(ICIP) 2017, pp.4577-4580
[16] J. Kim, J.K. Lee, K.M. Lee, “Accurate Image Super-Resolution Using Very Deep Convolutional Networks”, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1646-1654
[17] X. He, Q. Hu, X. Han, X. Zhang, C. Zhang, W. Lin, "Enhancing Hevc Compressed Videos With A Partition-Masked Convolutional Neural Network", International Conference on Image Processing(ICIP) 2018, pp.216-220
[18] Daowen Li, Lu Yu, “An In-Loop Filter Based on Low-Complexity CNN using Residuals in Intra Video Coding”, 2019 IEEE International Symposium on Circuits and Systems (ISCAS)
[19] S.M. Fan, “Study of A Deep Learning Architecture For HEVC Decoder”, Department of Communication Engineering National Central University, Taiwan 32054, R.O.C., Jan 2020.
[20] Y.C. Chang, “A Combined Support Vector Machine and Convolutional Neural Network Architecture for HEVC”, Department of Communication Engineering National Central University, Taiwan 32054, R.O.C., Jan 2020.
[21] Kaiming He , Xiangyu Zhang , Shaoqing Ren , Jian Sun, “Deep Residual Learning for Image Recognition”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
[22] C.H. Chan, “CNN-based post-processing for HEVC intra prediction”, Department of Communication Engineering National Central University, Taiwan 32054, R.O.C., JUL 2020.
[23] H. Zhang, L. Song, Z. Luo, X. Yang, “Learning a Convolutional Neural Network for Fractional Interpolation in HEVC Inter Coding”, 2017 IEEE Visual Communications and Image Processing (VCIP)
[24] C.H. Yeh, Z.T. Zhang, M.J. Chen, C.Y. Lin, “HEVC Intra Frame Coding Based on Convolutional Neural Network”, IEEE Access p.p. 50087 – 50095
[25] R. Yang, M. Xu, Z. Wang, “Decoder-side hevc quality enhancement with scalable convolutional neural network,” in Multimedia and Expo (ICME), 2017 IEEE International Conference on. IEEE, 2017, pp. 817–822.
[26] F. Li, W. Tan, B. Yan,“Deep Residual Network for Enhancing Quality of the Decoded Intra Frames of Hevc”, 2018 25th IEEE International Conference on Image Processing (ICIP)
[27] J. Xu, L. Song, R. Xie,"Shot boundary detection using convolutional neural networks", Visual Communications and Image Processing (VCIP), 2016. IEEE,
2016, pp. 1–4.
[28] Grand Challenge ICIP 2017, "Grand challenge on the use of image restoration for video coding efficiency improvement", Chttps://storage.googleapis.com/icip
2017/index.html.
[29] X. He, Q. Hu, X. Han, X. Zhang, C. Zhang, W. Lin, "Enhancing Hevc Compressed Videos With A Partition-Masked Convolutional Neural Network", International Conference on Image Processing(ICIP) 2018, pp.216-220
[30] Y.Dai, D. Liu, F.Wu, "A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding", MultiMedia Modeling(MMM) 2017, pp.2839

指導教授

林銀議(Yin-Yi Lin)

審核日期

2020-7-31

推文