基於空間特徵與摺積神經網路於 H.266/FVC 視訊畫面內編碼快速分割決策

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：39

、訪客IP：18.221.129.19

姓名

涂育良(Yu-Liang Tu) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

基於空間特徵與摺積神經網路於 H.266/FVC 視訊畫面內編碼快速分割決策
(Fast Partition Decision for H.266/FVC Video Intra Coding Based on Spatial Features and CNNs)

相關論文

★ 基於區域權重之衛星影像超解析技術	★ 延伸曝光曲線線性特性之調適性高動態範圍影像融合演算法
★ 實現於RISC架構之H.264視訊編碼複雜度控制	★ 基於卷積遞迴神經網路之構音異常評估技術
★ 具有元學習分類權重轉移網路生成遮罩於少樣本圖像分割技術	★ 具有注意力機制之隱式表示於影像重建三維人體模型
★ 使用對抗式圖形神經網路之物件偵測張榮	★ 基於弱監督式學習可變形模型之三維人臉重建
★ 以非監督式表徵分離學習之邊緣運算裝置低延遲樂曲中人聲轉換架構	★ 基於序列至序列模型之 FMCW雷達估計人體姿勢
★ 基於多層次注意力機制之單目相機語意場景補全技術	★ 基於時序卷積網路之單FMCW雷達應用於非接觸式即時生命特徵監控
★ 視訊隨選網路上的視訊訊務描述與管理	★ 基於線性預測編碼及音框基頻週期同步之高品質語音變換技術
★ 基於藉語音再取樣萃取共振峰變化之聲調調整技術	★ 即時細緻可調性視訊在無線區域網路下之傳輸效率最佳化研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

跟隨著網路和多媒體的科技迅速成長，我們人類平日生活的高解析度視訊重要性一日千里，最近市面已出現許多 4K 高解析度的視訊，可以看見未來高解析度視頻勢必會成為主流，而目前的最新視頻壓縮標準 H.265/HEVC 已經逐漸不夠能用，因此ITU-T VCEG 和 ISO/IEC MPEG 共同組成的JVET (Joint Video Exploration Team) 來制定新一代的視訊壓縮標準 H.266/FVC (Future Video Coding)，從2015年開始討論，且預計2021年正式發佈變國際視訊壓縮標準。
H.266/FVC與H.265/HEVC相比不僅採用了QT(QuadTree)且還新增了BT(Binary Tree)，不但將複雜的CU、PU和TU的組成元素除去，且能夠支援最大256x256到最小8x8的正方形區塊，以依據更多不同大小畫面的紋理特性編碼。QTBT的架構雖比QT提供更好的編碼效能，但其預測數量的增加造成在執行畫面編碼時間提高了5.6倍，於是針對畫面內編碼，如何發展增快CU編碼時間下的決策，這是非常重要的議題。
本論文結合近年十分熱門的人工智慧系統 (Artificial Intelligence, AI)，提出基於空間特徵與摺積神經網路於 H.266/FVC視訊畫面內編碼快速分割決策。主要分為兩部分探討：首先在第一部份如何使用空間特徵來分析紋理特徵，以來決策是否切割與減少CNN使用次數；在第二部份則是針對預測模型的訓練和訓練資料的選擇來討論，再將訓練好的預測模型結合至 H.266/FVC 壓縮參考軟體當中來執行編碼。

摘要(英)

With the rapid growth of Internet and multimedia technology, the high-resolution video of our daily lives is becoming important. Recently, there have been many 4K high-resolution videos on the market. It can be expected that high-resolution video will become the mainstream in the future. However, the video compression standard H.265/HEVC has gradually become insufficient. Therefore, the JVET (Joint Video Exploration Team) consist of ITU-T VCEG and ISO/IEC MPEG has developed a new generation of the video compression standard H.266/ FVC (Future Video Coding). JVET is expected to be officially released in 2021 to become an international video compression standard. H.266/FVC is different from H.265/HEVC, it uses QT (QuadTree), adds QTBT (QuadTree plus Binary Tree), and removes the complex elements of CU, PU, and TU. H.266/FVC codes according to the texture characteristics of more pictures of different sizes. Although the QTBT architecture provides better coding performance than QT. Therefore, for intra-picture coding, how to develop decision-making under CU coding time is very important.
This thesis uses the Artificial Intelligence (AI) system, and proposes a fast partition prediction and decision-making method based on the convolutional neural network in the H.266/FVC video coding frame. The discussion is mainly divided into two parts: First, using spatial features to analyze texture features is to decide whether to cut and reduce the number of CNN usage. Secondly, discussing the training of the prediction model and the selection of training data, and then combine the trained prediction model with H.266/FVC to perform encoding.

關鍵字(中)

★ 未來視訊壓縮編碼
★ 編碼單位
★ 畫面內編碼
★ 分區預測
★ 摺積神經網路
★ 空間特徵

關鍵字(英)

★ Future Video Coding (FVC)
★ Coding Unit (CU)
★ Intra Coding
★ Partition Prediction
★ Convolutional Neural Network (CNN)
★ Spatial Feature

論文目次

摘要 .................................................................................................................... I
ABSTRACT...................................................................................................... II
誌謝 ................................................................................................................. III
目錄 ................................................................................................................. IV
附圖索引 ........................................................................................................ VII
附表索引 ......................................................................................................... IX
第一章緒論 ..................................................................................................... 1
1.1 研究背景 ................................................................................................. 1
1.2 研究動機與目的 ..................................................................................... 2
1.3 論文架構 ................................................................................................. 3
第二章 H.266/FVC 視訊編碼標準介紹 .......................................................... 4
2.1 H.266/FVC 視訊編碼介紹 ...................................................................... 4
2.1.1 H.266/FVC 與 H.265/HEVC 差異 ................................................. 4
2.1.2 編碼流程介紹 .................................................................................. 6
2.2 H.266/FVC 視訊編碼架構介紹............................................................... 7
2.2.1 編碼單元 (Coding Unit, CU) ........................................................... 7
2.2.2 預測單元 (Prediction Unit, PU)..................................................... 13
2.2.2.1 畫面內預測 (Intra prediction) ................................................. 13
2.2.2.2 畫面間預測 (Inter prediction) ................................................. 16
2.2.3 轉換單元 (Transform Unit, TU) .................................................... 21
2.3 H.266/FVC 環境設定及視訊樣本介紹 ................................................. 22
2.3.1 環境設定 ........................................................................................ 22
2.3.2 視訊樣本介紹 ................................................................................ 24
第三章深度學習介紹 .................................................................................... 29
3.1類神經網路 ............................................................................................ 30
3.1.1 類神經網路的發展歷史................................................................. 30
3.1.2 倒傳遞神經網路及多層感知機 ..................................................... 33
3.2 深度學習 ............................................................................................... 36
3.2.1 摺積神經網路(Convolutional Neural Network) ............................. 36
3.3 基於深度學習的切割模式決策相關文獻............................................. 40
第四章基於空間特徵與摺積神經網路於 H.266/FVC視訊畫面內編碼快速分割決策 ......................................................................................................... 42
4.1 四分樹分割(QT DEPTH0和QT DEPTH1) .............................................. 42
4.1.1 四分樹(QTDepth0)特徵提取 ......................................................... 42
4.1.2 四分樹(QTDepth1)特徵提取 ......................................................... 44
4.1.2.1 QP在MeanVarianceGRAD的影響 ........................................... 45
4.1.3 四分樹快速決策與閥值設定 ......................................................... 46
4.2 CNN於四分樹(QTDEPTH2)快速分割決策 ........................................... 48
4.2.1 前處理部分 (Pre-processing section) ............................................ 48
4.2.1.1 根據CNN的快速決策 ........................................................... 51
4.2.2 訓練部分 (Training section) .......................................................... 51
4.2.3 驗證階段 (Validation stage) .......................................................... 54
4.3 提出方法的整體流程 ............................................................................ 56
第五章實驗結果與分析討論 ........................................................................ 58
5.1 實驗環境的設置 ................................................................................... 58
5.2 實驗結果 ............................................................................................... 60
第六章結論與未來展望 ................................................................................ 63
參考文獻 ......................................................................................................... 64

參考文獻

[1] J. Chen, E. Alshina, G. J. Sullivan, J. R. Ohm and J. Boyce, “Algorithm Description of Joint Exploration Test Model 7 (JEM7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting, Doc. JVET-G1001, Torino, July 2017.
[2] JEM reference software, https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/.
[3] High Efficiency Video Coding (HEVC), Rec. ITU-T H.265 and ISO/IEC 23008-2, Jan. 2013.
[4] G. J. Sullivan, J. R. Ohm, W. J. Han, and T. Wiegand, “Overview of the High Efficiency Video Coding (HEVC) Standard,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649-1668, Dec. 2012.
[5] C. Rosewarne, B. Bross, M. Naccari, K. Sharman, and G. J. Sullivan, “High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Update 4 of Encoder Description,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 22nd Meeting, Doc. JCTVC-V1002, Oct. 2015.
[6] HEVC reference software, https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.6/.
[7] H. Huang, K. Zhang, Y. W. Huang, and S. M. Lei, “EE2.1: Quadtree plus binary tree structure integration with JEM tools,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 3rd Meeting, Doc. JVET-C0024, Geneva, May 2016.
[8] M. Karczewicz and E. Alshina, “JVET AHG report: Tool evaluation (AHG1),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 8th Meeting, Doc. JVET-H0001, Macau, Oct. 2017.
[9] Y. Yamamoto and T. Ikai, “AHG5: Fast QTBT encoding configuration,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 4th Meeting, Doc. JVET-D0095, Chengdu, Oct. 2016.
[10] J. Chen, W. J. Chien, M. Karczewicz, X. Li, H. Liu, A. Said, L. Zhang, and X. Zhao, “Further improvements to HMKTA-1.0,” ITU-T SG16/Q6, Doc. VCEG-AZ07, Jun. 2015.
[11] J. Chen, Y. Chen, M. Karczewicz, X. Li, H. Liu, L. Zhang and X. Zhao, “Coding tools investigation for next generation video coding,” ITU-T SG16/Q6, Doc. COM16-C806, Feb. 2015.
[12] W. J. Chien, J. Chen, S. Lee, and M. Karczewicz, “Modification of merge candidate derivation,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 2nd Meeting, Doc. JVET-B0058, San Diego, Feb 2016.
[13] J. L. Lin, Y. W. Chen, Y. W. Huang, and S. M. Lei, “Motion Vector Coding in the HEVC Standard,” in IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 6, pp. 957-968, Dec. 2013.
[14] A. Said, X. Zhao, J. Chen, M. Karczewicz, W. J. Chien, and F. Zhou, “Position dependent intra prediction combination,” ITU-T SG16/Q6, Doc. COM16-C1016, Oct. 2015.
[15] K. Suehring and X. Li, “JVET common test conditions and software reference configurations,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting, Doc. JVET-G1010, Torino, July 2017.
[16] W. S. Mcculloch and W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bulletin of Mathematical Biophysics, vol.5, no.4,pp.115-133, Dec. 1943.
[17] D. O. Hebb, “Organization of Behavior,” New York: Wiley & Sons.
[18] V. Nair, and G. E. Hinton, “Rectified Linear Units Improve Restricted Boltzmann Machines,” in Proceedings of the 27th International Conference on Machine Learning (ICML-10), Jun. 2010.
[19] F. A. Makinde, C. T. Ako, O. D. Orodu, I. U. Asuquo, “Prediction of crude oil viscosity using feed-forward back-propagation neural network (FFBPNN),” Petroleum and Coal, vol. 54, pp. 120-131, 2012.
[20] P. J. Werbos, “Beyond regression: new tools for prediction and analysis in the behavioral sciences,” Ph.D. thesis, Harvard University, 1974.
[21] K. Alex, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in Advances in Neural Information Processing Systems, pp.1097-1105, 2012.
[22] Y. Lecun, et al., “Gradient-based learning applied to document recognition”, Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[23] I. Mrazova, M. Kukacka, “Hybrid convolutional neural networks,” Industrial Informatics INDIN 2008. 6th IEEE International Conference, 2008.
[24] T. Li, M. Xu, X. Deng, “A deep convolutional neural network approach for complexity reduction on intra-mode HEVC”, IEEE International Conference on Multimedia and Expo (ICME), pp. 1255-1260, Aug. 2017
[25] Z. Jin, P. An, C. Yang, L. Shen, “Fast QTBT Partition Algorithm for Intra Frame Coding through Convolutional Neural Network,” IEEE Access, vol. 6, pp. 54660-54673, Sep. 2018.
[26] T. Lin, H. Jiang, J. Huang,P. Chang, “Fast Intra Coding Unit Partition Decision in H.266/FVC Based on Spatial Features,” 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), pp. 1-2, May 2018.
[27] TensorFlow: an open source Python package for machine intelligence, https://www.tensorflow.org, retrieved Dec. 1, 2016.
[28] keras: https://keras.io/
[29] G. Bjontegaard, “Calculation of Average PSNR Difference Between RD-curves,” ITU-T Q.6/SG16 VCEG 13th Meeting, Doc. VCEG-M33, 2001.
[30] Wang Z., Wang S.Q, Zhang J., et al., "Effective Quadtree Plus Binary Tree Block Partition Decision for Future Video Coding". Data Compression Conference (DCC 2017), Snowbird, Utah, Apr.4-7, 2017.

指導教授

張寶基(Pao-Chi Chang)

審核日期

2021-9-16

推文