為了加速高效率視訊編碼的編碼時間，本論文提出一個快速編碼單元深度決策的演算法，透過即時訓練預測模型，可以準確地預測當前編碼單元是否要往下分割，決定此深度剩餘編碼運算是否要執行。在過去文獻中發現許多透過離線訓練預測模型的方法，大幅節省編碼器的編碼時間，但這些使用離線訓練的方法存在一些共同的缺點：1.無法因應所有情況發生、2.需要額外記憶體空間儲存許多套預測模型因應不同的狀況。即時訓練的特性剛好可以解決離線訓練所帶來的問題，依據當編碼當下得到的訓練樣本建立模型，有很強的適應力。因此本論文利用擷取特定的編碼資訊訓練預測模型，分別有預測誤差的相關特徵、編碼單元的旗標資訊、編碼單元的深度資訊及相關預測模式的位元失真率成本等特徵，以人工神經網路為模型架構，採用即時訓練，當收集到一定數量的訓練樣本，透過順向傳遞和反向傳遞訓練預測模型的權重和偏置値。為了避免即時訓練造成準確率不足，透過訓練週期性地更新模型的權重和偏置値和提供良好參考畫面，以及若預測機率落在0.3~0.7，會依原始HM的編碼流程，不省略當前的深度剩餘編碼運算。最後透過參考測試軟體，本論文提出的演算法能夠達到50%的時間節省，伴隨不到2%BDBR上升。

High Efficiency Video Coding (HEVC) standard introduces flexible quad-tree coding block partitioning structure which includes coding unit (CU), prediction unit (PU) and transform unit (TU). The flexible structure provides better performance than predecessor, however the exhaustive Rate-Distortion Optimization (RDO) process requires a dramatic increase in the encoding computational complexity. In order to alleviate the computational burden in HEVC inter coding, a novel fast CU decision method relies on run-time trained artificial neural networks (ANN) is proposed in this paper. Contrasting to state-of-the-art machine learning methods, our method does not require offline-training and provides a high adaptivity to the variety video content. By extracting specific features, such as spatial-temporal depth, code block flag (CBF), relevant computational information to assist decisions on CU splitting. The method is implemented on an HEVC test software (HM) 16.4. Experiments and results demonstrate that the proposed method could reduce the computational complexity and achieve about 51.91% encoding time saving with a 2% acceptable Bjǿntegaard delta bitrate loss in low delay configuration.

[1] T. Wiegand, G. J. Sullivan, G. Bjontegaard and A. Luthra, “Overview of the H.264/AVC Video Coding Standard,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 13, no. 7, pp.560-576, July 2003.

[2] T. Wiegand, J. R. Ohm, G. J. Sullivan, W. J. Han, R. Joshi, T. K. Tan and K. Ugur, “Special section on the joint call for proposals on High Efficiency Video Coding (HEVC) standardization,” IEEE Transactions on Circuits and Systems for Video Technology, pp.1661-1666, 2010.

[3] G. J. Sullivan, J. R. Ohm, W. J. Han, T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Transactions on Circuits and Systems for Video Technology, pp.1649–1668, 2012.

[4] G. J. Sullivan, J. R. Ohm, H. Schwarz, T. K. Tan, T. Wiegand, “Comparison of the coding efficiency of video coding standards including high efficiency video coding (HEVC),” IEEE Circuits and Systems for Video Technology, pp.1669–1684, 2012.

[5] I. K. Kim, J. Min, T. Lee, W. J. Han, J. H. Park, “Block Partitioning Structure in the HEVC Standard,” IEEE Transactions on Circuits and Systems for Video Technology, pp.1697–1706, 2012.

[6] Y. Yuan; I. K. Kim, X. Zheng, L. Liu, X. Cao, S. Lee, M. S. Cheon, T. Lee, Y. He, J. H. Park, “Quadtree Based Nonsquare Block Structure for Inter Frame Coding in High Efficiency Video Coding,” IEEE Transactions on Circuits and Systems for Video Technology, 2012.

[7] J. Lainema, F. Bossen, W. J. Han, J. Min, K. Ugar, “Intra coding of the HEVC standard,” IEEE Transactions on Circuits and Systems for Video Technology, pp.1792-1801, 2012

[8] P. Helle, S. Oudin, B. Bross, D. Marpe, K. Ugur, J. Jung, G. Glare, T. Wiegand, “Block merging for quad-tree based partitioning in HEVC,” IEEE Transactions on Circuits and Systems for Video Technology, pp.1720-1731, 2012

[9] K. Ugur, A. Alshin, E. Alshina, F. Bossen, W. Han, J. Park, J. Lainema, “Motion Compensated Prediction and Interpolation Filter Design in H.265/HEVC,” IEEE Journal of Selected Topics in Signal Processing, vol. 7, pp.946-956, 2013.

[10] JCT-VC, “High Efficiency Video Coding (HEVC) Test Model 16 (HM16) Encoder Description,” JCTVC-R102, 18th Meeting, Sapporo, 2014.

[11] JCT-VC, “Early SKIP Detection for HEVC,” JCTVC-G543, 7th Meeting, Geneva, 2011.

[12] JCT-VC, “Early termination of CU encoding to reduce HEVC complexity,” JCTVC-F045, 6th Meeting, Torino,IT, 2011.

[13] JCT-VC, “Coding tree pruning based CU early termination,” JCTVC-F092, 6th Meeting, Torino, IT, 2011.

[14] L.Shen, Z, Liu, X. Zhang, W. Z, Z. Zhang, “An Effective CU Size Decision Method for HEVC Encoders,” IEEE Transactions on Multimedia, vol. 15, pp.465-470, 2013.
 
[15] G. Correa, P. Assuncao, L. Agostini, “Complexity control of HEVC through quadtree depth estimation,” IEEE EUROCON, pp.81-86, 2013.

[16] C. Zhou, F. Zhou, Y. Chen, “Spatio-temporal correlation-based fast coding unit depth decision for high efficiency video coding,” Journal of Electronic Imaging, vol. 22, issue. 4, pp. 1-14, Oct. 2013.

[17] Y. Zhang, H. Wang, Z. Li, “Fast Coding Unit Depth Decision Algorithm for Interframe Coding in HEVC,” Data Compression Conference, pp.53-62, 2013.

[18] JCT-VC, “Adaptive CU Depth Range,” JCTVC-E090, 5th Meeting, Geneva, CH, 2011.

[19] H. L. Tan, F. Liu, Y. H. Tan, C. Yeo, “On fast coding tree block and mode decision for high-Efficiency Video Coding (HEVC),” IEEE International Conference on Acoustics, Speech and Signal Processing, pp.825-828, 2012.

[20] J. H. Lee, C. S. Park, B. G. Kim, “Fast coding algorithm based on adaptive coding depth range selection for HEVC,” IEEE International Conference on Consumer Electronics, pp.31-33, 2012,

[21] G. Zhong, X. He, L.Qing, Y, Li, “A fast inter-prediction algorithm for HEVC based on temporal and spatial correlation,” Multimed Tools Appl, pp.11023-11043, 2015.
 
[22] Z. Pan, S. Kwong, M. T. Sun, J. Lei, “Early MERGE Mode Decision Based on Motion Estimation and Hierarchical Depth Correlation for HEVC,” IEEE Transactions on Broadcasting, vol. 60, pp.405-412, 2014.

[23] Q. Xu, D. Hu, “A new inter prediction mode decision method based on temporal correlation for HEVC encoder,” IEEE Workshop on Advanced Research and Technology in Industry Applications, pp.1375-1378, 2014.

[24] L. Shen, Z. Zhang, Z. Liu, “Adaptive Inter-Mode Decision for HEVC Jointly Utilizing Inter-Level and Spatiotemporal Correlations,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, pp.1709-1722, 2014.

[25] S. Ahn, B. Lee, M. Kim, “A Novel Fast CU Encoding Scheme Based on Spatiotemporal Encoding Parameters for HEVC Inter Coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, pp.422-435, 2015.

[26] J. He, X. He, X. Li, L. Qing, “Fast Inter-Mode Decision Algorithm for High-Efficiency Video Coding Based on Textural Features,” Journal of Communications, vol. 9, 2014.

[27] M. B. Cassa, M. Nsccari, F. Pereira, “Fast rate distortion optimization for the emerging HEVC standard,” Picture Coding Symposium, pp.493-496, 2012.

[28] P. K. Podder, M. Paul, M. Murshed, “A novel motion classification based intermode selection strategy for HEVC performance improvement,” Neurocomputing, vol. 173, pp.1211-1220, 2016.

[29] X. Shen, L. Yu, J, Chen, “Fast Coding Unit Size Selection for HEVC based on Bayesian Decision Rule,” Picture Coding Symposium, pp.453-456, 2012

[30] X. Shen, L. Yu, “CU splitting early termination based on weighted SVM,” EURASIP Journal on Image and Video Processing, 2013.

[31] J, He, W. Yang, J. Wang, “Fast HEVC Coding Unit Decision Based on BP-Neural Network,” International Journal of Grid Distribution Computing, vol. 8, 2015.

[32] Y. Zhang, S. Kwong, X. Wang, H. Yuan, Z. Pan, L. Xu, “Machine learning-based coding unit depth decisions for flexible complexity allocation in high efficiency video coding,” IEEE Transactions on Image Processing, vol. 24, no.7, 2015.

[33] X. Yu, Z. Liu, J. Liu, Y. Gao, D. Wang, “VLSI friendly fast CU/PU mode decision for HEVC intra encoding: Leveraging convolution neural network,” IEEE International Conference on Image Processing, pp.1285-1289, 2015.

[34] Zeidenberg, Matthew, Neural Networks in Artificial Intelligence, 1990.

[35] Murphy, Kevin, Machine Learning: A Probabilistic Perspective, 2012

[36] JCT-VC, “Common test conditions and software reference configurations,” JCTVC-G1200, 7th VCTVE Meeting, Geneva, 2011.

[37] JCT-VC, “Test sequence material (AHG16),” JCTVC-P0016, 16th Meeting, San José, US, 2014.

[38] G. Bjóntegaard, “Calculation of average PSNR difference between RD curves,” VCEG-M33, 13th VCEG Meeting, Austin, 2011.

[39]G. Correa, P. Assuncao, L. Agostini, “Four-step algorithm for early termination in HEVC inter-frame prediction based on decision trees,” IEEE Visual Communications and Image Processing Conference, pp.65-68, 2014.