在科技日月異之下，為了滿足人們對於影像品質的追求，視訊編解碼標準及視訊訊號處理的演算法最近幾年非常蓬勃發展，其目標為提供更好的影像品質、壓縮效果與更快的執行效率。本論文針對多種視訊編標準和相關之演算法，增進其影像品質並設計其系統架構。H.264/AVC之位元流控制和其軟硬體共同設計、具有可計算真實位元率-失真量之H.264/AVC硬體架構設計、改善可調性視訊編碼標準畫質之品質層(quality layer)設定、H.264/AVC解碼程式流程最佳化、影片重要片段擷取、視訊畫面之空間與時間重新取樣等等，為主要研究重點。
首先在H.264/AVC的位元率控制方面，本文以軟硬體共同設計的方式開發出具有低複雜度與低成本之位元率控制演算法，使其更適合於嵌入式系統上實現，與現今的相關研究比較，在硬體面積與複雜度皆大幅降低的情況下，還是能保有較好的影像壓縮品質；此外還提出可算出真實位元率失真量之H.264/AVC編碼器硬體架構和其位元率控制演算法，此位元率控制演算法利用周圍區塊與本身區塊的動態向量和平均絕對值差距(MAD)等資訊來調整位元率控制演算法，跟目前的架構和演算法相比，不但減少真實位元率失真量模組的數量和硬體面積，且可避免H.264/AVC編碼器硬體架構中常見的資料相依性的問題，甚至在壓縮效率上也比目前H.264/AVC參考軟體中的演算法更好。
接著，在第一次視訊編碼時得到的相關影像參數再拿來供第二次編碼時參考，使得影像畫質更穩定或是畫質更好。在H.264/AVC二次編碼方面，本文利用ρ-domain位元率模組提出一個可變位元率編碼演算法使得畫面品質的變動減少。其基本原則為選取一個適當的量化參數(QP)給一群畫面(GOP)做編碼。然後再根據畫面的複雜度去微調量化參數。實驗結果顯示經由H.264/AVC編碼後，即使在有畫面轉換發生的時候，所提出的可變位元率編碼演算法可以明顯的減少畫面品質的變動。在可調性視訊編碼方面，本論文將編碼後的視訊串流中的所有封包做位元率與失真的分析並將第一個MGS的封包重要性提高和重新排列，實驗結果顯示經由所提出之演算法調整後的影像品質可再進一步提昇。
在解碼端方面，在本文中首先將H.264/AVC的解碼程式做資料流程最佳化處理並減少外部記憶體存取的次數，在嵌入式系統版上實做結果顯示我們的程式可加速約3.6倍。解碼之後的視訊資料可再經由所提出之影片重要片段擷取演算法分析出人們可能感興趣的視訊片段並搭配我們設計之硬體架構將此類片段即時地擷取出來；為了符合不同電視規格的播放標準，解碼後之視訊資料還可以搭配時間與空間重新取樣之演算法內插出不同尺寸與播放頻率的畫面，並藉由軟硬體共同設計的觀念設計系統架構，最後在虛擬平台(virtual platform)上實現及驗證此畫面從新取樣系統。模擬結果顯示所提出的方法具有較好的畫面品質。

In recent years, to satisfy the demands of the image quality for human being, the video coding standards and the algorithms for video signal processing are developed extensively. The goals of these standards and the algorithms are to provide better image quality, higher compression ratio, and faster execution speed. This study presents algorithms with improved performance for the various coding standards and their system architectures are proposed. The major researches include the algorithm and the hardware/software codesign architecture of rate control for H.264/AVC, the hardware architecture development for H.264/AVC with the actual rate-distortion cost, the improvement to quality layer assignation in the scalable video coding, the source code flow optimization for H.264/AVC decoders, the highlight extraction, and the system level design of a spatial-temporal video resampling architecture.
Firstly, a low complexity rate control framework for H.264/AVC encoders on embedded systems is proposed. The rate-distortion performance and its implementation complexity using an efficient hardware/software co-design are considered jointly. It is less complex than the rate control module adopted by H.264/AVC JM reference software, making it more suitable for embedded systems. Experiment results show that the proposed rate control outperforms H.264/AVC JM, and the hardware implementation cost is low for an H.264/AVC video encoder. Besides, the mechanisms of rate control without suffering from the data-dependency problem and the H.264/AVC hardware encoder with rate-distortion-optimization (RDO) are proposed. The proposed low complexity MB-level MAD prediction algorithm considers the motion information and the MAD value in neighboring and current MBs jointly. The proposed rate control algorithm does not suffer from the MB-pipeline problems which may decrease the rate-distortion performance, thus they are suitable for H.264/AVC hardware encoders. Furthermore, the RDO-based architecture for H.264/AVC encoder reduces the number of actual rate-distortion cost calculators and keeps the high rate-distortion performance simultaneously. Experimental results show that the proposed rate control algorithm with the RDO-based architecture can improve the rate-distortion performance of H.264/AVC.
The data which are generated in the first pass can be modified to select an appropriate quantization parameter (QP) which is used in the second pass. For H.264/AVC, a two-pass Variable Bit Rate (VBR) coding algorithm based on a group of picture (GOP) level ρ-domain model is proposed. Then, the QP is refined based on the textural complexity of every macroblock (MB). Experimental results demonstrate that the proposed VBR methods not only achieve excellent visual quality but also reduce the variation in distortion even when a scene change happens. For scalable video coding, an algorithm is proposed for improving image quality with original quality layer extraction mechanism. Since the information of the first MGS (medium grain scalability) layer is important, we carry more weight on it when estimating quality information and assigning priority IDs. It shows in our experiment, the PSNR (peak signal-to-noise ratio) is improved after extracting the encoded bitstream with inserted quality information by our proposed algorithm.
When decoding, we use a 4×4 block as an operation unit in the decoding process. The number of external memory access is reduced and the data flow is optimized. By using our proposed method, the average decoding MIPS of one frame is improved about 3.6 times as compared with that of the original one. The decoded video data can be analysed via the proposed hardware architecture of the video highlight extraction algorithm. With the hardware, the video data that people may be interested can be obtained immediately. Besides, the video data can be interpolated with the proposed spatial-temporal resampling system. The algorithm and its system architecture is implemented and verified on a virtual platform via a hardware/software codesign manner. Simulation results show that our method has better image quality.

[1] Information Technology—Generic Coding of Moving Pictures and Associated Audio Information: Video, ISO/IEC 13818-2 and ITU-T Rec. H.262, 1996.
[2] Information Technology—Coding of Audio-Visual Objects—Part 2: Visual, ISO/IEC 14496-2, 1999.
[3] L. Fan, S. Ma, and F. Wu, “Overview of AVS video standard,” in Proc. IEEE Int. Conf. Multimedia and Expo, Jun. 2004, pp. 423-426.
[4] Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification, Joint Video Team, ITU-T Recommendation H.264 and ISO/IEC 14496-10 AVC, May 2003.
[5] H. Schwarz, D. Marpe, and T. Wiegand, “Overview of the scalable extension of the H.264/MPEG-4 AVC video coding standard,” IEEE Trans. Circuits. Syst. Video Technol., vol. 17, no. 9, pp. 1103–1120, Sept. 2007.
[6] ISO-IEC/JTC1/SC29/WG11/N0400. Test Model 5, April 1993.
[7] ITU Telecom. Standardization Sector of ITU, “Video Codec Test Model Near-Term, Version 8 (TMN8), H.263 Ad Hoc Group, Jun. 1997.
[8] ISO/IEC 14496-2 MPEG4 Video VM—Version 8.0, ISO/IEC JTC1/SC29/WG11, 1997, Coding of Moving Pictures and Associated Audio MPEG 97/W1796. Stockholm, Sweden: Video Group.
[9] Z. Li, F. Pan, K. P. Lim, G. Feng, X. Lin, and S. Rahardja, “Adaptive basic unit layer rate control for JVT,” JVT-G012-r1, 7th Meeting, Pattaya II, Thailand, Mar. 2003.
[10] S. Takamura and N. Kobayashi, “MPEG-2 one-pass variable bit rate control algorithm and its LSI implementation,” in Proc. IEEE Int. Conf. Image Processing, Oct. 2001, pp. 942-945.
[11] S. C. Cheng, and H. M. Hang, “The impact of rate control algorithms on video codec hardware design,” in Proc. IEEE Int. Conf. Image Processing, Oct. 1997, pp. 807-810.
[12] H. C. Fang, Y. W. Huang, H. C. Fang, and L. G. Chen, “Hardware oriented rate control algorithm and implementation for realtime video coding,” in Proc. IEEE Int. Conf, Acoust., Speech, and Signal Processing, Apr. 2003, pp. 489-492.
[13] J. C. Tsai, "Rate control for low-delay video using a dynamic rate table," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 1, pp. 133-137, Jan. 2005.
[14] J. W. Gu, “A high quality hardwired rate controller for H.264/AVC real-time video encoding,” Master Thesis, Nat. Tsing Hua Univ., Hsinchu, July 2007.
[15] H. C. Chang, J. W. Chen, C. L. Su, Y. C. Yang, Y. Li, C. H. Chang, Z. M. Chen, W. S. Yang, C. C. Lin, C. W. Chen, J. S. Wang, and J. I. Guo, “A 7mw-to-183mw dynamic quality-scalable H.264 video encoder chip,” in ISSCC Dig. Tech. Papers, Feb. 2007, pp. 280-281.
[16] T. C. Chen, S. Y. Chien, Y. W. Huang, C. H. Tsai, C. Y. Chen, T. W. Chen, and L. G. Chen, “Analysis and architecture design of an HDTV720p 30 frames/s H.264/AVC encoder,” IEEE Trans. Circuits Syst. Video Technol., vol. 16, pp. 673-688, Jun. 2006.
[17] Z. Liu, Y. Song, M. Shao, S. Li, L. Li, S. I. M. Nakagawa, S. Goto, and T. Ikenaga, “A 1.41W H.264/AVC real-time encoder SoC for HDTV 1080P,” in Symp. VLSI Circuits Dig. Tech. Papers, Jun. 2007, pp. 12-13.
[18] K. Babionitakis, G. Lentaris, K. Nakos, D. Reisis, N. Vlassopoulos, G. Doumenis, G. Georgakarakos, and J. Sifnaios, “An efficient H.264 VLSI advanced video encoder,” in Proc. IEEE Int. Conf. Electron. Circuits Syst., Dec. 2006, pp. 545-548.
[19] T. C. Chen, Y. H. Chen, C. Y. Tsai, S. F. Tsai, S. Y. Chien, and L. G. Chen, “2.8 to 67.2mW low-power and power-aware H.264 encoder for mobile applications,” in Symp. VLSI Circuits Dig. Tech. Papers, Jun. 2007, pp. 222-223.
[20] P. T. Wu, T. C. Chang, C. L. Su, and J. I. Guo, “A H.264 basic-unit level rate control algorithm facilitating hardware realization,” in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing, Mar. 2008, pp. 2185-2188.
[21] P. T. Wu, T. C. Chang, J. I. Guo, and C. L. Su,“A H.264 basic-unit level rate control algorithm and architecture for H.264/AVC video encoders,” in Proc. IEEE Asia Pacific Conf. Circuits Syst., Nov. 2008, pp. 1300-1303.
[22] Y. I Jeon, C. H. Han, S. W. Lee, and H. S. Kang, “Fast intra mode decision algorithm using directional gradients for H.264” in Proc. Int. Conf. Image and Signal Processing, Oct. 2009, pp. 1-4.
[23] L. Shen, Z. Liu, Z. Zhang, and X. Shi, “Fast inter mode decision using spatial property of motion field” IEEE Trans. Multimedia, vol. 10, pp. 1208-1214, Oct. 2008.
[24] C. H. Kuo, L. C. Chang, K. W. Fan, and B. D. Liu, “Hardware/software co-design of low cost rate control scheme for H.264/AVC,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, pp. 250-261, Feb. 2010.
[25] L. C. Chang, C. H. Kuo, and B. D. Liu, “Low complexity MAD prediction algorithms for rate controllable H.264/AVC hardware encoders,” in Proc. IEEE Int. Symp. Circuits and Syst., May 2010, pp. 661-664.
[26] P. H. Westerink, R. Rajagopalan, and C. A. Gonzales, “Two-pass MPEG-2 variable-bit-rate encoding,” IBM Journal of Research and Development, vol. 43, pp. 471-488, July 1999.
[27] Y. Yu, J. Zhou, Y. Wang, and C. W. Chen, “A novel two-pass VBR coding algorithm for fixed-size storage application,” IEEE Trans. Circuits Syst.Video Technol., vol. 11, pp. 345-356, Mar. 2001.
[28] Text of ISO/IEC 14496-2 MPEG4 Video VM—Version 8.0, ISO/IEC JTC1/SC29/WG11, 1997, Coding of Moving Pictures and Associated Audio MPEG 97/W1796. Stockholm, Sweden: Video Group.
[29] J. Cai, Z. He, and C. W. Chen, “Optimal bit allocation for low bit rate video stream applications,” in Proc. IEEE Int. Conf. Image Processing, Sept. 2002, pp. 73-76.
[30] A. Jagmohan and K. Ratakonda, “MPEG-4 one-pass VBR rate control for digital storage,” IEEE Trans. Circuits Syst. Video Technol., vol. 13, pp. 447-452, May 2003.
[31] Y. Yokoyama and Y. Ooi, “A scene-adaptive one-pass variable bit rate video coding method for storage media,” in Proc. IEEE Int. Conf. Image Processing, Oct. 1999, pp. 827-831.
[32] B. C. Song and K. W. Chun, “A one-pass variable bit-rate video coding for storage media,” IEEE Trans. Consumer Electron., vol. 49, pp. 689-692, Aug. 2003.
[33] Z. He, W. Zeng, and C. W. Chen, “Low-pass filtering of rate-distortion functions for quality smoothing in real-time video communication,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, pp. 973-981, Aug. 2005.
[34] K. Noh, S. W. Jeong, J. Park, and B. Jeon, “A simple one-pass variable rate control method for fixed-size storage systems,” in Proc. Intl. Tech. Conf. Circuits/Systems, Computers, Communications, July 2002, pp. 289-292.
[35] D. Bagni, B. Biffi, and R. Ramalho, “A constant-quality, single-pass VBR control for DVD recorders,” IEEE Trans. Consumer Electron., vol. 49, pp. 653-662, Aug. 2003.
[36] T. Chiang and Y.-Q. Zhang, “A new rate control scheme using quadratic rate distortion model,” IEEE Trans. Circuits Syst. Video Technol., vol. 7, pp. 246-250, Feb. 1997.
[37] J. Katto and M. Ohta, “Mathematical analysis of MPEG compression capability and its application to rate control,” in Proc. IEEE Int. Conf. Image Processing, Oct. 1995, pp. 555-558.
[38] J. Wang, Z. Chen, Y. He, and Y. Chen, “A MAD-based rate control strategy,” Presented at the 4th Meeting of JVT-D070, Klagenfurt, Austria, Jul., 2002.
[39] Z. He, Y. Kim, and S. K. Mitra, “Low-delay rate control for DCT video coding via ρ-domain source modeling,” IEEE Trans. Circuits Syst. Video Technol., vol. 11, pp. 928-940, Aug. 2001.
[40] Z. He and S. K. Mitra, “A unified rate-distortion analysis framework for transform coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 11, pp. 1221-1236, Dec. 2001.
[41] Z. He and S. K. Mitra, “Optimum bit allocation and accurate rate control for video coding via ρ-domain source modeling,” IEEE Trans. Circuits Syst. Video Technol., vol. 12, pp. 840-849, Oct. 2002.
[42] Z. He and S. K. Mitra, “A linear source model and a unified rate control algorithm for DCT video coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 12, pp. 970-982, Nov. 2002.
[43] Z. He and S. K. Mitra, “ρ-domain bit allocation and rate control for real time video coding,” in Proc. IEEE Int. Conf. Image Processing, Oct. 2001, pp. 546-549.
[44] J. Cai, Z. He, and C. W. Chen, “Optimal bit allocation for low bit rate video stream applications,” in Proc. IEEE Int. Conf. Image Processing, Sept. 2002, pp. 73-76.
[45] I. Amonou, N. Cammas, S. Kervadec, and S. Pateux, “Optimized rate-distortion extraction with quality layers in the scalable extension of H.264/AVC,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, pp. 1186-1193, Sep. 2007.
[46] S. Wang, Y. Yang, C. Li, Y. Tung, and J. Wu, “The optimization of H.264/AVC baseline decoder on low-cost TriMedia DSP processor, ” in Proc. of SPIE Applicat. of Digital Image Processing XXVII, Nov. 2004, pp. 524-535.
[47] X. Zhou, E. Li, and Y. Chen, “Implementation of H.264 decoder on general-purpose processors with media instructions,” in Proc. of SPIE Image Video Commun. Processing, May 2003, pp. 224-235.
[48] K. Ramkishor, and V. Gunashree, “Real time implementation of MPEG-4 video decoder on ARM7TDMI,” in Proc. Int. Intelligent Multimedia, Video and Speech Processing, May 2001, pp. 522-526.
[49] Y. Li, S. H. Lee, C. H. Yeh and C.-C. Jay Kuo, “Techniques for movie content analysis and skimming,” IEEE Signal Processing Mag., vol. 23, no. 2, pp. 79-89, Mar. 2006.
[50] Y. Liu, Z. G. Li, and Y. C. Soh, “Adaptive MAD prediction and refined R-Q model for H.264 rate control,” in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing, May 2006, pp. 905-908.
[51] X. Jing and L. P. Chau, “Improved frame level MAD prediction and bit allocation scheme for H.264/AVC rate control,” in Proc. IEEE Int. Symp. Circuits and Syst., May 2007, pp. 3639-3642.
[52] H. J. Lee, T. Chiang, and Y. Q. Zhang, “Scalable rate control for MPEG-4 video,” IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 878-894, Sep. 2000.
[53] N. Kazakova, M. Margala and N. G. Durdle, “Sobel edge detection processor for a real-time volume rendering system,” in Proc. IEEE Int. Symp. Circuits and Syst., May 2004, pp. II-193-196.
[54] S. E. El-Khamy, M. Lotfy and N. El-Yamany, “A modified fuzzy sobel edge detector,” in Proc. Symp. Radio Science, Feb. 2000, pp. C32/1 - C32/9.
[55] C. H. Chen and C. J. Tsai, “Out-of-loop rate control for video codec hardware/software co-design,” in Proc. IEEE Int. Symp. Circuits and Syst., May 2004. pp. 85-88.
[56] H. Gish, and J. N. Pierce, “Asymptotically efficient quantizing,” IEEE Trans. Inform. Theory, vol. 14, pp. 676-683, Sept. 1968.
[57] A. Ortega and K. Ramchandran, “Rate-distortion methods for image and video Compression,” IEEE Signal Processing Mag., vol. 15, pp. 23-50, Nov. 1998.
[58] G. J. Sullivan and T. Wiegand, “Rate-distortion optimization for video compression,” IEEE Signal Processing Mag., vol. 15, pp. 74-90, Nov. 1998.
[59] X. Tian, T. M. Le, X. Jiang, and Y. Lian, “Full RDO-support power-aware CABAC encoder with efficient content access,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, pp. 1262-1273, Sept. 2009.
[60] R. Hashimoto, K. Kato, G. Fujita, and T. Onoye, “VLSI architecture of H.264 block size decision based on Rate-Distortion Optimization,” in Proc. Int. Symp. Intell. Processing Commun., Dec. 2006, pp. 618-621.
[61] Y. H. Su, “A progressive design flow and its application to H.264 BP RDO encoder VLSI design,” Master Thesis, Nat. Cheng Kung Univ., Tainan, July, 2006.
[62] H. B. Yin, X. Z. Lou, Z. L. Xia, and W. Gao, “An efficient VLSI architecture for rate distortion optimization in AVS video encoder,” in Proc. IEEE Int. Symp. Circuits and Syst., May 2008. pp. 2805-2808.
[63] H. Y. Lin, Y. C. Chao, C. H. Chen, B. D. Liu, and J. F. Yang, “Combined 2-D transform and quantization architectures for H.264 video coders,” in Proc. IEEE Int. Symp. Circuits and Syst., May 2005. pp. 1802-1805.
[64] C. K. Wong, O. C. Au, “Fast motion compensated temporal interpolation for video,” in Proc. SPIE, Visual Commun. Image Processing, May 1995, pp. 1108-1118.
[65] H. Sasai, S. Kondo, S. Kadono, Acoustics, ”Frame-rate up-conversion using reliable analysis of transmitted motion information,” in Proc. IEEE Int. Conf. Speech and Signal Processing, May 2004, pp. 257-260.
[66] W. Lai, X. D. Gu, R. H. Wang, L. R. Dai, H. J. Zhang, “A region based multiple frame-rate tradeoff of video streaming,” in Proc. IEEE Int. Conf. Image Processing, Oct. 2004, pp. 2067-2070.
[67] E. G. T. Jaspers, P. H. N. de With, “A generic 2D sharpness enhancement algorithm for luminance signals,” in Proc. IEEE. Int. Conf. Image Proessing and its Applcations, Jul. 1997, pp. 269-273.
[68] J. Zhai, K. Yu, J. Li, and S. Li, “A low complexity motion compensated frame interpolation method,” in Proc. IEEE Int. Symp. Circuits and Syst., May 2005, pp. 4927-4930.
[69] H. M. Jong, L. G. Chen, and T. D. Chiueh, “Parallel architectures for 3-step hierarchical search block-matching algorithm,” IEEE Trans. Circuit and Syst. Video Technol., vol. 4, pp. 406-417, Aug. 1994.