位元率控制 (Rate control) 在視訊編碼和多媒體串流上扮演相當重要的角色。
一個好的位元率控制機制能夠 1)在有限的編碼預算位元下，有效地調整量化參數，以達到一個好的壓縮畫面品質，以及 2)在所估測的可用頻寬下，能夠調整影音資料的傳輸速率，避免網路的擁塞狀況。在本論文中，我將設計與分析位元率控制在H.264 視訊編碼與在異質網路上之傳輸應用，以下包含了三個重要的議題: 位元率分配與量化參數決定(編碼階段)，頻寬管理與可調式 (scalable) 影音串流同步(串流階段)。
在編碼階段的位元率控制中，我將提出一個利用4-D 以人類感知為基礎之量化模型，包含二個主要模組: 1-D 畫面層上的位元率分配與3-D 巨方塊層(macroblock) 上的量化參數決定機制。一維的時間模式 (temporal pattern) 主要用來估測畫面的複雜度與決定編碼的預算位元率; 三維的位元率模式 (rate pattern)，則是用來建構一個三維的位元率-複雜度-量化模型。其中，此模型的曲線斜率被視為一個決定量化參數的重要因素。此外，為了符合新進畫面的特性，我將利用一個加權最小平方估測 (weighted least square estimation) 來更新此模型的相關係數。
在串流階段的位元率控制中，為了配合時變的頻寬來調整視訊串流的傳輸速率，一個可預測之隨選視訊頻寬管理與回饋式的緩衝區設計是有必要性的。我將根據所量測到的封包往返時間、遺失率、延遲抖動 (delay jitter) 與所接收的位元數，利用卡曼濾波器(Kalman filter) 來估測目前的有效頻寬，並根據解碼端的緩
衝區狀況來調整傳輸位元率。卡曼濾波器中的狀態轉移矩陣 (transition matrix)、錯誤共變異數 (error covariance)，將會被適當地初始化、收歛以配合目前網路的特性。
除了上述的頻寬管理外，可調式影音串流同步也是我們所關心的議題之一。由於異質網路與設備能力的多樣性，串流影音的品質應該能夠被適當地調整。為達到此目的，在過去的可調式多媒體編碼方法，包括FGS (fine-granular scalability)與BSAC (bit-sliced arithmetic coding)，能夠將視訊、音訊資料切割成一個基礎層與多個加強層。考量可調式影音編碼特性、可用頻寬和視訊音訊解碼複雜度，我將設計包含消除延遲抖動機制，條件式的重傳機制與撥放同步機制，來同步地傳輸多階層可調式的影音串流資料。
最後，實驗結果將呈現 1) 所提出之H.264 的位元率控制，相較H.264 JM10.2，能夠有效地維持穩定編碼緩衝區，並改善影像品質; 2) 利用卡曼濾波器所設計的
頻寬管理機制，能夠得到較精確的頻寬大小，並減少串流封包在有線/無線/3G 網路上的遺失率; 3) 所提出的可調式影音串流同步機制能有效地減少延遲抖動的影響，並提高即時 (in-time) 解碼與影音同步撥放的比率。

Bit-rate control plays a major role in video coding and multimedia streaming. Well-designed
bit-rate control mechanisms are able to 1) efficiently adjust encoding quantizers for achieving fine visual qualities in subject to encoding bit budgets and 2) adjust the transmission rate for avoiding network congestion in subject to the predicted available bandwidth. In the dissertation, bit-rate control
mechanisms for H.264 video coding and ransmission over heterogeneous networks are designed and analyzed, including three critical issues: bit allocation and quantizers decision (at the encoding stage), bandwidth management and scalable audiovisual streaming synchronization (at the streaming stage).

For encoding-stage bit-rate control, H.264 bit-rate control using a 4-D perceptual quantization
modeling (PQrc) is proposed, including two major encoding modules: the perceptual frame-level
bit-allocation using a 1-D temporal pattern and the macroblock-level quantizer decision using a 3-D rate pattern. The temporal pattern is used to predict frame complexity and determine proper bit budgets further. The rate pattern is depicted as a bit-complexity-quantization (B.C.Q.) model, in which a tangent slope of a B.C.Q. curve is a piece of unique information to find a proper quantizer.
For newly generated video clips, the B.C.Q. model can be updated continuously using a weighted least square estimation.

For streaming-stage bit-rate control, in order to adapt the transmission rate of streamed ondemand videos to the time-varying channel bandwidth, predictive video-on-demand (VoD) bandwidth management and a feedback-based buffer control scheme are required. According to the measured information of packet round-trip-time, loss-rate, delay jitter and received bit-rate, an improved Kalman filter is adopted to predict an available channel bandwidth recursively and to determine a proper transmission rate in consideration of buffer fullness of a decoder. The optimal parameters of the Kalman filter, e.g., a transition matrix and error covariances, can be initialized, converged and adapted to
characteristics of the current network.

In addition to the aforementioned bandwidth management, scalable audiovisual streaming synchronization is also a main issue we concern. Considering characteristics of heterogeneous networks and device capabilities, streamed audiovisual content is requested to be scaled with the proper spatial/temporal resolution, format and quality levels. Past technologies of scalable audiovisual coding, e.g., Fine-granular scalability (FGS) and bit-sliced arithmetic coding (BSAC), are used to segment video and audio data into one base-layer and multiple enhancement-layer bitstreams. With the advantages
of scalable audiovisual coding, a de-jitter procedure, a conditional retransmission mechanism
and a playout synchronization mechanism are designed to transmit hybrid scalable (multi-layered) audiovisual bitstreams in consideration of the result of a network bandwidth adaptation and distinct decoding time-complexity.
Finally, experimental results show that 1) the proposed H.264 rate control can keep stable buffer fullness and improve the SNR quality and control accuracy effectively in comparison with H.264 JM10.2; 2) the proposed bandwidth management using the improved Kalman filter can obtain more precise and stable estimation results of bandwidth in comparison with pathChirp and decrease the packet loss-rate when data are streamed in real wired/WiFi/3G networks; 3) the proposed scalable
audiovisual streaming synchronization can eliminate effects of the delay jitter, increase the in-time decoding ratio and perceive audiovisual playout smoothly.

[1] E.C. Reed and F. Dufaux. Constrained Bit-rate Control for Very Low Bit-rate Streaming Video
Applications. IEEE Transactions on Circuits and Systems for Video Technology, (7):882–889,
July 2001.
[2] I.M. Pao and M.T. Sun. Encoding Stored Video for Streaming Applications. IEEE Transactions
on Circuits and Systems for Video Technology, (2):199–209, February 2001.
[3] J. Ribas-Corbera and S.M. Lei. A Frame-Layer Bit Allocation for H.263+. IEEE Transactions
on Circuits and Systems for Video Technology, (7):1154–1158, October 2000.
[4] Z. Zhang, G. Liu, H. Li, and Y. Li. A Novel PDE-based Rate-Distortion Model for Rate Control.
IEEE Transactions on Circuits and Systems for Video Technology, (11):1354–1364, November
2005.
[5] L.J. Lin and A. Ortega. Bit-Rate Control Using Piecewise Approximated Rate-Distortion Characteristics.
IEEE Transactions on Circuits and Systems for Video Technology, (4):446–459,
August 1998.
[6] T. Chiang and Y.Q. Zhang. A New Rate Control Scheme Using Quadratic Rate Distortion
Model. IEEE Transactions on Circuits and Systems for Video Technology, (1):246–250, February
1997.
[7] J.H. Park, J.K. Han, and B.C. Song. An Adaptive Quantization using Modified QP in H.264.
Proceedings of IEEE International Conference on Consumer Electronics, pages 229–230, January
2005.
83
[8] K.H. Yang, A. Jacquin, and N.S. Jayant. A Normalized Rate-distortion Model for H.263-
compatible Codecs and Its Application to Quantizer Selection. Proceedings of IEEE International
Conference on Image Processing, pages 41–44, October 1997.
[9] Z. He, Y.K. Kim, and S.K. Mitra. Low-delay Rate control for DCT Video Coding via ρ-Domain
Source Modeling. IEEE Transactions on Circuits and Systems for Video Technology, VOL.11,
NO.8, pp.928-940, (8):928–940, August 2001.
[10] H. Song and C.-C. J. Kuo. Rate Control for Low-bit-rate video via variable-encoding frame
rates. IEEE Transactions on Circuits and Systems for Video Technology, (4):512–521, April
2001.
[11] Z. Li, W. Gao, and F. Pan. Adaptive Rate Control with HRD Consideration, JVT-H017, 8th
Meeting: Geneva. May 2003.
[12] ITU-T SG16 Video Coding Experts Group. Video Codec Test Model, Near-Term, Version 8
(TMN8). September 1997.
[13] D. Longuinov and H. Radha. End-to-End Rate-based Congestion Control: Convergence Properties
and Scalability Analysis. IEEE/ACM Transactions on Networking, (4):564–577, August
2003.
[14] T. Phan, K. Xu, R. Guy, and R. Bagrodia. Handoff of Application Sessions across Time and
Space. Proceedings of International Conference on Communications (ICC), pages 1367–1372,
June 2001.
[15] A. Vetro. MPEG-21 Digital Item Adaptation: Enabling Universal Multimedia Access. IEEE
Multimedia, (1):84–87, January-March 2004.
[16] M. Handley R. Rejaie and D. Estrin. Layered Quality Adaptation for Internet Video Streaming.
IEEE Journal on Selected Areas in Communications, (12):2530–2543, December 2000.
[17] Y. Ishibashi and S. Tasaka. A Comparative Survey of Synchronization Algorithms for Continuous
Media in Network Environments. Proceedings of IEEE International Conference on Local
Computer Networks, pages 337–348, November 2000.
84
[18] N. Laoutaris and I. Stavrakakis. Intrastream Synchronization for Continuous Media Streams: a
Survey of Playout Schedulers. IEEE Multimedia, (3):30–40, May-June 2002.
[19] C.M. Huang, C.W. Lin, C.C. Yang, and X.Y. Lin. Network-aware Multimedia Streaming Using
the Kalman Filter over theWired/Wireless/3G Networks. Proceedings of the IEEE International
Conference on Multimedia & Expo, pages 923–926, 2007.
[20] C.M. Huang, C.W. Lin, and X.Y. Lin. A Predictive Video-on-Demand Bandwidth Management
Using the Kalman Filter over Heterogeneous Networks. The Computer Journal,
doi:10.1093/comjnl/bxn011, March 2008.
[21] J. Jo and J.K. Kim. Evolution on the Performance of Adaptive Playout for the Multicast Streaming
of Store Media. Proceedings of IEEE International Conference on Communications, pages
542–546, May 2003.
[22] H. Liu and M. EI Zarki. An Adaptive Delay and Synchronization Control Scheme for Wi-Fi
based Audio/Video Conferencing. ACM Wireless Networks, (4):511–522, July 2006.
[23] H. Zhu, J.A. Cobb I. Chlamtac, and G. Zeng. SMART: a Synchronization Scheme for Providing
Multimedia Quality in Emerging Wireless Internet. Proceedings of IEEE International
Conference on Vehicular Technology, pages 3390–3394, October 2003.
[24] C. Perkins, O. Hodson, and V. Haardman. A Survey of Packet Loss Recovery Techniques for
Streaming Audio. IEEE Network, (5):40–48, September 1998.
[25] H. Liu, W.J. Zhang, S.Y. Yu, and J. Cai. A Client-driven Scalable Cross-layer Retransmission
Scheme for 3G Video Streaming. Proceedings of IEEE International Conference on Multimedia
and Expo., CD-ROM 2005.
[26] H. Liu,W. Zhang, and X. Yang. Retransmission-based Error spreading for layered video streaming
over wireless LANs. Proceedings of IEEE International Symposium on Circuits and Systems,
pages 21–24, May 2006.
85
[27] S. Aramwith, C.W. Lin, S. Roy, and M.T. Sun. Wireless Video Transport using Conditional
Retransmission and Low-delay Interleaving. IEEE Transactions on Circuits and Systems for
Video Technology, (6):558–565, June 2002.
[28] S. Tasaka, T. Nunome, and Y. Ishibashi. Live Media Synchronization Quality of a
Retransmission-based Error Recovery Scheme. Proceedings of IEEE International Conference
on Communications, pages 1535–1541, June 2000.
[29] ISO/IEC JTC1/SC29/WG11. MPEG-4 Video Verification Model v18.0, Coding of Moving
Pictures and Audio N3908. January 2001.
[30] A. Vetro, H. Sun, and Y. Wang. MPEG-4 Rate Control for Multiple Video Objects. IEEE
Transactions on Circuits and Systems for Video Technology, (1):186–199, Febuary 1999.
[31] F. Pan, Z. Li, K. Lim, and G. Feng. A Study of MPEG-4 Rate Control Scheme and Its Improvements.
IEEE Transactions on Circuits and Systems for Video Technology, (5):440–446, May
2003.
[32] H.J. Lee, T. Chiang, and Y.Q. Zhang. Scalable Rate Control for MPEG-4 Video. IEEE Transactions
on Circuits and Systems For Video Technology, (6):878–894, September 2000.
[33] K. N. Ngan, T. Meier, and Z. Chen. Improved Single Video Object Rate Control for MPEG-4.
IEEE Transactions on Circuits and Systems for Video Technology, (5):385–393, May 2003.
[34] ISO/IEC JTC1, Information Technology- Coding of Audio-Visual Objects- Part 10: Advanced
Video Coding, ISO/IEC FDIS 14496-10. 2003.
[35] P. Lambert, W. De Neve, P. De Neve, I. Moerman, P. Demeester, and R. Vande Walle. Ratedistortion
Performance of H.264/AVC Compared to State-of-the-art Video Codecs. IEEE Transactions
on Circuits and Systems For Video Technology, (1):134–140, January 2006.
[36] S. Miyaji, Y. Takishima, and Y. Hatori. A Novel Rate Control Method for H.264 Video Coding.
Proceedings of IEEE International Conference on Image Processing, pages 309–312, September
2005.
86
[37] S. Ma, W. Gao, and Y. Lu. Rate-Distortion Analysis for H.264/AVC Video Coding and its
Application to Rate Control. IEEE Transactions on Circuits and Systems for Video Technology,
(2):1533–1544, December 2005.
[38] JVT/AVC reference software, ”http://iphome.hhi.de/suehring/tml/download/”.
[39] T. Wiegand, G.J. Sullivan, G. Bjntegaard, and A. Luthra. Overview of the H.264/AVC Video
Coding Standard. IEEE Transactions on Circuits and Systems for Video Technology, (7):560–
576, July 2003.
[40] [Online]. Available: http://www.pixeltools.com/rate control paper.html .
[41] C.T. Chen. Linear System Theory and Design. Third edition. Oxford University Press, 1999.
[42] Casetti, Gerla C., Mascolo M., Sanadidi S., M.Y., and R. Wang. TCP Westwood: End-to-End
Congestion Control for Wired/Wireless Networks. ACM Transactions on Wireless Networks,
(8):467–479, 2002.
[43] F. Martignon and A. Capone. TCP with Bandwidth Estimation over Wireless Networks. Proceedings
of International Conference on Vehicular Technology, pages 1422–4126, September
2002.
[44] A. Capone, L. Fratta, and F. Martignon. Bandwidth Estimation Schemes for TCP over Wireless
Networks. IEEE Transactions on Mobile Computing, (2):129–143, April-June 2004.
[45] B. Melander, M. Bjorkman, and P. Gunningberg. A New End-to-End Probing and Analysis
Method for Estimating Bandwidth Bottlenecks. Proceedings of International Conference on
Global Telecommunications (Globecom), pages 415–421, December 2000.
[46] M. Jain and C. Dovrolis. End-to-End Available Bandwidth: Measurement Methodology, Dynamics,
and Relation with TCP Throughput. IEEE/ACM Transactions on Networking, (4):536–
549, August 2003.
87
[47] M. Jain and C. Dovrolis. Pathload: A measurement Tool for End-to-End Available Bandwidth.
Proceedings of International Workshop on Passive and Active Measurements (PAM), March
2002.
[48] V.J. Ribeiro, R.H. Riedi, and R.G. Baraniuk. pathChirp: Efficient Available Bandwidth Estimation
for Network Paths. Proceedings of International Workshop on Passive and Active
Measurements (PAM), Apirl 2003.
[49] M. Chen and A. Zakhor. AIO-TFRC: A Light-weight Rate Control Scheme for Streaming over
Wireless. Proceedings of International Conference on Wireless Networks, Communications and
Mobile Computing, pages 1124–1129, June 2005.
[50] C. Song, P.C. Cosman, and G.M. Voelker. End-to-end Differentiation of Congestion and Wireless
Losses. IEEE/ACM Transactions on Networking, (5):703–717, October 2003.
[51] F. Yang, Q. Zhang, W. Zhu, and Y.Q. Zhang. End-to-End TCP-Friendly Streaming Protocol and
Bit-Allocation for Scalable Video over Wireless Internet. IEEE Journal on Selected Areas in
Communications, (4):777–790, May 2004.
[52] M. Chen and A. Zakhor. Rate Control for Streaming Video over Wireless. IEEE Wireless
Communication, (4):32–41, August 2005.
[53] P. van Beek, S. Deshpande, H. Pan, and I. Sezan. Adaptive Streaming of High-Quality Video
over Wireless LANs. Proceedings of SPIE on Visual Communications and Image Processing,
pages 647–660, January 2004.
[54] D.H. Hoang and D. Reschke. An Adaptive Control Scheme for Multimedia Flows overWireless
Networks. Proceedings of International Forum on Multimedia and Image Processing, pages
325–330, June 2002.
[55] W.P. Li. Overview of Fine Granularity Scalability in MPEG-4 Video Standard. IEEE Transactions
on Circuits and Systems for Video Technology, (3):301–317, March 2001.
88
[56] M. van der Schaar and H. Radha. A Hybrid Temporal-SNR Fine-Granular Scalability for Internet
Video. IEEE Transactions on Circuits and Systems for Video Technology, (3):318–331,
March 2001.
[57] S. Lee, M. Choi, J. Kim, D. Kim, N. Eum, and H. Jung. The MPEG-4 BSAC Audio Decoder
in Terrestrial DMB Receiver. Proceedings of IEEE International Conference on Consumer
Electronics, pages 257–258, January 2006.
[58] B. Feiten, I. Wolf, E. Oh, J. Seo, and H.K. Kim. Audio Adaptation according to Usage Environment
and Perceptual Quality Metrics. IEEE Transactions on Multimedia, (3):446–453, June
2005.
[59] M. Ries, R. Puqlia, T. Tebaldi, O. Nemethova, and M. Rupp. Audiovisual Quality Estimation
for Mobile Streaming Services. Proceeding of IEEE Symposium on Wireless Communication
Systems, pages 173–177, 2005.
[60] J. Webb and K. Oehler. A Simple Rate-Distortion Model, Parameter Estimation, and Application
to Real-Time Rate Control For DCT-Based Coders. Proceedings of IEEE International
Conference on Image Processing, pages 13–16, October 1997.
[61] M. Jiang and N. Ling. An Improved Frame and Macroblock Layer Bit Allocation Scheme for
H.264 Rate Control. Proceedings of IEEE International Symposium on Circuits and Systems,
pages 1501–1504, May 2005.
[62] K.C. Lai, S.C. Wang, and D. Lun. A Rate Control Algorithm Using Human Visual System
for Video Conferencing Systems. Proceedings of IEEE International Conference on Signal
Processing, pages 656–659, August 2002.
[63] R.E. Kalman. A New Approach to Linear Filtering and Prediction Problems. Transaction of the
American Society of Mechanical Engineers-Journal of Basic Engineering, pages 35–45, 1960.
[64] E.W. Kamen and J.K. Su. Introduction to Optimal Estimation. Springer-verlag, 1999.
[65] C.M. Huang, C.W. Lin, and C.Y. Chuang. A Multiple Layered Audiovisual Streaming System
Using the Two-phase Synchronization and FGS/BSAC Techniques. Proceedings of the 22th
89
IEEE International Conference on Advanced Information Networking and Applications, pages
174–180, March 2008.
[66] C.U. Castellanos, D.L. Villa, O.M. Teyeb, J. Elling, and J. Wigard. Comparison of Available
Bandwidth Estimation Techniques in Packet-switched Mobile Networks. Proceedings of IEEE
International Symposium on Personal, Indoor and Mobile Radio Communications, September
2006.
[67] Z. Dziong, M. Juda, and L.G. Mason. A Framework for Bandwidth Management in ATM
Networks-aggregate Equivalent Bandwidth Estimation Approach. IEEE/ACM Transactions on
Networking, (1):134–147, Febuary 1997.
[68] G. Razzano, H.T. Tran, and C. Cantarella. 17 GHz Wireless LAN: Performance Analysis of
CAC Algorithms. Proceedings of International Symposium on Computers and Communications
(ISCC), pages 786–791, June 2005.
[69] Technical Report CSE-TR-432-00 (2000) SANE: Stable Agile Network Estimation. University
of Michigan, Department of Electrical Engineering and Computer Science, Ann Arbor, MI,
2000.
[70] E. Hartikainen and S. Ekelin. Tuning the Temporal Characteristics of a Kalman-Filter Method
for End-to-End Bandwidth Estimation. Proceedings of IEEE/IFIP International Workshop on
End-to-End Monitoring Techniques and Services, pages 58–65, April 2006.
[71] S. Ekelin, M. Nilsson, E. Hartikainen, A. Johnsson, J.E. Mangs, M. Melander, and M. Bjorkman.
Real-Time Measurement of End-to-End Available Bandwidth using Kalman Filtering.
Proceedings of IEEE/IFIP International Symposium on Network Operations and Management,
pages 73–84, April 2006.
[72] C.M. Huang and C.W. Lin. H.264 Bit-rate Control Using the 3-D Perceptual Quantization
Modeling. Proceedings of the 49th IEEE Globecom Conference, 2006.
[73] C.M. Huang and C.W. Lin. A Novel 4-D Perceptual Quantization Modeling for H.264 Bit-rate
Control. IEEE Transactions on Multimedia, (6):1113–1124, October 2007.
[74] C.M. Huang, C.W. Lin, and C.C. Yang. Layer 5 Session Handoff: Mobility Management for
Video Sessions among Heterogeneous Networks and Devices. accepted by IEEE Multimedia,
2008.