近年來，數位影像及視訊已變得越來越熱門且大眾化。當人們在使用這些科技時，人們最在乎的是這些影像及視訊的品質。通常人們會期望可以有高解析度影像，因為高解析度影像可以提供比低解析影像更多的影像細節。然而，在很多情況下人們無法得到高解析度影像，例如受限於通訊頻寬的限制、資料傳輸、硬體製造等問題。為了克服這些障礙，很多方法被提出。本論文由訊號處理的觀點來解決這個問題，即是所謂的超解析度重建。近來超解析度重建的研究領域已延伸到壓縮視訊上，因為現今視訊壓縮的普遍性。

本論文探討的是如何由低解析度視訊序列重建出相對的高解析度視訊序列。本論文採用Bayesian maximum a-posteriori (MAP)估測技術，因它具有合併事前資訊的能力，也可以使用Huber-Markov random field (HMRF)作為影像模型。本論文提出以Cross-Diamond-Hexagonal Search (CDHS)作為運動向量估測方法來節省處理時間。此外，小波模組被用來取代傳統高通濾波器，因為它可以準確保留影像邊緣，也具有良好的雜訊對抗能力。

實驗證明本論文所提出的方法不僅在視覺上或數值上都比鄰近點、雙線性、雙立方影像內插表現好，而且也能大幅減少原始超解析度重建方法的處理時間。當使用未壓縮序列，論文提出的方法比上述三種內插法在peak signal to noise ratio (PSNR)上分別改善 4.12dB、1.86dB、1.04dB。而與原始的MAP超解析度法相比，雖然在PSNR僅略有改善，但它在處理時間上大幅減少了約50%。在壓縮過的序列情形下，本論文的方法在PSNR比上述三種內插法分別改善 3.13dB、1.24dB、0.65dB，與原始的超解析度法比較，雖然在PSNR上僅略有改善，但與未壓縮的情形相同，它在處理時間上有明顯的減少。

Digital image and video have become more and more popular in a couple of decades. The first of our concerns is the visual quality of the video and image. In general, we would like to have a media source which is of high-resolution (HR) since it can provide more detail information than a low-resolution (LR) one. Nevertheless, HR images and videos are often unavailable in many applications due to their native objection, such as bandwidth limitation, the data transmission problems, and the hardware manufacturing. To solve the problem, people have developed many techniques. The method we utilize in this thesis mainly focuses on the approach presented from signal processing point of view. More recently, the attention of Super-resolution (SR) algorithms has been put on the compressed video sequences because of the prosperity of video compression, like the MPEG and ITU standards. In the thesis, we apply the SR reconstruction method to uncompressed and compressed video sequences.

This thesis investigates how to find the HR video sequence which is closest to the original one. The proposed method adopts the Bayesian maximum a-posteriori (MAP) estimation technique to estimate the best solution because it has an excellent ability of incorporating useful prior information, like the Huber-Markov random field (HMRF) image model. The proposed method uses the Cross-Diamond-Hexagonal Search (CDHS) as the block-based motion estimation strategy to reduce the processing time. Furthermore, the wavelet modulus is used to replace the traditional high-pass filer (HPF) because it enhances the ability of detecting edges in HR frames as well as blockade the sporadic noises.

The experiments demonstrate that the proposed MAP-based method outperforms the conventional image interpolation methods in both visual and quantitative comparisons and saves most of the processing time of the traditional SR method. In the case of uncompressed video sequences corrupted by Gaussian noises, the proposed method gives 4.12dB, 1.86dB, and 1.04dB improvements over the nearest-neighbor, bilinear, and bicubic interpolation, respectively in terms of PSNR. When compared to the original Bayesian MAP SR method, the proposed method performs slightly better but saves 50% of the processing time of the former. In the other case of compressed video, the improvements of PSNR over the three image interpolation methods are 3.13dB, 1.24dB, and 0.65dB, respectively. Like the case of the uncompressed video, the proposed method performs slightly better than the original Bayesian MAP SR method but saves about 50% of the processing time for the compressed video.

[1] Y. Altunbasak, A. J. Patti, and R. M. Mersereau, “Super-Resolution Still and Video Reconstruction from MPEG-Coded Video,” IEEE Trans. Circuits and Systems for Video Technology, Vol. 12, No. 4, pp. 217-226, April 2002.
[2] L. D. Alvarez, R. Molina, and A. K. Katsaggelos, “Multi-Channel Reconstruction of Video Sequences from Low-Resolution and Compressed Observations” in 8th Iberoamerican Congress on Pattern Recognition (CIARP’2003), edited by Alberto Sanfeliu and José Ruiz-Schulcloper, Vol. I, 46-53, Havana (Cuba), November 2003.
[3] M. R. Banham and A. K. Katsaggelos, “Digital Image Restoration,” IEEE Signal Proc. Magazine, Vol. 14, No. 2, pp. 24-41, March 1997.
[4] S. Borman and R. L. Stevenson, “ Super-Resolution from Image Sequences – A Review,” in Proc. 1998 Midwest Symp. Circuits and Systems, 1999, pp. 374-378.
[5] S. Borman and R. L. Stevenson, “Simultaneous Multi-Frame MAP Super-Resolution Video Enhancement using Spatio-Temporal Priors,” in Proc. 1999 IEEE Int. Conf. Image Processing, Vol. 3, pp. 469-473, Kobe, Oct. 1999.
[6] N. K. Bose, S. Lertrattanapanich, and J. Koo, “Advances in Superresolution Using L-curve,” in Proc. Int. Symp. Circuits and Systems, Vol. 2, pp. 433-436, 2001.
[7] K. R. Castleman, Digital Image Processing. NJ: Prentice Hall, 1996.
[8] C. H. Cheung and L. M. Po, “Novel Cross-Diamond-Hexagonal Search Algorithms for Fast Block Motion Estimation”, IEEE Trans. Multimedia, Vol. 7, No. 1, pp. 16-22, February 2005.
[9] M. Elad and A. Feuer, “Super-Resolution Reconstruction of Image Sequences,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21, No. 9, pp. 817-834, Sep. 1999.
[10] H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems. Kluwer Academic Publishers, 1996.
[11] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd. NJ: Prentice Hall, 2002.
[12] J. Hadamard, Lectures on the Cauchy Problem in Linear Partial Differential Equations. New Haven: Yale University Press, 1923.
[13] M. C. Hong, M. G. Kang, and A. K. Katsaggelos, “An Iterative Weighted Regularized Algorithm for Improving the Resolution of Video Sequences,” in Proc. Int. Conf. Image Processing, Vol. 2, pp.474-477, 1997.
[14] K. P. Hong, J. K. Paik, H. J. Kim, and C. H. Lee, “An Edge-Preserving Image Interpolation System for a Digital Camcorder,” IEEE Trans. Consumer Electronics, Vol. 42, No. 3, pp. 279-284, August 1996.
[15] H. S. Hou and H. C. Andrews, “Cubic Splines for Image Interpolation and Digital Filtering,” IEEE Trans. Acoust. Speech, Signal Proc., Vol. ASSP-26, pp. 508-517, December 1978.
[16] M. Irani and S. Peleg, “Improving Resolution by Image Registration,” CVGIP: Graphical Models and Image Processing, Vol. 53, pp.231-239, May 1991.
[17] M. Irani and S. Peleg, “Motion Analysis for Image Enhancement: Resolution, Occlusion and Transparency. Journal of Visual Communication and Image Representation, Vol. 4, pp. 324-335, Dec. 1993.
[18] M. G. Kang and A. K. Katsaggelos, “Simultaneous Iterative Image Restoration and Evaluation of the Regularization Parameter,” IEEE Trans. Signal Process., Vol. 40, No. 9, pp. 2329~2334, Sept. 1992.
[19] P. V. Karunaratne, “Compressed Video Enhancement using Regularized Iterative Restoration Algorithms,” Master Thesis, Dept. of Electrical and Computer Engineering, Northwestern University, Dec. 1999.
[20] J. B. Keller, “Inverse Problems,” American Mathematical Monthly, Vol. 83, No.2, pp. 107-118, 1976.
[21] S. P. Kim, N. K. Bose, and H. M. Valenzuela, “Recursive Reconstruction of High Resolution Image from Noisy Undersampled Multiframes,” IEEE Trans. Acoust. Speech, and Signal Proc., Vol. 38, No. 6, pp. 1013-1027, June 1990.
[22] S. P. Kim and W. Y. Su, “Recursive High-Resolution Reconstruction of Blurred Multiframe Images,” IEEE Trans. Image Processing, Vol. 2, pp. 534-539, Oct. 1993.
[23] A. Kirsch, An Introduction to the Mathematical Theory of Inverse Problems, volume 120 of applied mathematical sciences. Springer, 1996.
[24] C.W. Lam, L.M Po, and C. H. Cheung, “A Novel Kite-Cross-Diamond Search Algorithm for Fast Video Coding and Videoconferencing Applications”, in Proc. of the IEEE ICASSP, Vol. 3, pp. 365-368, May 17-24, 2004.
[25] Stan Z. Li, Markov Random Field Modeling in Image Analysis. NY: Springer-Verlag, 2001.
[26] Stan Z. Li, “Modeling Image Analysis Problems Using Markov Random Fields.” Handbook of Statics 21: Stochastic Process Modeling and Simulation. Ed. D.N. Shanbhag and C.R. Rao. Boston: Elsevier, 2003.
[27] J. Mateos, A. K. Katsaggelos, and R. Molina, “Simultaneous Motion Estimation and Resolution Enhancement of Compressed Low Resolution Video,” in Proc. 2000 IEEE Int. Conf. Image Processing, Vol. 2, pp. 653-656, Vancouver, BC, Canada, Sept. 2000.
[28] R. Pan and S. J. Reeves, “ Fast Restoration with Edge-Preserving Regularization,” in Proc. 2005 Southeastern Symp. System Theory, pp. 471-474, 2005.
[29] S. C. Park, M. K. Park, and M. G. Kang, “ Super-Resolution Image Reconstruction: A Technical Overview,” IEEE Signal Processing Magazine, Vol. 20, No. 3, pp. 21-36, May 2003.
[30] D. Rajan and S. Chaudhuri, “An MRF-Based Approach to Generation of Super-Resolution Images from Blurred Observations,” Journal of Mathematical Imaging and Vision, Vol. 16, No. 1, pp. 5-15, Jan. 2002.
[31] R. R. Schultz and R. L. Stevenson, “A Bayesian Approach to Image Expansion for Improved Definition,” IEEE Trans. Image Processing, Vol. 3, No. 3, pp. 233-242, May 1994.
[32] R. R. Schultz and R. L. Stevenson, “ Extraction of High-Resolution Frames from Video Sequences,” IEEE Trans. Image Processing, Vol. 5, No. 6, pp. 996-1011, June 1996.
[33] C. A. Segall, R. Molina, A. K. Katsaggelos, and J. Mateos, “Bayesian High-Resolution Reconstruction of Low-Resolution Compressed Video,” in Proc. 2001 IEEE Int. Conf. Image Processing, Vol. 2, pp. 25-28, Thessaloniki, Greece, Oct. 2001.
[34] C. A. Segall, R. Molina, and A. K. Katsaggelos, “High-Resolution Images from Low-Resolution Compressed Video,” IEEE Signal Processing Magazine, Vol. 20, No. 3, pp. 37-48, May 2003.
[35] C. A. Segall and A. K. Katsaggelos, “Enhancement of Compressed Video using Visual Quality Measurements,” in Proc. 2000 IEEE Int. Conf. Image Processing, Vol. 2, pp. 645-648, Vancouver, BC, Canada, Sept. 2000.
[36] C. A. Segall and A. K. Katsaggelos, “Pre- and Post-Processing Algorithms for Compressed Video Enhancement,” in Proc. of the Asilomar Conference on Systems, Signals and Computer, Vol. 2, pp. 1369-1373, Pacific Grove, CA, 29 Oct.-1 Nov. 2000.
[37] C. A. Segall and A. K. Katsaggelos, “A New Constraint for the Regularized Enhancement of Compressed Video,” in Proc. of the IEEE ICASSP, Vol. 3, pp. 1849-1852, Salt Lake City, Utah, May 7-11, 2001.
[38] C. A. Segall, A. K. Katsaggelos, R. Molina, and J. Mateos, “Super-Resolution from Compressed Video,” Super-Resolution Imaging, S. Chaudhuri, editor, Chapter 9, pp.211-242, Kluwer Academic Publishers, 2001.
[39] C. A. Segall, A. K. Katsaggelos, R. Molina, and J. Mateos, “Bayesian Resolution Enhancement of Compressed Video,” IEEE Trans. Image Processing, Vol. 13, No. 7, pp. 898-911, July 2004.
[40] H. Stark and P. Oskoui, “High-Resolution Image Recovery from Image-Plane Array, using Convex Projection,” J. Opt. Soc. Amer. A, Vol. 6, No. 11, pp. 1715-1726, 1989.
[41] R. L. Stevenson and S. Borman, “Image Sequence Processing,” Encyclopedia of Optical Engineering. Ed. Ronald Driggers. NY: Marcel Dekker, 840-879, Sep. 2003.
[42] S. R. Subramanya, H. Patel, and I. Ersoy, “Performance evaluation of block-based motion estimation algorithms and distortion measures”, in Proc. 2004 IEEE Int. Conf. Information Technology: Coding and Computing, Vol. 2, No. 1, pp. 2-7, 2004.
[43] A. M. Tekalp, Digital Video Processing. NJ: Prentice Hall, 1995.
[44] A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-Resolution Image Reconstruction from Low-Resolution Image Sequences and Space-Varying Image Restoration,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, pp. 169-172, Mar. 1992.
[45] P. Thévenaz, T. Blu, Member, and M. Unser, “Interpolation Revisited,” IEEE Trans. Medical Imaging, Vol. 19, No. 7, pp. 739 -758, July 2000.
[46] A. Tikhonov and V. Arsenin, Solutions of Ill-Posed Problems. Washington, DC: V. H. Winston & Sons, 1977.
[47] B. C. Tom and A. K. Katsaggelos, “Reconstruction of a High-Resolution Image by Simultaneous Registration, Restoration, and Interpolation of Low-Resolution Images.” in Proc. 1995 IEEE Int. Conf. Image Processing, Vol. 2, pp. 539-542, Washington, DC, Oct. 1995.
[48] R. Y. Tsai and T. S. Huang, “Multiframe Image Restoration and Registration,” Advances in Computer Vision and Image Processing, Vol. 1, pp. 317-339, 1984.
[49] J. H. Xu, “A Wavelet-Based Edge-Preserving Method for Superresolution Reconstruction from a Single Image,” Master Thesis, Dept. of Electrical Engineering, NCKU, June 2004.
[50] XviD Homepage, < http://www.xvid.org >
[51] 戴顯權, 陳瀅如, 王春清, 多媒體通訊: 原理﹑標準與系統, 高雄市: 紳藍, 2001.