近年來，影像超解析(super resolution)技術已在日常生活中被廣泛的使用。隨著現今數位顯示器解析度的改善，從1920 X 1080 (1080p, full HD)提升到3840 X 2160 (2160p, ultra HD)或是更高，這個議題也變得備感興趣且重要。超解析主要的目的在於由低解析度(low-resolution, LR)的影像來獲得高解析度(high-resolution, HR)的影像，而且更要使得這些高解析度的影像看起來就像是經由預期的高解析度感應器(sensor)所取得，或者是至少要讓它們看起來盡可能的"自然"。一般來說，超解析演算法可大致上區分為四種類別，分別為以內插為基底(interpolation-based)、以重建為基底(reconstruction-based)、以小波為基底(wavelet-based)以及以學習為基底(learning-based)等的方式。每個類別的方法通常都具備了各自的優點及缺點，因此在近期的研究中，這些方法的結合被進一步的提出以達到更好的效能。

在本論文中首先提出的演算法結合了以重建為基底以及以小波為基底的方法之優點。在此演算法中，我們對小波轉換後之影像的3個高頻(high-frequency)子頻帶(LH、HL和HH)中的小波係數進行迭代精鍊(iterative refinement)的處理而不是直接對於空間域(spatial domain)中的原影像像素進行處理。此外，為了進一步滿足硬體上的需求，我們也提出了一個可以適應特定的暫存器(buffer)大小的修改版本。在模擬的結果中可以明顯的看到，所提出的演算法可以提供較佳的效能。

以重建為基底的演算法通常在細節的呈現上有不錯的效能，而在這個類別中一個很普遍使用的概念是使用預先定義好的條件來對於初步放大得到的高解析度影像進行迭代精鍊以獲得最終的高解析度影像。也就是說，這可以視為是把細節適當地加回到初步放大的高解析度影像以得到最後的高解析度影像。而在這樣的類別中，迭代式的精鍊處理通常會導致可觀的計算複雜度，而且一些固定的參數值有時候無法充分地根據各式各樣的影像的特性來對細節進行進行適應性地精鍊。

線性迴歸(linear regression)是在統計中一個被廣泛使用的工具，而因其預測的能力，近期也被應用於在超解析技術中。在本論文中，我們妥善地運用了線性迴歸以及一對低解析度與高解析度影像之間的自相似性來解決上述的問題。不同於其他以迴歸來預測內插像素的超解析演算法，我們提出的演算法是利用迴歸以及以重建為基底之超解析演算法的優點來建立細膩且自然的高解析度影像。經由快速內插所得到的高解析度影像通常看起來比較模糊(blurred)，而這可以視為在影像放大的過程中有部分的細節遺失了，而如何去預估這些細節是我們演算法中最重視的事情。首先我們提出一個有效率的超解析演算法，基於一對低解析度以及高解析度影像之間的自相似性(self-similarity)，我們將經由低解析度影像的區塊中所取得的細節所建立的簡單線性迴歸模型用來預測高解析度影像中的細節。而模擬的實驗結果也可看出我們所提出的超解析演算法不但是具有成效的，而且也是有效率的。

在另外一個進一步提出的超解析演算法中，我們假設這些各式各樣的細節可以視為由幾個具有方向性的細節(oriented detail)所組成。首先我們設計了8個濾波器(filter)來適當地取得對應的方向性細節。接下來我們把從低解析度影像中經由所設計的濾波器所取得的方向性細節用來建立多重線性迴歸模型(multiple linear regression model)，並且利用這個模型以及從初步的高解析度影像中經由相同濾波器所取得的方向性細節來預測高解析度影像中的細節。為了能夠更適應性地利用影像中不同區域的特性，我們也提出一個預先將輸入影像切割再處理的修改版本。由模擬的結果可以清楚的看到，我們所提出的演算法不管在主觀或是客觀的量測標準中都有很好的表現。

以學習為基底的超解析演算法最大的優點是在於建立"自然"細節的能力，而向量量化(vector quantization, VQ)則是在此類別中一個普遍使用的工具。在本論文中，我們也考慮了這個優點並提出一個使用加權向量量化(weighted VQ)的超解析演算法。不同於其他使用向量量化的超解析演算法，由於我們使用多重線性迴歸來組合從編碼簿中所選取碼向量(code vector)而不是使用從編碼簿(codebook)中直接選取碼向量來做為要加回的細節，因此在選擇最相似於輸入區塊(input patch)的碼向量時，我們提出了以相關係數(correlation coefficient)取代歐幾里德距離(Euclidean distance)來做為判斷依據的概念。如此一來就可以降低因為編碼簿的大小而對於向量量化之效能所造成的影響，換句話說，我們可以降低儲存編碼簿所需花費的成本並且同時保持原本的效能甚至可以做的更好。由模擬的結果也可以清楚的看到，的確達到了所提出演算法的目的而且真實的實驗數據也都有達到預期。

在本論文中也提出了一個可以呈現更多的細節並同時避免引起明顯的鋸齒狀效應(jaggy artifact)的超解析演算法。因為其具有可以建立清楚的細節能力，我們採用快速的碎型(fractal)超解析技術來獲得初步放大的高解析度影像。接下來，我們提出了一個後處理(post-processing)。經由使用預先設計的方向性濾波器(oriented filter)跟圖樣(pattern)來進行方向性的模糊(blurring)跟強化(enhancing)，我們可以減少沿著斜邊的那些明顯的鋸齒狀效應。實際上，這個後處理可以用來有效的減少因為超解析演算法所引起的鋸齒狀效應。由模擬的結果中也可以發現，高解析度影像中的細節可以清楚地呈現出來而且斜邊上的鋸齒狀效應也大幅的減少了。

In recent years, applications of image super-resolution (SR) technologies have been widely used in daily lives. With the resolution improvement of digital displays nowadays, from 1920 X 1080 (1080p, full HD) to 3840 X 2160 (2160p, ultra HD) even more, this issue becomes more and more interesting and important. The main purpose of super resolution is to obtain high-resolution (HR) images from low-resolution (LR) ones and moreover, makes the former look like they have been acquired with a sensor having the expected high resolution or at least, as "natural" as possible. In general, SR algorithms can be roughly classified into 4 categories that respectively are interpolation-based, reconstruction-based, wavelet-based, and learning-based algorithms. There are usually respective advantages and drawbacks in each category, and thus some combinations of these are further proposed in recent researches to achieve better performance.

In this thesis, an SR algorithm combining advantages of reconstruction-based and wavelet-based categories is first proposed in which an iterative refinement process is performed on wavelet coefficients in high-frequency subbands (LH, HL, and HH) of a wavelet transformed image rather than directly on the original pixels in the spatial domain. Besides, to further meet the hardware requirement, a modified version which is adaptive to a specified buffer size is also presented. In the experimental results, it is obvious that the proposed algorithms provide better performance.

Reconstruction-based SR algorithms usually perform well on detail exhibition, and a popular concept used in this category is to iteratively refine a preliminary HR image using a certain predefined constrain to obtain a final HR one. In other words, it can be taken as properly adding back details to a preliminary HR image to obtain a final HR one. In such a category iterative refining processing usually induces great computing complexity and sometimes fixed values of parameters do not sufficiently and adaptively refine details according to characteristics of various images.

In this thesis, linear regression, which is a useful tool in statistics and has been recently used in super resolution due to its ability of estimation, is properly used in conjunction with the self-similarity of a pair of LR and HR images to solve the aforementioned problems. Different from other SR algorithms where regression is used to estimate interpolated pixels, the proposed algorithms take the advantages of regression in conjunction with that of reconstruction-based SR algorithms to construct detailed and natural HR images. An HR image obtained by fast interpolation is usually more blurred and this can be taken as the fact that some details are lost in the enlarging process. How to estimate these lost details is the most concerned issue in the proposed algorithms. An efficient SR algorithm is proposed where simple linear regression models are established with details acquired from patches of LR images and then are used to estimate details of HR images due to the self-similarity of a pair of LR and HR images. The experimental results show that the proposed SR algorithm is not only effective but also efficient.

Another SR algorithm assuming that various details can be taken as a combination of several oriented ones is further proposed. Eight filters are designed to properly acquire corresponding oriented details. Multiple linear regression models are established with oriented details acquired from LR images by the designed filters and then used to estimate details of HR images with the corresponding oriented details acquired from corresponding preliminary HR images by the same filters. For more adaptively utilizing the characteristics of different regions in an image, a modified version that preliminarily segments the input LR image is also presented. From experimental results, it is clear that the proposed algorithms perform well in both objective and subjective measurements.

The greatest advantage of learning-based SR algorithm is the ability of constructing "natural" details, and vector quantization (VQ) is a popular technique used in this category. In this thesis, this advantage is also taken into consideration and an SR algorithm using weighted VQ is proposed. Different from other SR algorithms utilizing VQ to construct HR images, the concept of finding code vectors that are the most similar to the input patch by correlation coefficients instead of Euclidean distance is proposed since the added-back details are obtained by using multiple linear regression to combine the code vectors selected from codebooks rather than by using code vectors directly selected from codebooks. In this manner, the influence of the performance of VQ due to the limit of codebook sizes can be decreased. That is, the cost for saving codebooks can be decreased and meanwhile, the performance is still maintained and is even better. In the experimental results, it is clear that the goal of the proposed algorithm is achieved and the actual experimental results are also as expected.

An SR algorithm that can present more details and meanwhile avoid inducing obvious jaggy artifacts is also exhibited in this thesis. A fast fractal super resolution technique is adopted to obtain the preliminary HR image due to its ability of constructing clear details. Then, post-processing is proposed to decrease the obvious jaggy artifacts along slanted edges by directional blurring and enhancing using the pre-designed oriented filters and patterns. Actually, the post-processing can be used to effectively decrease the jaggy artifacts caused by SR algorithms. In the experimental results, it can be found that the details of HR images are exhibited clearly and the artifacts along strong slanted edges are greatly decreased.

[1] G. Arce, “A general weighted median _lter structure admitting negative weights,”
IEEE Transactions on Signal Processing, vol. 46, no. 12, pp. 3195 - 3205, Dec 1998.
[2] T. Aysal and K. Barner, “Quadratic weighted median filters for edge enhancement of noisy images,” IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3294 -3310, Nov. 2006.
[3] S. Baker and T. Kanade, “Hallucinating faces,” in International Conference on Automatic Face and Gesture Recognition, 2000, pp. 83 - 88.
[4] A. C. Bovik, Handbook of Image and Video Processing (Communications, Net-working and Multimedia). Orlando, FL, USA: Academic Press, Incorporated, 2005.
[5] A. Bruckstein, M. Elad, and R. Kimmel, “Down-scaling for better transform compression,” IEEE Transactions on Image Processing, vol. 12, no. 9, pp. 1132 - 1144, Sep. 2003.
[6] H. Chang, D.-Y. Yeung, and Y. Xiong, “Super-resolution through neighbor embedding,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, june - 2 july 2004, pp. I-275 – I-282 Vol.1.
[7] M. Crouse, R. Nowak, and R. Baraniuk, “Wavelet-based statistical signal processing using hidden markov models,” IEEE Transactions on Signal Processing, vol. 46, no. 4, pp. 886 - 902, apr 1998.
[8] S. Dai, M. Han, Y. Wu, and Y. Gong, “Bilateral back-projection for single image super resolution,” in IEEE International Conference on Multimedia and Expo, july 2007, pp. 1039 - 1042.
[9] W. Dong, D. Zhang, G. Shi, and X.Wu, “Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization,” IEEE Transactions on Image Processing, vol. 20, no. 7, pp. 1838 - 1857, 2011.
[10] W. Dong, L. Zhang, G. Shi, and X. Wu, “Nonlocal back-projection for adaptive image enlargement,” in IEEE International Conference on Image Processing, nov. 2009, pp. 349 - 352.
[11] C. E. Duchon, “Lanczos filtering in one and two dimensions,” J. Appl. Meteor., vol. 18, no. 8, pp. 1016 - 1022, Aug. 1979. [Online]. Available: http://dx.doi.org/10.1175/1520-0450(1979)018%3C1016:l_oat%3E2.0.co;2
[12] W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, “Learning low-level vision,” Int. J. Comput. Vision, vol. 40, no. 1, pp. 25 - 47, Oct. 2000.
[13] D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in IEEE International Conference on Computer Vision, 2009, pp. 349 - 356.
[14] R. C. Gonzalez and R. E. Woods, Digital Image Processing (3rd Edition). Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 2006.
[15] H. L. Haichao Zhang, Yanning Zhang and T. S. Huang, “Generative Bayesian image super resolution with natural image prior,” IEEE Transactions on Image Processing, vol. 21, no. 9, pp. 4054 - 4067, SEPTEMBER 2012.
[16] S.-H. Hong, R.-H. Park, S. Yang, and J.-Y. Kim, “Image interpolation using interpolative classified vector quantization,” Image and Vision Computing, vol. 26, no. 2, pp. 228 - 239, 2008.
[17] R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 29, no. 6, pp. 1153 - 1160, Dec 1981.
[18] H. Li, L. Xu, and G. Liu, “Face hallucination via similarity constraints,” Signal Processing Letters, IEEE, vol. 20, no. 1, pp. 19 - 22, 2013.
[19] K. M. Li and S. C. Tai, “A sharpness enhancment algorithm with adaptive acutance compensation,” Master's thesis, National Cheng Kung University, Tainan, Taiwan, R.O.C., 2009.
[20] X. Li, K. M. Lam, G. Qiu, L. Shen, and S. Wang, “Example-based image super-resolution with class-specific predictors,” J. Vis. Comun. Image Represent., vol. 20, no. 5, pp. 312 - 322, Jul. 2009.
[21] X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Transactions on Image Processing, vol. 10, pp. 1521 - 1527, 2001.
[22] W. Lin and L. Dong, “Adaptive downsampling to improve image compression at low bit rates,” IEEE Transactions on Image Processing, vol. 15, no. 9, pp. 2513 - 2521, Sept. 2006.
[23] Y. Linde, A. Buzo, and R. Gray, “An algorithm for vector quantizer design,” IEEE Transactions on Communications, vol. 28, no. 1, pp. 84 - 95, jan 1980.
[24] X. Liu, D. Zhao, R. Xiong, S. Ma, W. Gao, and H. Sun, “Image interpolation via regularized local linear regression,” IEEE Transactions on Image Processing, vol. 20, no. 12, pp. 3455 - 3469, dec. 2011.
[25] S. Mallat and G. Yu, “Super-resolution with sparse mixing estimators,” IEEE Transactions on Image Processing, vol. 19, no. 11, pp. 2889 - 2900, nov. 2010.
[26] B. Ramamurthi and A. Gersho, “Edge-oriented spatial filtering of images with application to post-processing of vector quantized images,” in IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 9, Mar 1984, pp. 573
- 576.
[27] P. Rieder and G. Sche_er, “New concepts on denoising and sharpening of video signals,” IEEE Transactions on Consumer Electronics, vol. 47, no. 3, pp. 666 - 671, Aug 2001.
[28] S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science, vol. 290, no. 5500, pp. 2323 - 2326, 2000.
[29] J. A. P. Tegenbosch, P. M. Hofman, and M. K. Bosma, “Improving nonlinear upscaling by adapting to the local edge orientation,” Visual Communications and Image Processing 2004, vol. 5308, no. 1, pp. 1181 - 1190, 2004.
[30] A. Temizel, “Image resolution enhancement using wavelet domain hidden markov tree and coefficient sign estimation,” in IEEE International Conference on Image Processing, vol. 5, 16 2007-oct. 19 2007, pp. V -381 - V -384.
[31] A. Temizel and T. Vlachos, “Wavelet domain image resolution enhancement,” Vision, Image and Signal Processing, IEE Proceedings -, vol. 153, no. 1, pp. 25 -
30, feb. 2006.
[32] A. Temizel and T. Vlachos, “Image resolution upscaling in the wavelet domain using directional cycle spinning,” J. Electronic Imaging, 2005.
[33] S. Thurnhofer and S. Mitra, “A general framework for quadratic volterra filters
for edge enhancement,” IEEE Transactions on Image Processing, vol. 5, no. 6, pp. 950 - 963, Jun 1996.
[34] J. Tian, L. Ma, and W. Yu, “Ant colony optimization for wavelet-based image interpolation using a three-component exponential mixture model,” Expert Systems with Applications, vol. 38, no. 10, pp. 12 514 - 12 520, 2011.
[35] R. Y. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Advances in Computer Vision and Image Processing, pp. 317 - 339, 1984.
[36] J. Wang and Y. Gong, “Fast image super-resolution using connected component enhancement,” in IEEE International Conference on Multimedia and Expo., 23 2008-april 26 2008, pp. 157 - 160.
[37] Z. Wang and A. Bovik, “Mean squared error: Love it or leave it? a new look at signal fidelity measures,” Signal Processing Magazine, IEEE, vol. 26, no. 1, pp. 98 - 117, jan. 2009.
[38] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600 - 612, April 2004.
[39] Y. C. Wee and H. J. Shin, “A novel fast fractal super resolution technique,” IEEE Transactions on Consumer Electronics, vol. 56, no. 3, pp. 1537 - 1541, aug. 2010.
[40] D. H. Woo, I. K. Eom, and Y. S. Kim, “Image interpolation based on inter-scale dependency in wavelet domain,” in International Conference on Image Processing, vol. 3, oct. 2004, pp. 1687 - 1690 Vol. 3.
[41] X. Wu, X. Zhang, and X. Wang, “Low bit-rate image compression via adaptive down-sampling and constrained least squares upconversion,” IEEE Transactions on Image Processing, vol. 18, no. 3, pp. 552 - 561, March 2009.
[42] J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Transactions on Image Processing, vol. 19, no. 11, pp. 2861 - 2873, 2010.
[43] L. Yin, R. Yang, M. Gabbouj, and Y. Neuvo, “Weighted median filters: a tutorial,” IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 43, no. 3, pp. 157 - 192, Mar 1996.
[44] K. Zhang, X. Gao, D. Tao, and X. Li, “Single image super-resolution with non-local means and steering kernel regression," IEEE Transactions on Image Processing, vol. 21, no. 11, pp. 4544 - 4556, nov. 2012.
[45] X. Zhang and X. Wu, “Image interpolation by adaptive 2-d autoregressive modeling and soft-decision estimation,” IEEE Transactions on Image Processing, vol. 17, no. 6, pp. 887 - 896, june 2008.
[46] Y. Zhang, D. Zhao, J. Zhang, R. Xiong, and W. Gao, “Interpolation-dependent image downsampling,” IEEE Transactions on Image Processing, vol. 20, no. 11, pp. 3291 - 3296, 2011.
[47] D. Zhou and S. Xiaoliu, “An e_ective color image interpolation algorithm,” in International Congress on Image and Signal Processing, vol. 2, oct. 2011, pp. 984 - 988.