超解析度影像重建（super-resolution image reconstruction）是一個常見用來增加解析度的方法，其原理為利用現有的一些低解析度影像去重建出一張高解析度影像。然而在傳統的超解析度影像重建中，解析度的增加是建立在低解析度影像均為照度一致(consistent illumination)的假設下。在不同時期獲得的衛星影像之間，由於陽光照射或大氣狀況的不同，常會有光譜反應(spectral response)不一致的情況。例如，農田、植被等地面在相同地區會隨著氣候變遷而有不同的季節變化，故由於照度不一致（illumination inconsistency）的問題，我們無法直接採用傳統的超解析重建方法於衛星影像上；而在另一方面，點光源擴散函數（point spread function）在超解析度影像重建模組中常被假設為已知的，然而實際上卻是無法事先知道的。
為了解決照度不一致和點光源擴散函數的問題，我們提出了新的方法來改善解析度。這個方法分為兩個階段，其一是改良式超解析度影像重建，其二是盲目反旋積技術；在第一階段中，我們發展出一個對照度非敏感(illumination-insensitive)的超解析重建方法，一方面提升解析度一方面改善照度不一致的問題，從頻率分析來說，照度不一致在空間域(spatial domain)中對低頻成分比對高頻成分有更大的影響，我們將多光譜(multispectral)的低解析度影像轉換成灰階(panchromatic)影像並萃取其強度（intensity）域的高頻成分，再把不同影像間的高頻成分整合並加入到ㄧ張升頻（upsample）的低解析度影像中以得到高解析度影像，有別於直接整合像素值（pixel value）的作法，我們的方法消除了不同影像間對比的高低差異，只保留了高頻的成分；在第二階段中，我們利用影像本身的邊緣（edge）特徵來估算點光源擴散函數，並將之代入現有的反旋積方法來復原影像。
我們對不同月份獲得的福衛二號影像進行實驗，由結果中可以顯示出我們提出的方法相對於傳統超解析度影像重建方法的優越性，縱使在影像有照度不一致問題的情況下，影像解析度仍然能夠獲得改善。

Super-resolution image reconstruction has been used for image resolution enhancement. By super resolution, a series of low resolution images can be reconstructed to be a single high resolution image. In the traditional super-resolution algorithm, the enhancement of spatial resolution is accomplished under the assumption of consistent illumination through the series of low resolution images. Remote sensing images obtained from different dates are usually inconsistent in their spectral responses due to solar illumination and atmospheric conditions. For example, farmlands and vegetation covers have distinct seasonal changes with the climate transition over the same area. With the problem of illumination inconsistency, we cannot apply the super-resolution model directly to the satellite images. Furthermore, the traditional super-resolution approach assumes that the point spread function (PSF) is known in advance. In practice, it is not always the case.
To solve the problems of illumination inconsistency and PSF indetermination, we propose a new approach to enhance the spatial resolution. This approach consists of two stages: modified super-resolution image reconstruction and blind image deconvolution. In the first stage, we developed an illumination-insensitive super-resolution method for enhancement of spatial resolution as well as alleviation of illumination inconsistency. From the aspect of frequency analysis, the inconsistent illumination has a more significant effect on the low frequency contents than on the high frequency contents in the spatial domain. We transformed the multispectral low resolution images into panchromatic images and extracted the high frequency contents from the intensity component. The high frequency contents in all panchromatic low resolution images were integrated and added to an upsampled multispectral low resolution image to form a high resolution image. Instead of a direct integration of pixel values, our method canceled the spectral contrast between the low resolution images and retains only the high frequency information for integration. In the second stage, the PSF were estimated by the edge method that exploits the edge feature of the image itself. The image was restored by a deblurring method such as Richardson-Lucy algorithm and Wiener filter using the estimated PSF.
The experiments on FORMOSAT-2 images in different seasons demonstrated the outperformance of the proposed method over the traditional super-resolution approach. The spatial resolution of multispectral images was improved even though the images were not consistent in illumination.

[1] T. S. Huang and R. Y. Tsai, "Multi-frame image restoration and registration," Adv. Comput. Vis. Image Process., vol. 1, pp. 317-339, 1984.
[2] N. K. Bose, H. C. Kim, and H. M. Valenzuela, "Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes," Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, vol. 5, pp. 269-272, 1993.
[3] R. Seunghyeon and K. Moon Gi, "Discrete cosine transform based regularized high-resolution image reconstruction algorithm," Optical Engineering, vol. 38, pp. 1348-1356, 1999.
[4] M. Irani and S. Peleg, "Improving resolution by image registration," CVGIP: Graphical Models and Image Processing, vol. 53, pp. 231-239, 1991.
[5] H. Min-Cheol, K. Moon Gi, and K. K. Aggelos, "Regularized multichannel restoration approach for globally optimal high-resolution video sequence," Proc. SPIE Visual Communications and Image Processing, vol. 3024, pp. 1306-1316, 1997.
[6] H. Min-Cheol, K. Moon Gi, and A. K. Katsaggelos, "An iterative weighted regularized algorithm for improving the resolution of video sequences," Proc. IEEE Int. Conf. Image Processing, vol. 2, pp. 474-477, 1997.
[7] C. H. Russell, J. B. Kenneth, G. B. John, E. A. Ernest, and A. W. Edward, "High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system," Optical Engineering, vol. 37, pp. 247-260, 1998.
[8] K. Moon Gi, "Generalized multichannel image deconvolution approach and its applications," Optical Engineering, vol. 37, pp. 2953-2964, 1998.
[9] N. K. Bose, S. Lertrattanapanich, and J. Koo, "Advances in superresolution using L-curve," Proc. IEEE Int. Symp. Circuits and Systems, vol. 2, pp. 433-436, 2001.
[10] Katsaggelos and K. Aggelos, "Digital image restoration," Springer Series in Information Science, 1991.
[11] H. Stark and P. Oskoui, "High resolution image recovery from image-plane arrays, using convex projections," Optical Society of America, Journal, A: Optics and Image Science, vol. 6, pp. 1715-1726, 1989.
[12] A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, "High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration," Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, vol. 3, pp. 169-172 vol.3, 1992.
[13] A. J. Patti, M. I. Sezan, and A. Murat Tekalp, "Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time," IEEE Trans. Image Processing, vol. 6, pp. 1064-1076, 1997.
[14] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed: Prentice Hall, 2002.
[15] R. J. Hanisch, R. L. White, and R. L. Gilliland, "Deconvolution of hubbles space telescope images and spectra" in Deconvolution of images and spectra, 2nd ed: Academic Press, Inc., 1996, pp. 310-360.
[16] A. S. Robert, W. B. Robert, and E. C. John, "Measurement of the HYDICE system MTF from flight imagery," Proc. SPIE Hyperspectral Remote Sensing and Applications, vol. 2821, pp. 127-136, 1996.
[17] R. A. Schowengerdt, C. Archwamety, and R. C. Wrigley, "Landsat thematic mapper image-derived MTF," Photogrammetric Engineering and Remote Sensing, vol. 51, pp. 1395-1406, 1985.
[18] L. Liao, P. Jarecke, D. Gleichauf, and T. Hedman, "Performance characterization of the hyperion imaging spectrometer instrument," Proc. SPIE, vol. 4101, pp. 22-26, 2000.
[19] N. R. Nelson and P. S. Barry, "Measurement of Hyperion MTF from on-orbit scenes," Proc. IEEE Int. Symp. Geoscience and Remote Sensing, vol. 7, pp. 2967-2969 vol.7, 2001.
[20] G. R. Ayers and J. C. Dainty, "Iterative blind image deconvolution method and its applications," Optics Letters, vol. 13, pp. 547-549, 1988.
[21] C. T.F. and W. C.K., "Convergence of the alternating minimization algorithm for blind deconvolution," Linear Algebra and its Applications, vol. 316, pp. 259-285(27), 2000.
[22] Y. Kim-Hui, G. Ling, and L. Wanquan, "A recursive soft-decision approach to blind image deconvolution," IEEE Trans. Signal Processing[see also Acoustics, Speech, and Signal Processing, IEEE Transactions on], vol. 51, pp. 515-526, 2003.
[23] Y. Yu-Li and M. Kaveh, "A regularization approach to joint blur identification and image restoration," IEEE Trans. Image Processing, vol. 5, pp. 416-428, 1996.
[24] T. F. Chan and W. Chiu-Kwong, "Total variation blind deconvolution," IEEE Trans. Image Processing, vol. 7, pp. 370-375, 1998.
[25] M. Elad and Y. Hel-Or, "A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur," IEEE Trans. Image Processing, vol. 10, pp. 1187-1193, 2001.
[26] N. Nhat, P. Milanfar, and G. Golub, "A computationally efficient superresolution image reconstruction algorithm," IEEE Trans. Image Processing, vol. 10, pp. 573-583, 2001.
[27] A. Zomet, A. Rav-Acha, and S. Peleg, "Robust super-resolution," Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. I-645-I-650, 2001.
[28] B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision (DARPA)," Proceedings of the 1981 DARPA Image Understanding Workshop, pp. 121-130, 1981.
[29] A. M. Tekalp, Digital Video Processing: Prentice Hall PTR, 1995.
[30] R. Leonid, O. Stanley, and F. Emad, "Nonlinear total variation based noise removal algorithms," Physica D: Nonlinear Phenomena, vol. 60, pp. 259-268, 1992.
[31] C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," Proc. IEEE Int. Conf. Computer Vision, pp. 839-846, 1998.
[32] S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Processing, vol. 13, pp. 1327-1344, 2004.
[33] D. Keren and M. Osadchy, "Restoring subsampled color images," Machine Vision and Applications, vol. 11, pp. 197-202, 1999.
[34] R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, "A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method," Proc. IEEE Int. Conf. Image Processing, vol. 1, pp. I-857-60, 2005.
[35] S. Farsiu, M. Elad, and P. Milanfar, "Multiframe demosaicing and super-resolution of color images," IEEE Trans. Image Processing, vol. 15, pp. 141-159, 2006.
[36] S. Lertrattanapanich and N. K. Bose, "High resolution image formation from low resolution frames using Delaunay triangulation," IEEE Trans. Image Processing, vol. 11, pp. 1427-1441, 2002.
[37] M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, "Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames," IEEE Trans. Instrumentation and Measurement, vol. 49, pp. 915-923, 2000.
[38] C. Myungjin, K. Rae Young, N. Myeong-Ryong, and K. Hong Oh, "Fusion of multispectral and panchromatic satellite images using the curvelet transform," IEEE Geoscience and Remote Sensing Letters, vol. 2, pp. 136-140, 2005.
[39] T. Te-Ming, P. S. Huang, H. Chung-Ling, and C. Chien-Ping, "A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery," IEEE Geoscience and Remote Sensing Letters, vol. 1, pp. 309-312, 2004.
[40] J. G. Liu, "Smoothing Filter-based Intensity Modulation: a spectral preserve image fusion technique for improving spatial details," International Journal of Remote Sensing, vol. 21, pp. 3461-3472, 2000.
[41] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, "Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis," IEEE Trans. Geoscience and Remote Sensing, vol. 40, pp. 2300-2312, 2002.
[42] R. A. Schowengerdt, Remote Sensing:Models and Methods for Image Processing, 2nd ed: Elsevier Science & Technology Books, 1997.
[43] Y. Kim-Hui and G. Ling, "A computational reinforced learning scheme to blind image deconvolution," IEEE Trans. Evolutionary Computation, vol. 6, pp. 2-15, 2002.
[44] R. L. Lagendijk, A. K. Katsaggelos, and J. Biemond, "Iterative identification and restoration of images," Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, pp. 992-995 vol.2, 1988.
[45] D. Kundur and D. Hatzinakos, "Blind image deconvolution revisited," IEEE Signal Processing Magazine, vol. 13, pp. 61-63, 1996.
[46] C. Li and Y. Kim-Hui, "Efficient discrete spatial techniques for blur support identification in blind image deconvolution," IEEE Trans. Signal Processing[see also Acoustics, Speech, and Signal Processing, IEEE Transactions on], vol. 54, pp. 1557-1562, 2006.
[47] K. Daniel. and G. A., "Denoising color images using regularization and correlation terms," Journal of Visual Communication and Image Representation, vol. 9, pp. 352-365, 1998.