日常生活中充斥著各種RGB影像，和RGB影像不同的是衛星影像能夠拍攝出更多的頻譜資訊，然而卻也充斥著各種待解議題，其中多光譜衛星影像融合即是一個非常常見的議題，並且能夠應用在許多領域當中如變遷偵測、精準農業等等。不同的衛星有著不同的空間解析度以及時間解析度，時間解析度也可以稱為拍攝週期。當代的衛星通常無法以高頻率的方式拍攝出同個區域的高解析影像，因此想取得所有日期的高解析衛星影像變得格外困難。在這篇碩士論文中，我們將透過融合(A)低空間解析但高時間解析的影像、與(B)高空間解析但低時間解析的影像，進而得到(C)高空間解析且高時間解析的衛星影像，我們將此影像處理問題稱為高時空超解析。有別於深度學習，我們不需要蒐集大數據來訓練類神經網路，我們採用機器學習中的凸分析理論來處理高時空超解析問題。此篇論文的主要貢獻為高時空超解析問題的學習準則設計，旨在比較不同數據擬合項搭配不同正則化器於高時空超解析應用的效能，我們採用了高斯模糊矩陣及平均模糊矩陣兩種不同的模糊矩陣來設計數據擬合項，配合上總變量差或是自相似性之正則化器，來獲得四種學習準則。在實驗後我們發現平均模糊矩陣、搭配上自相似性之正則化器有著最好的高時空超解析表現。演算法也在實驗中成功估計出變化的區域及高解析之影像，且在和其他經典演算法比較後我們的演算法也在許多場景中有著較佳的表現。

RGB images are easily available everywhere in daily life. Unlike RGB images, remote sensing images capture more critical information about frequency variations. However, there are a number of unsolved issues, among which the multispectral satellite image fusion problem is of great interest with wide applicability, including change detection, precision agriculture, etc. Different satellites have different spatial resolutions and temporal resolutions; the temporal resolution is also known as (a.k.a.) the shooting cycle. It is hard to get the high-spatial-resolution (HSR) images at the exact location in a short cycle for the contemporary satellites. Therefore, it is not easy to obtain daily HSR images. In this thesis, we fuse (i) low-spatial-resolution, but high-temporal-resolution images and (ii) HSR low-temporal-resolution images in order to obtain (iii) desired HSR high-temporal-resolution images. This problem is known as spatiotemporal super-resolution (STSR). Unlike deep learning, we do not require collecting big data for training the neural networks; we utilize the convex analysis theory, which is very popular in machine learning to deal with the STSR problem. The contribution of this thesis mainly consists of designing the STSR learning criteria. This thesis aims to compare the efficacy of different data fitting terms combing with different regularizers. We adopt two different blurring matrices (i.e., Gaussian blurring matrix and uniform blurring matrix) for designing the data fitting terms and two different regularizers (i.e., total variation and self-similarity regularizers) to combinatorially obtain four STSR learning criteria. With extensive experimental studies, we observe that the STSR criterion combining the uniform blurring matrix and the self-similarity regularizer yields the best super-resolution performance. The corresponding algorithms successfully estimate the changed part of the high-resolution images, an outstanding property for the STSR technique, and outperform several benchmark STSR methods in a number of scenarios.

[1] W.-C. Zheng, C.-H. Lin, K.-H. Tseng, C.-Y. Huang, T.-H. Lin, C.-H. Wang, and C.-Y. Chi, Unsupervised change detection in multitemporal multispectral satellite images: A convex relaxation approach,” in Proc. IEEE IGARSS, Yokohama, Japan, Jul. 28- Aug. 2, 2019, pp. 1546–1549.

[2] B. Datt, T. McVicar, T. Van Niel, D. Jupp, and J. Pearlman, “Preprocessing EO-1
Hyperion hyperspectral data to support the application of agricultural indexes,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 6, pp. 1246–1259, Jun. 2003.

[3] F. Gao, J. Masek, M. Schwaller, and F. Hall, “On the blending of the Landsat and MODIS surface reflectance: Predicting daily Landsat surface reflectance,” IEEE Trans. Geosci. Remote Sens., vol. 44, no. 8, pp. 2207–2218, Aug. 2006.

[4] X. Zhu, J. Chen, F. Gao, X. Chen, and J. G. Masek, “An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions,” Remote Sens. Environ., vol. 114, no. 11, pp. 2610–2623, Feb. 2010.

[5] Q. Wang and P. M. Atkinson, “Spatio-temporal fusion for daily Sentinel-2
images,” Remote Sens. Environ., vol. 204, pp. 31–42, Jan. 2018.

[6] X. Li, G. M. Foody, D. S. Boyd, Y. Ge, Y. Zhang, Y. Du, and F. Ling, “SFSDAF: An enhanced FSDAF that incorporates sub-pixel class fraction change information for spatio-temporal image fusion,” Remote Sens. Environ., vol. 237, p. 111537, Feb 2020.

[7] C.-H. Lin and J. M. BioucasDias, “An explicit and scene-adapted definition of convex self-similarity prior with application to unsupervised Sentinel-2
super-resolution,” IEEE Trans. Geosci. Remote Sens., vol. 58, no. 5, pp. 3352–3365, May 2020.

[8] M. Simões, J. Bioucas‐Dias, L. B. Almeida, and J. Chanussot, “A convex formulation for hyperspectral image superresolution via subspace-based
regularization,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 6, pp. 3373–3388, Jun. 2015.

[9] R. Molina, M. Vega, J. Mateos, and A. K. Katsaggelos, “Variational posterior distribution approximation in Bayesian super-resolution reconstruction of multispectral images,” Applied and Computational Harmonic Analysis, vol. 24, no. 2, pp. 251–267, Mar. 2008.

[10] M.-D. Iordache, J. M. Bioucas-Dias, and A. Plaza, “Total variation spatial regularization for sparse hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 11, pp. 4484–4502, Nov 2012.

[11] T. Goldstein and S. Osher, “The split Bregman method for L1-regularized
problems,” SIAM journal on imaging sciences, vol. 2, no. 2, pp. 323–343, Apr. 2009.

[12]K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3­D transform-­domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080–2095, Aug. 2007.

[13] C.-Y. Chi, W.-C. Li, and C.-H. Lin, Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications. CRC Press, Boca Raton, FL, 2017.

[14] Q. Wei, N. Dobigeon, and J.-Y. Tourneret, “Fast fusion of multi-band
images based on solving a Sylvester equation,” IEEE Trans. Image Process., vol. 24, no. 11, pp. 4109–4121, Nov. 2015.

[15] L. Zhuang and J. M. BioucasDias, “Fast hyperspectral image denoising and inpainting based on low-rank and sparse representations,” IEEE J-STARS,
vol. 11, no. 3, pp. 730–742, Mar. 2018.

[16]L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images,” Photogramm. Eng. Remote Sens., vol. 63, no. 6, pp. 691–699,1997.

[17] C.-H. Lin, F. Ma, C.-Y. Chi, and C.-H. Hsieh, “A convex optimization-based
coupled nonnegative matrix factorization algorithm for hyperspectral and multispectral data fusion,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 3, pp. 1652–1667, Mar. 2018.

[18] A. Horé and D. Ziou, “Image quality metrics: PSNR vs. SSIM,” in 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey, Aug. 23-26
2010, pp. 2366–2369.

[19]M. Drusch, U. Del Bello, S. Carlier, O. Colin, V. Fernandez, F. Gascon, B. Hoersch, C. Isola, P. Laberinti,P. Martimort, A. Meygret, F. Spoto, O. Sy, F. Marchese, and P. Bargellini, “Sentinel­2: ESA’s optical high­resolution mission for GMES operational services,” Remote Sens. Environ., vol. 120, pp. 25–36, May 2012.