超解析度成像是一種將低解析度影像提升解析度形成高解析度的技術。隨著深度學習技術的發展，超解析度模型可以由具有反卷積層的人工神經網絡建構而成，用以取代傳統式的內插法及重建法。超解析度使用生成對抗網路(Generative adversarial network for SR,SRGAN)為目前先進的超解析度模型之一，該模型中包含了深度殘差網路和生成對抗網路結構。使用近景影像所訓練的深度學習模型無法有效地應用於具有多光譜波段和豐富地貌的衛星影像，又由於GPU記憶體的限制，通常會將大尺寸的衛星圖像劃分為小尺寸進行深度學習模型訓練，可能會導致影像與影像之間的邊界不連續。為了解決這些問題，此實驗中提出了一種以SRGAN為基底的多光譜多地物模型深度學習模型。此模型包含多個多光譜SRGAN模型，每個模型都由特定地物進行訓練。在生成超解析度影像過程中，基於地物分類機率的加權影像融合可以同時包含多個多光譜模型的超解析度結果，同時採用移動窗口策略及逐像素加權影像融合解決邊界不連續問題並提高超解析度表現。此外，模型訓練中採用了兩個新的損失函數，即近紅外波段的感知損失和包含所有光譜波段的光譜角損失。在實驗過程中，使用空間解析度為10公尺的 Sentinel-2 level-2影像作為訓練資料集，並使用空間解析度為 30 米的 Landsat 8 level-2A 影像作為測試資料。比較測量峰值信噪比和結構相似性指數，使用新的損失函數和策略能提升超解析度模型在衛星影像上的表現。在光譜相似度的評估中，採用常態化差異植生植被指數圖和光譜角製圖來觀察超解析度成果是否得到優化。經由超解析度的Sentienl-2降解析度圖像的峰值信噪比高於30分貝，且結構相似性指數可以達到0.9左右，在常態化差異植生植被指數圖和光譜角製圖的結果也非常相似。但由於 Sentinel-2 和 Landsat-8 的影像本質上存在差異，超解析度的Landsat-8影像和Sentinel-2的參考影像兩者結果存在較大差距。

Image super-resolution (SR) is the task of deriving high-resolution images from their low-resolution versions. With the development of deep learning technologies, the SR algorithms are developed based on artificial neural networks (ANN) with deconvolution layers instead of deconvolution-based image restorations. SRGAN (Generative adversarial network for SR) is one of the state-of-the-art SR ANN models, which contains a deep residual network and a GAN learning structure. A deep learning model trained by close-range images cannot be well applied to satellite images containing multispectral bands with a rich variety of land-covers. Moreover, a large-size satellite image is generally partitioned into small-size patches for deep-learning model training because of limited GPU memories, which generally lead to discontinuous boundaries between image patches. To solve these problems, a multispectral and multi-landcover deep-learning model using SRGAN as the backbone is proposed. The proposed model contains multiple multi-spectral SRGAN models, and each model is trained by a dataset for a specific land-cover. During data inferencing, a weighted image blending, which is based on the probabilities of land-covers, is conducted to integrate the multiple multi-spectral SRGAN outputs. The boundary discontinuous problem is solved by adopting a moving window strategy, which achieves average pixel-wise prediction. Furthermore, two new loss functions, that is, perceptual loss for near-infrared band and spectral angle loss for all spectral bands, are employed in the model training. In experiments, Sentinel-2 level-2 images with 10 meters spatial resolution are utilized as the training dataset while Landsat 8 level-2A images with 30 meters spatial resolution are used as the testing dataset. Based on the measurements PSNR (Peak signal-to-noise ratio) and SSIM (Structural similarity index), the proposed method is better than related methods. In the evaluation of spectral similarity, NDVI (Normalized difference vegetation index) mapping and Spectral angle mapper are used to observed. The PSNR of the resulting super-resolution Sentienl-2 downscaling images are higher than 30dB and the SSIM of that are around 0.9, and the results of NDVI map and Spectral angle mapper are similarity. However, due to the differences of basic characteristic between Sentinel-2 and Landsat-8, there is the big gap between SR results of Landsat-8 and Sentinel-2 references.

Barsi, J. A., Alhammoud, B., Czapla-Myers, J., Gascon, F., Haque, M. O., Kaewmanee, M., Leigh, L., Markham, B. L. (2018). Sentinel-2A MSI and Landsat-8 OLI radiometric cross comparison over desert sites. European Journal of Remote Sensing, 51(1), 822-837.
C. E. Duchon (1979). Lanczos filtering in one and two dimensions, Journal of Applied Meteorology, vol. 18, no. 8, pp. 1016–1022.
Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
Dong, C., Loy, C. C., He, K. M., & Tang, X. O. (2016). Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 295-307.
Dong, C., Loy, C. C., & Tang, X. (2016, October). Accelerating the super-resolution convolutional neural network. European conference on computer vision, 391-407.
Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., Hoersch, B., Isola, C., Laberinto, P., Martimort, P., Meygret, A., Spoto, F., Sya, O., Marchese, F., Bargellini, P. (2012). Sentinel-2: ESA's optical high-resolution mission for GMES operational services. Remote sensing of environment, 120, 25-36.
EX, E. (2009). ENVI EX user's guide. Cited at: https://www.l3harrisgeospatial.com/portals/0/pdfs/enviex/envi_ex_user_guide.pdf
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.
He, K., Zhang, X., Ren, S., Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE international conference on computer vision.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.0486.
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. The proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
Huang, Y., Cheng, Y., Bapna, A., Firat, O., Chen, M. X., Chen, D., Chen, M, Lee, H., Ngiam, J., Le, Q. V, Wu, Y. (2018). GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism. arXiv:1811.06965
Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T., & Keutzer, K. (2014). Densenet: Implementing efficient convnet descriptor pyramids. arXiv:1404.1869.
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. International conference on machine learning.
Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. The European conference on computer vision.
Karra, K., Kontgis, C., Statman-Weil, Z., Mazzariello, J. C., Mathis, M., & Brumby, S. P. (2021). Global land use/land cover with Sentinel 2 and deep learning. 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS.
Kim, J., Lee, J. K., & Lee, K. M. (2016a). Accurate image super-resolution using very deep convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
Kim, J., Lee, J. K., & Lee, K. M. (2016b). Deeply-recursive convolutional network for image super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
Kruse, F. A., Lefkoff, A., Boardman, J., Heidebrecht, K., Shapiro, A., Barloon, P., & Goetz, A. (1993). The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer data. Remote sensing of environment, 44(2-3), 145-163.
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. The proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
Liebel, L., & Körner, M. (2016). Single-image super resolution for multispectral remote sensing data using convolutional neural networks. ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 41, 883-890.
Lim, B., Son, S., Kim, H., Nah, S., & Mu Lee, K. (2017). Enhanced deep residual networks for single image super-resolution. The proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPR).
Lu, T., Wang, J., Zhang, Y., Wang, Z., & Jiang, J. (2019). Satellite Image Super-Resolution via Multi-Scale Residual Deep Neural Network. Remote Sensing, 11(13), 1588.
Mao, X.-J., Shen, C., & Yang, Y. (2016). Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections. arXiv preprint arXiv:1606.08921.
R. Keys, (1981) Cubic convolution interpolation for digital image processing, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 6, pp. 1153–1160.
Ridwan, M., Radzi, N. A. M., Ahmad, W., Mustafa, I. S., Din, N. M., Jalil, Y. E., Isa, A.M., Othman, N.S., Zaki, W. (2018). Applications of landsat-8 data: A Survey.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Bernstein, M, et al. (2015). Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3), 211-252.
Shamal, S., & Weatherhead, K. (2014). Assessing spectral similarities between rainfed and irrigated croplands in a humid environment for irrigated land mapping. Outlook on AGRICULTURE, 43(2), 109-114.
Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
Tan, M., & Le, Q. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. International conference on machine learning.
USGS. (2016). Landsat 8 (L8) data users handbook. Retrieved June 22, 2017, https://landsat.usgs.gov/sites/default/files/documents/Landsat8DataUsers Handbook.pdf.
ESA. (2017). Sentinel-2 MSI User Guide. Cited at: https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi.