高光譜影像擁有著豐富的頻譜資訊，使其在遙測影像領域有著大量的應用，例如物質分類與辨識任務，然而受限於硬體上的限制，這類的影像通常有著不高的空間解析度。為了得到足以滿足需求的高解析之高光譜影像，一種具有高實用價值的技巧是使用影像融合，它將高空間解析度的多光譜影像中的空間細節合併到相同拍攝範圍對應的低空間解析之高光譜影像中。現存的許多經典影像融合演算法基於凸優化來設計，並且展現了出色的結果，但其複雜的正則化項通常導致繁重的數學優化過程。隨著高端電腦計算設備的發展，近年來深度學習以其更短的執行時間與強大的性能，逐漸成為一種替代方法，並且其避免了複雜的數學簡化過程。儘管如此，深度學習的表現非常依賴於訓練數據的品質和數量，收集足夠的高品質數據通常是昂貴和耗時的，特別是對於需要特定設備拍攝和人工預處理的高光譜影像而言。為了解決這兩種方法的限制，我們提出了一套結合了凸優化與深度學習優點的CDIF演算法。與一般的深度學習將網路輸出作為最終解不同，CDIF將深度學習的解作為引導凸優化演算法的角色，也因此只需要與其他方法相比大約20%訓練數據量的小數據驅動簡單網絡就足夠了。另外，透過基於Q-泛數的正則化器來有效的提取網路解中重要的統計信息，我們可以得到數學形式相當簡單的凸優化演算法。CDIF演算法在避免複雜數學的前提下，僅使用小數據實現了優秀的融合性能，並有著完整且嚴謹的數學推導過程。

Hyperspectral image (HSI), with abundant spectral information, is extensively utilized in remote sensing applications, especially in material classification and identification tasks. Nevertheless, the spatial resolution of HSI is often limited. Image fusion is an economical approach to obtain a desired high spatial resolution (HSR) HSI, which incorporates spatial information in the HSR multispectral image (MSI) into the counterpart low spatial resolution (LSR) HSI. To tackle this challenging problem, convex optimization has shown impressive results in numerous benchmark image fusion algorithms, but their complicated regularization schemes often lead to a math-heavy optimization procedure. Recently, the development of computing devices has made deep learning a popular alternative approach that provides faster implementation speed and satisfactory performance without requiring complicated mathematical design procedures. Nevertheless, the effectiveness of deep learning is heavily dependent on both quality and quantity of training data. Collecting such data is usually time-consuming and expensive, especially for HSI, which requires specific hardware and manual preprocessing after acquisition. To overcome such limitations, we propose a convex deep image fusion (CDIF) that blends their advantages. CDIF reconsiders the role of deep learning as a guide for the convex algorithm to search for the fusion solution, rather than directly serving as the final solution. Therefore, a small-data-driven simple network with approximately 20% of the training data compared to other peer methods is enough. Additionally, with the design of a Q-quadratic norm regularizer, important statistical information can be efficiently extracted from the network solution, yielding a simple convex algorithm. The proposed CDIF algorithm achieves state-of-the-art fusion performance and has all closed-form algorithmic expressions derived.

[1] H. Su, W. Yao, Z. Wu, P. Zheng, and Q. Du, “Kernel low-rank representation with elastic net for china coastal wetland land cover classification using gf-5 hyperspectral imagery,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 171, pp. 238–252, 2021.
[2] C.-I. Chang, Hyperspectral Data Exploitation: Theory and Applications. John Wiley & Sons, 2007.
[3] Y. Dong, T. Liang, Y. Zhang, and B. Du, “Spectral–spatial weighted kernel manifold embedded distribution alignment for remote sensing image classification,” IEEE Transactions on Cybernetics, vol. 51, no. 6, pp. 3185–3197, 2021.
[4] C.-H. Lin, F. Ma, C.-Y. Chi, and C.-H. Hsieh, “A convex optimization-based coupled nonnegative matrix factorization algorithm for hyperspectral and multispectral data fusion,” IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 3, pp. 1652–1667, Mar. 2018.
[5] N. Yokoya, C. Grohnfeldt, and J. Chanussot, “Hyperspectral and multispectral data fusion: A comparative review of the recent literature,” IEEE Geoscience and Remote Sensing Magazine, vol. 5, no. 2, pp. 29–56, Jun. 2017.
[6] C.-H. Lin and T.-H. Lin, “All-addition hyperspectral compressed sensing for metasurface-driven miniaturized satellite,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–15, 2021.
[7] C.-Y. Chi, W.-C. Li, and C.-H. Lin, Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications. CRC Press, Boca Raton, FL, 2017.
[8] C.-H. Lin, Y.-C. Lin, and P.-W. Tang, “ADMM-ADAM: A new inverse imaging framework blending the advantages of convex optimization and deep learning,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–16, 2022.
[9] N. Yokoya, T. Yairi, and A. Iwasaki, “Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 2, pp. 528–537, Feb. 2012.
[10] M. Simões, J. M. Bioucas-Dias, L. Almeida, and J. Chanussot, “A convex formulation for hyperspectral image superresolution via subspace-based regularization,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 6, pp. 3373–3388, 2015.
[11] Q. Wei, N. Dobigeon, J.-Y. Tourneret, J. M. Bioucas-Dias, and S. Godsill, “R-FUSE: Robust fast fusion of multiband images based on solving a Sylvester equation,” IEEE Signal Processing Letters, vol. 23, no. 11, pp. 1632–1636, July 2016.
[12] Q. Yuan, Y. Wei, X. Meng, H. Shen, and L. Zhang, “A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 3, pp. 978–989, Mar 2018.
[13] X. Zhang, W. Huang, Q. Wang, and X. Li, “SSR-NET: Spatial–spectral reconstruction network for hyperspectral and multispectral image fusion,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 7, pp. 5953–5965, 2021.
[14] J. Xiao, J. Li, Q. Yuan, and L. Zhang, “A dual-UNet with multistage details injection for hyperspectral image fusion,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–13, 2022.
[15] K. Li, W. Zhang, D. Yu, and X. Tian, “Hypernet: A deep network for hyperspectral, multispectral, and panchromatic image fusion,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 188, pp. 30–44, 2022.
[16] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “CBAM: Convolutional block attention module,” in Proceedings European Conference on Computer Vision (ECCV), Munich, Germany, Sep. 2018, pp. 3–19.
[17] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings IEEE conference on computer vision and pattern Recognition (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 770–778.
[18] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2017.
[19] C.-H. Lin and J. M. Bioucas-Dias, “An explicit and scene-adapted definition of convex self-similarity prior with application to unsupervised Sentinel-2 super-resolution,” IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 5, pp. 3352–3365, 2020.
[20] J. Chen, C. Richard, and P. Honeine, “Nonlinear estimation of material abundances in hyperspectral images with ℓ1-norm spatial regularization,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 5, pp. 2654–2665, Jun. 2014.
[21] M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging. CRC press, 2020.
[22] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 2004.
[23] Q. Wei, J. M. Bioucas-Dias, N. Dobigeon, and J. Y. Tourneret, “Hyperspectral and multispectral image fusion based on a sparse representation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 7, pp. 3658–3668, Jul. 2015.
[24] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE international conference on computer vision (CVPR), Boston, MA, USA, 2015, pp. 1–9.
[25] L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images,” Photogrammetric Engineering and Remote Sensing, vol. 63, no. 6, pp. 691–699, Jun. 1997.
[26] N. Yokoya and A. Iwasaki, “Airborne hyperspectral data over chikusei,” Space Application Laboratory, University of Tokyo, Japan, Tech. Rep. SAL-2016-05-27, May 2016.
[27] R. W. Basedow, D. C. Carmer, and M. E. Anderson, “HYDICE system: Implementation and performance,” Proceedings of the Society of Photo-Optical Instrumentation Engineers, vol. 2480, pp. 258–267, Jun. 1995.
[28] 2013 IEEE GRSS Data Fusion Contest. [Online]. Available: http://www.grss-ieee.org/community/technical-committees/data-fusion/
[29] D. P. Kingma and J. Ba, “ADAM: A method for stochastic optimization,” in Proc. International Conference for Learning Representations (ICLR), San Diego, USA, May 7-9, 2015.