成功大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	高齊鴻 Kao, Chi-Hung
論文名稱：	基於小數據之非監督高光譜影像超解析 Unsupervised Hyperspectral Image Super-Resolution Using Just Small Data
指導教授：	林家祥 Lin, Chia-Hsiang
學位類別：	碩士 Master
系所名稱：	電機資訊學院 - 電腦與通信工程研究所 Institute of Computer & Communication Engineering
論文出版年：	2021
畢業學年度：	109
語文別：	英文
論文頁數：	42
中文關鍵詞：	逆問題轉換、深度學習、高光譜影像、單一影像超解析、耦合非負矩陣分解
外文關鍵詞：	Inverse problem transform, deep learning, hyperspectral image, single-image super-resolution, coupled nonnegative matrix factorization (CNMF)
相關次數：	點閱：255 下載：3
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

我們可以藉由高光譜影像豐富的頻譜資訊來辨識拍攝區域的材料及物質，但由於許多硬體上的限制，高光譜影像的空間解析度通常不高。通常會仰賴拍攝範圍相同的高解析全色圖來提升高光譜影像之解析度。然而，在不加強硬體、也沒有其他高解析影像幫助的條件下，來達到高光譜影像超解析，是個艱鉅但具備高實用價值的技術。我們提出一套凸耦合非負矩陣分解單一影像超解析 (SICO-CNMF) 演算法，成功達成上述任務。和深度學習相比，最佳化演算法的優勢在於它不需要任何其他數據，即可完成演算法的計算，非常的有彈性；深度學習雖然需要蒐集數據，但扣除訓練過程所花的時間，其計算速度是最佳化演算法難以匹敵的。在此篇論文中，我們巧妙的將「單一影像超解析問題」轉換為另一個逆問題「影像融合問題」。我們使用低解析的高光譜影像生成低解析的多光譜影像，並使用本文提出的高通對偶重建抗式生成網路 (HP-DRGAN) 達到空間超解析，其中最佳化演算法讓深度學習必須為每個不同波段的場景分別訓練的限制獲得彈性，而深度學習加強了最佳化演算法的效能，兩者相輔相成，進而達到高效能之單一高光譜影像超解析技術。

With the abundant spectrum information of the hyperspectral image (HSI), one can effectively detect minerals and substances in the region of interest (ROI). However, the spatial resolution of HSI is often limited. It usually relies on a panchromatic image with the same ROI to improve the spatial resolution of HSI. Without relying on more advanced hardware or high-resolution counterpart image, achieving HSI super-resolution (SR) is challenging but highly valuable solution for practical applicability. We achieve so by proposing an algorithm, termed Single Image based Convex Optimization Algorithm via Coupled Nonnegative Matrix Factorization (SICO-CNMF). In comparison with deep learning, the optimization algorithm has the advantage that it does not need any other data for the subsequent algorithmic calculation. Even though deep learning requires data collection, its computational speed, after deducting the time spent on the training process, is outstanding comparing to those optimizationbased ones. In this thesis, we cleverly transform the “single image SR problem” into another inverse problem, i.e., “image fusion problem.” We generated low-resolution (LR) multispectral images (MSIs) using LR HSIs and achieved SR by high-pass dual reconstruction generative adversarial network (HPDRGAN). With optimization algorithm, deep learning becomes more flexible since each scene with different bands can be trained together. Deep learning is provably enhancing the optimization algorithm, and accordingly providing a highly effective single HSI SR technique.

Abstract in Chinese i
Abstract in English ii
Acknowledgements iii
Contents iv
List of Tables vii
List of Figures viii
Symbol ix
Introduction 1
1 Hyperspectral Images and Super-Resolution Problem 1
2 Peer Methods 2
2.1 LRTV 3
2.2 SMASSG  3
2.3 Deep Hyperspectral Prior  3
2.4 SSPSR 4
Related Background 5
1 General Framework of Inverse Problem Transform (IPT) 5
2 SR Problem Formulation and IPT-based Solution  6
3 Overview of the Proposed HPDRGAN  7
4 CO-CNMF 8
New SICO-CNMF Theory for Hyperspectral Super-resolution 10
1 Signal Model and Problem Formulation 11
2 High-Pass Dual Reconstruction Generative Adversarial Network (HP-DRGAN) 12
2.1 High-Pass Filter 15
2.2 Feature Enhancement 15
2.3 Spatial Resolution Enhancement 18
2.4 Spectral Resolution Enhancement 18
2.5 Loss of the Generative Network 19
2.6 Summary of HP-DRGAN 20
3 Affine Set Fitting 20
4 Single Image based Convex Optimization Algorithm via Coupled Nonnegative Matrix Factorization 21
Experimental Results and Analysis 24
1 Quantitative and Qualitative Results 24
2 The Hyperparameters of Training HP-DRGAN 26
3 Experimental Results 26
3.1 Results on Chikusei Dataset 27
3.2 Results on Pavia Dataset 29
3.3 Results on Harvard Dataset 31
3.4 Results on CAVE Dataset 33
4 Ablation Study 35
Conclusion 37
References 38
                                    

[1] C.-Y. Chi, W.-C. Li, and C.-H. Lin, Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications. CRC Press, Boca Raton, FL, 2017.
[2] L. J. Rickard, R. W. Basedow, E. F. Zalewski, P. R. Silverglate, and M. Landers, “HYDICE: An airborne system for hyperspectral imaging,” vol. 1937, FL, USA, Sep. 23 1993, pp. 173–179.
[3] H. Akbari, Y. Kosugi, K. Kojima, and N. Tanaka, “Detection and analysis of the intestinal ischemia using visible and invisible hyperspectral imaging,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 8, pp. 2011–2017, Aug. 2010.
[4] J. M. Bioucas-Dias, A. Plaza, G. CampsValls, P. Scheunders, N. Nasrabadi, and J. Chanussot, “Hyperspectral remote sensing data analysis and future challenges,” IEEE Geoscience and Remote Sensing Magazine, vol. 1, no. 2, pp. 6–36, Sep. 2013.
[5] E. Villeneuve and H. Carfantan, “Nonlinear deconvolution of hyperspectral data with mcmc for studying the kinematics of galaxies,” IEEE Transactions on Image Processing, vol. 23, no. 10, pp. 4322–4335, Oct. 2014.
[6] W. Xie, Y. Shi, Y. Li, X. Jia, and J. Lei, “High-quality spectral-spatial reconstruction using saliency detection and deep feature enhancement,” Pattern Recognition, vol. 88, pp. 139–152, Nov. 2018.
[7] Y. Yuan, D. Ma, and Q. Wang, “Hyperspectral anomaly detection by graph pixel selection,” IEEE Transactions on Cybernetics, vol. 46, no. 12, pp. 3123–3134, Dec. 2016.
[8] H. Wu and S. Prasad, “Semi-supervised deep learning using pseudo labels for hyperspectral image clas sification,” IEEE Transactions on Image Processing, vol. 27, no. 3, pp. 1259–1270, Mar. 2018.
[9] T. Liu, Y. Gu, J. Chanussot, and M. Dalla Mura, “Multimorphological superpixel model for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 12, pp. 6950– 6963, Dec. 2017.
[10] D. Ravi, H. Fabelo, G. M. Callic, and G.Z. Yang, “Manifold embedding and semantic segmentation for intraoperative guidance with hyperspectral brain imaging,” IEEE Transactions on Medical Imaging, vol. 36, no. 9, pp. 1845–1857, Sep. 2017.
[11] C.I. Chang, Hyperspectral Data Exploitation: Theory and Applications. John Wiley & Sons, 2007.
[12] F. Li, L. Xin, Y. Guo, D. Gao, X. Kong, and X. Jia, “Super-resolution for GaoFen-4 remote sensing images,” IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 1, pp. 28–32, Jan. 2018.
[13] L. Zhang, W. Wei, C. Bai, Y. Gao, and Y. Zhang, “Exploiting clustering manifold structure for hyperspectral imagery superresolution,” IEEE Transactions on Image Processing, vol. 27, no. 12, pp. 5969–5982, Dec. 2018.
[14] C.-H. Lin, F. Ma, C.-Y. Chi, and C.-H. Hsieh, “A convex optimizationbased coupled nonnegative matrix factorization algorithm for hyperspectral and multispectral data fusion,” IEEE Transactions Geoscience and Remote Sensing, vol. 56, no. 3, pp. 1652–1667, Mar. 2018.
[15] R. Dian, S. Li, A. Guo, and L. Fang, “Deep hyperspectral image sharpening,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 11, pp. 5345–5355, Nov. 2018.
[16] M. Simoes, J. Bioucas-Dias, L. B. Almeida, and J. Chanussot, “A convex formulation for hyperspectral image superresolution via subspacebased regularization,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 6, pp. 3373–3388, Jun. 2015.
[17] Y. Wang, X. Chen, Z. Han, S. He et al., “Hyperspectral image super-resolution via nonlocal lowrank tensor approximation and total variation regularization,” Remote Sensing, vol. 9, no. 12, p. 1286, Dec. 2017.
[18] J. Li, Q. Yuan, H. Shen, X. Meng, and L. Zhang, “Hyperspectral image super-resolution by spectral mixture analysis and spatial–spectral group sparsity,” IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 9, pp. 1250–1254, Sep. 2016.
[19] O. Sidorov and J. Yngve Hardeberg, “Deep hyperspectral prior: Single-image denoising, inpainting, superresolution,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Work shops, Seoul, Korea, Oct. 28 2019, pp. 0–0.
[20] J. Jiang, H. Sun, X. Liu, and J. Ma, “Learning spatial-spectral prior for superresolution of hyperspectral imagery,” IEEE Transactions on Computational Imaging, vol. 6, pp. 1082–1096, 2020.
[21] F. Shi, J. Cheng, L. Wang, P.T. Yap, and D. Shen, “LRTV: MR image superresolution with lowrank and total variation regularizations,” IEEE Transactions on Medical Imaging, vol. 34, no. 12, pp. 2459–2466, Dec. 2015.
[22] J. Liu, P. Musialski, P. Wonka, and J. Ye, “Tensor completion for estimating missing values in visual data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 208–220, Jan. 2013.
[23] D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Deep image prior,” in Proceedings of the IEEE conference on computer vision and pattern recognition, UT, USA, June. 1823 2018, pp. 9446–9454.
[24] C.-H. Lin and P.-W. Tang, “Inverse problem transform: Solving hyperspectral inpainting via deterministic compressed sensing,” in Proc. IEEE Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing, Amsterdam, Netherlands, Mar. 24-26 2021.
[25] C.-H. Lin, C.-Y. Chi, Y.-H. Wang, and T.-H. Chan, “A fast hyperplane-based minimum-volume enclosing simplex algorithm for blind hyperspectral unmixing,” IEEE Transactions on Signal Processing, vol. 64, no. 8, pp. 1946–1961, Apr. 2016.
[26] C.-H. Lin, R. Wu, W.-K. Ma, C.-Y. Chi, and Y. Wang, “Maximum volume inscribed ellipsoid: A new simplexstructured matrix factorization framework via facet enumeration and convex optimization,” SIAM Journal on Imaging Sciences, vol. 11, no. 2, pp. 1651–1679, Jun. 2018.
[27] M. Zhou, H. Chen, J. Paisley, L. Ren, L. Li, Z. Xing, D. Dunson, G. Sapiro, and L. Carin, “Nonparametric bayesian dictionary learning for analysis of noisy and incomplete images,” IEEE Transactions on Image Processing, vol. 21, no. 1, pp. 130–144, Jan, 2012.
[28] C.-H. Lin, J. M. B. Dias, T.-H. Lin, Y.-C. Lin, and C.-H. Kao, “A new hyperspectral compressed sensing method for efficient satellite communications,” in 2020 IEEE 11th Sensor Array and Multichannel Signal Processing Workshop, Hangzhou, China, Jun. 8-11 2020, pp. 1–5.
[29] N. Yokoya, T. Yairi, and A. Iwasaki, “Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 2, pp. 528–537, Feb. 2012.
[30] H. Li, P. Xiong, J. An, and L. Wang, “Pyramid attention network for semantic segmentation,” arXiv preprint arXiv:1805.10180, Sep. 2018.
[31] X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. Change Loy, “ESRGAN: Enhanced super resolution generative adversarial networks,” in Proceedings of the European Conference on Computer Vision Workshops, Munich, Germany, Sep. 8-14 2018.
[32] C.-C. Hsu and C.-H. Lin, “Dual reconstruction with densely connected residual network for single image superresolution,” in Proc. IEEE International Conference on Computer Vision, Seoul, Korea, Oct. 27 - Nov. 2, 2019, pp. 3643–3650.
[33] C.-C. Hsu, C.-H. Lin, C.-H. Kao, and Y.-C. Lin, “DCSN: Deep compressed sensing network for efficient hyperspectral data transmission of miniaturized satellite,” IEEE Transactions on Geoscience and Remote Sensing, 2020.
[34] M. Berman, H. Kiiveri, R. Lagerstrom, A. Ernst, R. Dunne, and J. F. Huntington, “ICE: A statistical approach to identifying endmembers in hyperspectral images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 42, no. 10, pp. 2085–2095, Oct. 2004.
[35] J. Chen, C. Richard, and P. Honeine, “Nonlinear estimation of material abundances in hyperspectral images with ℓ1-norm spatial regularization,” Transactions on Geoscience and Remote, vol. 52, no. 5, pp. 2654–2665, Jun. 2014.
[36] Q. Wei, J. BioucasDias, N. Dobigeon, and J.Y. Tourneret, “Hyperspectral and multispectral image fusion based on a sparse representation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 7, pp. 3658–3668, Jul. 2015.
[37] I. Goodfellow, J. PougetAbadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in Neural Information Processing Systems, vol. 27, pp. 2672– 2680, Dec. 813 2014.
[38] J. Yang, X. Fu, Y. Hu, Y. Huang, X. Ding, and J. Paisley, “PanNet: A deep network architecture for pan sharpening,” in Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, Oct. 22-29 2017, pp. 5449–5457.
[39] C. Tchoku, A. Karnieli, A. Meisels, and J. Chorowicz, “Detection of drainage channel networks on digital satellite images,” International Journal of Remote Sensing, vol. 17, no. 9, pp. 1659–1678, Aug. 1995.
[40] J.-F. Hu, T.-Z. Huang, L.-J. Deng, T.-X. Jiang, G. Vivone, and J. Chanussot, “Hyperspectral image super resolution via deep spatiospectral attention convolutional neural networks,” IEEE Transactions on Neural Networks and Learning Systems, 2021.
[41] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, NV, USA, Jun. 26 - Jul. 1, 2016, pp. 770–778.
[42] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional net works,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, HI, USA, Jul. 21-26 2017, pp. 4700–4708.
[43] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video superresolution using an efficient subpixel convolutional neural network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, NV, USA, Jun. 26 -Jul. 1, 2016, pp. 1874–1883.
[44] R. Hecht-Nielsen, “Theory of the backpropagation neural network,” in Neural Networks for Perception. Elsevier, 1992, pp. 65–93.
[45] T.-H. Chan, W.-K. Ma, C.-Y. Chi, and Y. Wang, “A convex analysis framework for blind separation of nonnegative sources,” IEEE Transactions on Signal Processing, vol. 56, no. 10, pp. 5120–5134, Oct. 2008.
[46] I. K. Fodor, “A survey of dimension reduction techniques,” Lawrence Livermore National Lab., CA, US, Tech. Rep., 2002.
[47] J. Gorski, F. Pfeuffer, and K. Klamroth, “Biconvex sets and optimization with biconvex functions: a survey and extensions,” Mathematical Methods of Operations Research, vol. 66, no. 3, pp. 373–407, Jun. 2007.
[48] F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum,” IEEE Transactions on Image Processing, vol. 19, no. 9, pp. 2241–2253, Sep. 2010.
[49] A. Chakrabarti and T. Zickler, “Statistics of realworld hyperspectral images,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. NW, USA: IEEE, Jun. 21-25 2011, pp. 193– 200.
[50] ROSIS Free Pavia University Data. [Online]. Available: http://www.ehu.eus/ccwintco/index.php?title= Hyperspectral_Remote_Sensing_Scenes.
[51] N. Yokoya and A. Iwasaki, “Airborne hyperspectral data over Chikusei,” Space Appl. Lab., Univ. Tokyo, Tokyo, Japan, Tech. Rep. SAL20160527.
[52] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, Apr. 2004.
[53] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017.
[54] W.-S. Lai, J.-B. Huang, N. Ahuja, and M.-H. Yang, “Deep laplacian pyramid networks for fast and accurate super-resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, HI, USA, Jul. 2126 2017, pp. 624–632.

2026-08-08公開

簡易檢索 / 詳目顯示

相關論文