許多影像處理的現存演算法都是基於凸優化設計的，這些方法大多需要相對艱深的數學，因此近年來許多軟體工程師改而使用深度學習破解問題。然而深度學習通常需要大量的時間與人力來蒐集大數據，雖然一般要取得大量數位影像並不會太難，但在許多實際情況許多大數據是無法輕易獲得的，像是高光譜遙測影像這種仰賴較昂貴之攝影與儲存硬體之影像就不容易獲得，且蒐集後還須經過人力對數據進行前處理，因此小數據學習成了近幾年熱門的議題。此碩論透過結合凸優化裡的交替方向乘子法 (ADMM)、以及深度學習裡的適應性矩估計 (ADAM)，提出了全新的理論框架，解決了在多數現存演算法使用交替方向乘子法經常面臨到需要複雜數學設計的正則化器、也避免了在多數深度學習方法中需要蒐集大量的數據，成功結合凸優化與深度學習的關鍵乃成功使用了數學形式很簡單的凸函數Q-泛數來萃取小數據中的重要統計特性。在深度學習缺乏大數據且基於設計簡單的網路架構的情況下，深度學習輸出的答案往往不盡人意，但是透過Q-泛數的高效正則化器設計，我們能從小數據中萃取出有用的資訊來設計數學形式相當簡單的調節子。我們將提出之新穎理論框架應用於高光譜影像張量還原問題，其中利用小數據訓練出抗式生成網路，在實驗部分也證實了此框架之可行性。此全新框架結合了「凸優化」與「深度學習」的優勢，軟體設計師不再需要面對複雜數學、或花大量時間蒐集大數據，只需要數百組數據即可破解複雜的高光譜影像逆問題。

Many benchmark image processing algorithms are designed based on convex optimization. These methods often require math-heavy optimization procedure, and hence many software engineers eventually turn to use deep learning to solve the target problem to avoid the daunting math-heavy design procedures. However, using deep learning usually requires significant amount of time and human resources to collect big data. Unlike the RGB images readily available from many open online sources, big data are often lacking in real applications, for example, in hyperspectral remote sensing (HRS). HRS images are not easily obtainable because the acquisition relies on expensive hardware onboard the satellite, and data must be preprocessed manually after the acquisition. Thus, small data learning (SDL) gets more and more popular in HRS. This thesis proposes a novel theoretical framework by combining the alternating direction method of multipliers (ADMM) optimizer in convex optimization (CO) with the adaptive moment estimation (ADAM) optimizer in deep learning (DL). The CO-DL framework solves inverse problems without requiring complicated mathematical regularizer design (disadvantage in CO), and, in the meanwhile, avoids the need of collecting big data (disadvantage in DL). The key for successfully combining CO with DL lies within the Q-quadratic norm, a mathematically simple convex function to extract important statistical properties from small data. In the absence of big data, a simple DL network architecture usually outputs unpleasing results. However, we can still extract useful information from small data to design a regularizer with quite simple mathematical form. We apply the proposed CO-DL framework to the hyperspectral image tensor completion problem. We also demonstrate the practical applicability of this framework by using small data to train a generative adversarial network. Under our theoretical framework, software designers no longer need to face complicated math or big data collection; a hundred-scaled dataset is sufficient to solve the highly challenging hyperspectral image inpainting problem, as will be demonstrated in this thesis.

[1] N. Yokoya, C. Grohnfeldt, and J. Chanussot,“Hyperspectral and multispectral data fusion: A comparative review of the recent literature,”IEEE Geoscience and Remote Sensing Magazine, vol.5, no.2, pp.29–56, Jun. 2017.
[2] N. Keshava and J. F. Mustard,“Spectral unmixing,”IEEE Signal Processing Magazine, vol. 19, no. 1, pp. 44–57, Jan. 2002.
[3] J. M. Bioucas-Dias, A. Plaza, G. Camps-Valls, P. Scheunders, N. Nasrabadi, and J. Chanussot,“Hyperspectral remote sensing data analysis and future challenges,”IEEE Geoscience and Remote Sensing Magazine, vol. 1, no. 2, pp. 6–36, Jun. 2013.
[4] D. Stein, S. Beaven, L. Hoff, E. Winter, A. Schaum, and A. Stocker,“Anomaly detection from hyperspectral imagery,”IEEE Signal Processing Magazine, vol. 19, no. 1, pp. 58–69, Jan. 2002.
[5] C.-I. Chang, C.-C. Wu, C. S. Lo, and M.-L. Chang,“Real-time simplex growing algorithms for hyperspectral endmember extraction,”IEEE Transactionson Geoscience and Remote Sensing, vol. 48, no. 4, pp. 1834–1850, Apr. 2010.
[6] C.-H. Lin, R. Wu, W.-K. Ma, C.-Y. Chi, and Y. Wang,“Maximum volume inscribed ellipsoid: A new simplex-structured matrix factorization framework via facet enumeration and convex optimization,”SIAM Journal on Imaging Sciences, vol. 11, no. 2, pp. 1651–1679, Jun. 2018.
[7] M. K. Pal and A. Porwal,“Destriping of Hyperion images using low-pass-filter and local-brightness-normalization,”in Proc. IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy, Jul. 26-31, 2015, pp. 3509–3512.
[8] W. He, H. Zhang, H. Shen, and L. Zhang,“Hyperspectral image denoising using local low-rank matrix recovery and global spatial–spectral total variation,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 3, pp. 713–729, Mar. 2018.
[9] D. Cerra, R. Müller, and P. Reinartz,“Unmixing-based denoising for destriping and inpainting of hyperspectral images,”in Proc. IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Quebec City, Canada, Jul. 13-18, 2014, pp. 4620–4623.
[10] J. D′Errico,“Inpaintingnanelementsin3d,”MATLAB Central File Exchange. MathWorks, Natick, MA, USA, 2008,[Online]. Available: https://www.mathworks.com/matlabcentral/fileexchange/21214inpaintingnanelementsin3d.
[11] L. Zhuang and J. M. Bioucas-Dias,“Fast hyperspectral image denoising and inpainting based on low-rank and sparse representations,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 3, pp. 730–742, Feb. 2018.
[12] R. Wong, Z. Zhang, Y. Wang, F. Chen, and D. Zeng,“HSIIPNet: Hyperspectral imagery inpainting by deep learning with adaptive spectral extraction,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 4369–4380, Jul. 2020.
[13] O. Sidorov and J. Y. Hardeberg,“Deep hyperspectral prior: Single image denoising, inpainting, super resolution,”in Proc. IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea, Oct. 27-28, 2019, pp. 3844–3851.
[14] X.-L. Zhao, W.-H. Xu, T.-X. Jiang, Y. Wang, and M. K. Ng,“Deep plug-and-play prior for low-rank tensor completion,”Neurocomputing, vol. 400, pp. 137–149, Aug. 2020.
[15] C.-Y. Chi, W.-C. Li, and C.-H. Lin, Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications. CRC Press, Boca Raton, FL, 2017.
[16] C.-H. Lin and J. M. Bioucas-Dias,“Nonnegative blind source separation for ill-conditioned mixtures via John ellipsoid,”IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, Jul. 2020.
[17] C.-H. Lin and J. M. Bioucas-Dias,“An explicit and scene-adapted definition of convex self-similarity prior with application to unsupervised Sentinel-2 super-resolution,”IEEE Transactionson Geoscience and Remote Sensing, vol. 58, no. 5, pp. 3352–3365, May 2020.
[18] C.-H. Lin, J. M. Bioucas-Dias, T.-H. Lin, Y.-C. Lin, and C.-H. Kao,“A new hyperspectral compressed sensing method for efficient satellite communications,”in Proc. 11th IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM), Hangzhou, China, Jun. 2020.
[19] T. Long, W. Jiao, and G. He,“Rpc estimation via ℓ1-norm-regularized least squares (L1LS),”IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 8, pp. 4554–4567, Mar. 2015.
[20] M.-D. Iordache, J. M. Bioucas-Dias, and A. Plaza,“Total variation spatial regularization for sparse hyperspectral unmixing,”IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 11, pp. 4484–4502, May 2012.
[21] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition,”in Proc. International Conference on Robotics and Automation (ICRA), Leuven, Belgium, May 16-21, 1998, pp.2278–2324.
[22] A. Krizhevsky, I. Sutskever, and G. E. Hinton,“Imagenet classification with deep convolutional neural networks,”in Proc. 25th International Conference on Neural Information Processing Systems (NIPS), Lake Tahoe, USA, Dec. 3-8, 2012, pp. 1097–1105.
[23] K. Simonyan and A. Zisserman,“Very deep convolutional networks for large-scale image recognition,”in Proc. 3rd International Conference on Learning Representations (ICLR), SanDiego, USA, May 7-9, 2015.
[24] C.-C. Hsu and C.-H. Lin,“Dual reconstruction with densely connected residual network for single image super-resolution,”in Proc. IEEE International Conference on Computer Vision (ICCV), Seoul, Korea, Oct. 27-Nov. 2, 2019, pp. 3643–3650.
[25] X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. Change Loy,“ESRGAN: Enhanced super-resolution generative adversarial networks,”in Proc. the European Conference on Computer Vision (ECCV), Munich, Germany, Sep. 2018, pp. 63–79.
[26] C.-C. Hsu, C.-H. Lin, C.-H. Kao, and Y.-C. Lin,“DCSN: Deep compressed sensing network for efficient hyperspectral data transmission of miniaturized satellite,”IEEE Transactions on Geoscience and Remote Sensing, pp. 1–17, Nov. 2020.
[27] D. P. Kingma and J. Ba,“ADAM: A method for stochastic optimization,”in Proc. International Conference for Learning Representations (ICLR), SanDiego, USA, May 7-9, 2015.
[28] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian,“Image denoising by sparse 3-D transform-domain collaborative filtering,”IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, Aug. 2007.
[29] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein,“Distributed optimization and statistical learning via the alternating direction method of multipliers,”Foundations and Trends in Machine Learning, vol. 3, no. 1, pp. 1–122, Jul. 2011.
[30] C.-H. Lin, F. Ma, C.-Y. Chi, and C.-H. Hsieh,“A convex optimization-based coupled nonnegative matrix factorization algorithm for hyperspectral and multispectral data fusion,”IEEE Transactions on Geoscience and RemoteSensing, vol. 56, no. 3, pp. 1652–1667, Mar. 2018.
[31] K. He, X. Zhang, S. Ren, and J. Sun,“Deep residual learning for image recognition,”in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Neveda, USA, Jun. 2016, pp. 770–778.
[32] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio,“Generative adversarial nets,”in Proc. Advances in Neural Information Processing Systems (NIPS), Montreal, Canada, Dec. 8-13, 2014, pp. 2672–2680.
[33] R. B. Myerson, Game Theory. Harvard University Press, 2013.
[34] D. Ulyanov, A. Vedaldi, and V. Lempitsky,“Deep image prior,”in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, Jun. 18-23, 2018, pp. 9446–9454.
[35] N. Qian,“On the momentum term in gradient descent learning algorithms,”Neural Networks: The Official Journal of the International Neural Network Society, vol. 12, no. 1, pp. 145–151, Jan. 1999.
[36]“Landsat sensors: pushbroom vs whiskbroom,”[Online]. Available: https://svs.gsfc.nasa.gov/12754.
[37] C.-H. Lin and Y. Liu,“Blind hyperspectral inpainting via John ellipsoid,”in Proc. IEEE Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Amsterdam, Netherlands, Mar. 24-26, 2021.
[38] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 2004.
[39] C.-H. Lin, C.-Y. Chi, Y.-H. Wang, and T.-H. Chan,“A fast hyperplane-based minimum-volume enclosing simplex algorithm for blind hyperspectral unmixing,” IEEE Transactions on Signal Processing, vol. 64, no. 8, pp. 1946–1961, Apr. 2016.
[40]“AvirisDataPortal,”[Online]. Available: https://aviris.jpl.nasa.gov/dataportal/.
[41] L. Wald, T. Ranchin, and M. Mangolini,“Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images,”Photogrammetric Engineering and Remote Sensing, vol. 63, no. 6, pp. 691–699, Jun. 1997.
[42]“Hyperion Bhilwara hyperspectral data cube,”[Online]. Available: https://earthexplorer.usgs.gov/.
[43] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli,“Image quality assessment: From error visibility to structural similarity,”IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, Apr. 2004.
[44] N. Puletti, N. Camarretta, and P. Corona,“Evaluating EO1-Hyperion capability for mapping conifer and broadleaved forests,”European Journal of Remote Sensing, vol. 49, no. 1, pp. 157–169, Feb. 2017.
[45] T. F. Chan, M. K. Ng, A. C. Yau, and A. M. Yip,“Superresolution image reconstruction using fast inpainting algorithms,”Applied and Computational Harmonic Analysis, vol. 23, no. 1, pp. 3–24, Mar. 2007.