此篇論文介紹了一基於深度學習的圖像盲超解析度演算法，旨在提升對不同模糊核產生低解析度圖像的恢復品質。相較於傳統非盲超解析度演算法，我們認為盲超解析度演算法須能適應更多不確定性。為了考量更多不確定性，我們突破現有盲超解析度方法僅使用定值進行計算的限制，提出了以分布形式進行運算的架構，並基於平均場近似理論實現，我們提出的架構命名為均場多階網路架構。我們通過預測模糊核和超解析度圖像中每個元素的分布，並引入處理變異數相關資訊的模塊，將不確定性學習融入估計過程。在損失函數方面，我們採用平均絕對誤差以提高模型的強韌性，並引入Kullback-Leibler正則化項以監督變異數。實驗結果顯示，我們的方法在合成和實際數據集上均實現了卓越的性能，並且模型參數量相對較少。

This paper introduces a deep learning-based blind super-resolution algorithm aiming to enhance the restoration quality of low-resolution images generated by different blur kernels. In contrast to traditional non-blind super-resolution methods, we emphasize the need for blind super-resolution algorithms to adapt to increased uncertainty. To address limitations in existing blind super-resolution approaches, which often operate with deterministic values, we propose the Mean Field Multi-stage Network (MFMN) based on mean field approximation. MFMN predicts distributions for each element in the blur kernel and super-resolution image, incorporating a module to handle variance-related information and integrate uncertainty learning into the estimation process. The loss function employs Mean Absolute Error (MAE) to improve model robustness, and a Kullback-Leibler regularization term is introduced to supervise variance. Experimental results demonstrate outstanding performance on both synthetic and real-world datasets with a relatively lower number of model parameters.

[1] S. Bell-Kligler, A. Shocher, and M. Irani, "Blind Super-Resolution Kernel Estimation using an Internal-GAN," Advances in Neural Information Processing Systems 32, 2019.
[2] A. Shocher, N. Cohen, and M. Irani, "“Zero-Shot” Super-Resolution using Deep Internal Learning," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[3] Y. Huang, S. Li, L.Wang, T. Tan, et al., "Unfolding the Alternating Optimization for Blind Super Resolution," Advances in Neural Information Processing Systems, vol. 33, pp.5632–5643, 2020.
[4] F. Rosenblatt, "The perceptron: A probabilistic model for information storage and organization in the brain," Psychological review, vol. 65, no. 6, p. 386, 1958.
[5] D. E. Rumelhart, G. E. Hinton, R.J. Williams, "Learning representations by back-propagation errors," nature, vol. 323, no. 6088, pp. 533-536, 1986.
[6] W. Luo, et al., "Understanding the Effective Receptive Field in Deep Convolutional Neural Networks," Advances in Neural Information Processing Systems, vol. 29, 2016.
[7] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and Polosukhin, I, "Attention Is All You Need," Advances in Neural Information Processing Systems, vol. 30, 2017
[8] D. Han, "Comparison of Commonly Used Image Interpolation Methods." Conference of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013). Atlantis Press, 2013.
[9] C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a Deep Convolutional Network for Image Super-Resolution," in The European Conference on Computer Vision, pp.184-199, Springer, 2014.
[10] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
[11] B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136-144, 2017.
[12] Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image Super-Resolution using Very Deep Residual Channel Attention Networks.," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 286-301, 2018.
[13] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin and B. Guo, "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012-10022, 2021.
[14] J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool and Timofte, R, "SwinIR: Image Restoration using Swin Transformer," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1833-1844, 2021.
[15] J. Gu, H. Lu, W. Zuo and C. Dong, "Blind Super-Resolution with Iterative Kernel Correction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1604-1613, 2019.
[16] J. Fu, H. Wang, Q. Xie, Q. Zhao, D. Meng, and Z. Xu, "KXNet: A Model-Driven Deep Neural Network for Blind Super-Resolution," in European Conferenceon Computer Vision, pp.235–253,Springer, 2022.
[17] L. Wang, Y. Wang, X. Dong, Q. Xu, J. Yang, W. An and Y. Guo, "Unsupervised Degradation Representation Learning for Blind Super-Resolution," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10581-10590, 2021.
[18] S. Y. Lee, "Gibbs Sampler and Coordinate Ascent Variational Inference: A Set-Theoretical Review," Communications in Statistics-Theory and Methods, vol. 51, no.6, pp. 1549-1568, 2022.
[19] D. Nguyen, "An In Depth Introduction to Variational Bayes Note," Available at SSRN, 2023.
[20] Z. Fang, W. Dong, X. Li, J. Wu, L. Li and G. Shi, "Uncertainty Learning in Kernel Estimation for Multi-Stage Blind Image Super-Resolution," in European Conference on Computer Vision, pp. 144-16, October 2022.
[21] J. Chang, Z. Lan, C. Cheng and Y. Wei, "Data Uncertainty Learning in Face Recognition," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5710-5719, 2020.
[22] E.Agustsson and R.Timofte, "Ntire 2017 Challenge on Single Image Super Resolution: Dataset and Study," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.126–135, 2017.
[23] M. Bevilacqua, A. Roumy, C. Guillemot and M. L. Alberi-Morel, "Low Complexity Single-Image Super-Resolution Based on Nonnegative Neighbor Embedding," 2012.
[24] R. Zeyde, M. Elad, and M. Protter, "On Single Image Scale-Up using Sparse Representations," in Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7, pp.711730, Springer, 2012.
[25] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Cological Statistics," in Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–423, IEEE, 2001.
[26] J. -B. Huang, A. Singh, and N. Ahuja, "Single Image Super-Resolution from Transformed Self-Exemplars," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206, 2015.
[27] K. Zhang, J. Liang, L. VanGool, and R. Timofte, "Designing a Practical Degradation Model for Deep Blind Image Super-Resolution," In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 47914800, 2021.
[28] X. Wang, L. Xie, C. Dong, and Y. Shan, "Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1905-1914, 2021.
[29] D. P. Kingma and J. Ba, "Adam: A Model for Stochastic Optimization," arXiv preprint arXiv:1412.6980, 2014.
[30] X. Chen, X. Wang, J. Zhou, Y. Qiao, and C. Dong, "Activating More Pixels in Image Super-Resolution Transformer," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22367-22377, 2023.
[31] M. Schuster, and K.K. Paliwal, "Bidirectional recurrent neural networks," IEEE transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997.
[32] S. Hochreiter, and J. Schmidhuber, "Long Short-Term Memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[33] D. Britz, A. Goldie, M.T. Luong, and Q.V. Le, "Massive Exploration of Neural Machine Translation Architectures," CoRR, abs, 1703.03906, 2017.
[34] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual Dense Network for Image Super-Resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2472-2481, 2018.
[35] H. Chen, X. He, L. Qing, Y. Wu, C. Ren, R. E. Sheriff, and C. Zhu, " Real-World Single Image Super-Resolution: A Brief Review," Information Fusion, no. 79, pp. 124-145, 2022.
[36] Common datasets for image super-resolution – cv notes. https://cvnote.ddlee.cc/2019/09/22/image-super-resolution-datasets (accessed Feb. 05, 2024).