影像縮放是電腦視覺領域中的一個重要的研究議題，因為在影像解析度改變的過程中常伴隨著影像模糊與品質降低的挑戰。近年來，隨著高解析度的設備普及，因此針對高解析度的影像放大方法逐漸受到重視，這類方法我們也稱為超解析度成像。
為了克服上述挑戰，有越來越多的研究投入以卷積神經網路為基底的超解析度成像，原因是此架構不僅能夠完善地提升影像解析度，也能將影像細節處做到更清晰地呈現。然而考量到執行成本，部分資源有限的硬體裝置將會無法負荷其執行所伴隨的運算成本，因此如何以低成本來達到高品質的放大影像，是我們的研究重點。
本研究提出之方法以適合硬體實作去考量，我們提出的一維插補架構大幅地減少了運算成本，並透過機器學習的方式與一維影像紋理分類器來有效地提升影像品質。本研究的主要的貢獻為在使用相對少的運算與儲存空間成本的條件下可達到具有競爭力的影像放大品質。同時，我們更驗證能夠將此方法應用於不同倍率的超解析度成像任務，與本研究中，我們將展示超解析度成像倍率為2、3與4倍的成像結果。
在實驗中，相比於相關研究OLM-SI的表現，我們的方法可減少了大約七成的整體運算成本，針對影像放大品質的訊噪比(PSNR)比較中，我們的結果也只略遜色於OLM-SI，在與使用卷積神經網路的相關研究，如FSRCNN相比，本研究所提出之方法在更是能大幅地減少運算成本，並且能減少約8倍的執行時間。

關鍵字：影像插補、超解析度成像、機器學習、低成本、多倍率

The process of up-scaling an image causes blurring of the image and its quality degrades. Therefore, image scaling has been an important research topic, and the up-scaling methods using Convolutional Neural Network (CNN), known as Super-Resolution (SR), have become popular recently.
To solve this problem, Super-Resolution (SR) becomes an important research topic. SR can achieve better image upscaling and thus provide clarity in the details of the image. Notably implementing the CNN model into resource-limited hardware devices has become a great challenge. Therefore, we present a hardware-friendly low-cost, and learning-based interpolation for High Resolution (HR) image reconstruction. This novel up-scaling method is based on a learning-based scheme and only has one-time computation. Following the concept, we added vertical and horizontal interpolation architecture and proposed a defined 1D feature classifier of image texture. Our main contribution could reduce the usage of weights and computation efficiently. Moreover, we also extend our work to prove that our method can be applied for multi-magnification factors. So, we show and compare our results with the magnification factor of 2X, 3X, 4X.
Experimentally, we achieved almost similar PSNR Compared to the related study OLM-SI (One linear learning mapping-SI), our method supports an overall 70% cost reduction with little loss of performance. This proposed method surpasses all CNN and interpolation methods in the execution time.

Keywords: Image Interpolation, Super-resolution, Machine Learning, Low-cost, Multi-magnification.

[1] J. A. Parker, R. V. Kenxyon and D. E. Troxel, “Comparison of Interpolating Methods for Image Resampling,” in IEEE Transactions on Medical Imaging, vol. 2, no. 1, pp. 31-39, March 1983.

[2] Hsieh Hou and H. Andrews, “Cubic splines for image interpolation and digital filtering,” in IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 26, no. 6, pp. 508-517, December 1978.

[3] Chung-Chi Lin, Ming-Hwa Sheu, Huann-Keng Chiang, Chishyan Liaw, Zeng-Chuan Wu, Wen-Kai Tsai, “An Efficient Architecture of Extended Linear Interpolation for Image Processing”, Journal of Information Science and Engineering, pp. 631-648, 2010.

[4] C. Huang, P. Chen and C. Ma, “A Novel Interpolation Chip for Real-Time Multimedia Applications,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 10, pp. 1512-1525, Oct. 2012.

[5] C. Dong, C. C. Loy, K. He and X. Tang, “Image Super-Resolution Using Deep Convolutional Networks,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 1 Feb. 2016.

[6] C. C. Dong, C. C. Loy, and X. Tang, “Accelerating the super-resolution convolutional neural network,” in Proc. ECCV, 2016, pp. 391–407.

[7] J. Kim, J. Kwon Lee, and K. Mu Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proc. IEEE Conf. Comput.Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 1646–1654.

[8] Y. Kim et al, “A Real-Time Convolutional Neural Network for Super-Resolution on FPGA With Applications to 4K UHD 60 fps Video Services,” in IEEE Trans. Circuits Syst. for Video Technology, vol. 29, no. 8, pp. 2521-2534, Aug. 2019

[9] J.-S. Choi and M. Kim, “Super-interpolation with edge-orientation-based mapping kernels for low complex 2× upscaling,” IEEE Trans. Image Process., vol. 25, no. 1, pp. 469–483, Jan. 2016.

[10] J.-S. Choi and M. Kim, “Single image super-resolution using global regression based on multiple local linear mappings,” IEEE Trans. Image Process., vol. 26, no. 3, pp. 1300–1314, Mar. 2017.

[11] O. Russakovsky, J. Deng, H. Su et al., “Imagenet large scale visual recognition challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015.

[12] E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: dataset and study,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1122–1131, IEEE, Honolulu, HI, USA, July 2017.

[13] B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee. Enhanced deep residual networks for single image super-resolution. In CVPRW, 2017.

[14] Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, Yun Fu. Residual Dense Network for Image Super-Resolution, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15] P. Chen, et al. “Hardware Implementation of an Image Interpolation Method with Controllable Sharpness.” J. Inf. Sci. Eng. 34 (2018): 51-64.

[16] Y. Kim et al, “2X Super-Resolution Hardware Using Edge-Orientation-Based Linear Mapping for Real-Time 4K UHD 60 fps Video Applications,” in IEEE Transactions on Circuits Syst. II Exp. Briefs, vol. 65, no. 9, pp. 1274-1278, Sept. 2018

[17] Zhu, Xining, Lin Zhang, Lijun Zhang, Xiao Liu, Ying Shen, and Shengjie Zhao. 2020. 'GAN-Based Image Super-Resolution with a Novel Quality Loss', Mathematical Problems in Engineering, 2020: 5217429.