在數位相機影像重建的過程中，彩色濾波陣列插補法或稱解馬賽克法是關鍵的技術之一，許多的消費型電子產品需要使用此技術來顯示影像和/或影片。而影像縮放演算法在影像處理中，亦是一個十分重要的技術，常使用在來源影像之解析度不同於目標顯示裝置上之解析度時。許多數位感光元件的應用場景如數位相機、智慧手機或是近年興起的自動駕駛、無人四旋翼，為了要將輸出影像顯示或是進行運算，頻繁地使用解馬賽克以及縮放演算法。因此，我們提出一個新的演算法同時處理解馬賽克以及影像放大，以減少重複的處理流程並且降低誤差累加。
根據我們的實驗，解馬賽克以及縮放有許多相似的條件判斷，亦有一些不同面向的議題。在解馬賽克的主題中，失去的通道值通常與周圍的通道值有很高的光譜相關性，所以需要在相對平滑的平面上進行插補還原以減少色彩差異誤差；而在縮放方面，為了要插補出失去的細節，則需要針對區域頻率高低給出相對應的插補核心，而更高頻的插補也更容易使解馬賽克的誤差區域被放大。我們以實驗驗證找出平衡上述兩種影像處理的插補流程與參數，得到兼顧品質與複雜度的演算法。
我們提出的演算法以最靠近的通道值來求出色差平面，利用此平面來抓取邊緣資訊和插補失去的通道，並以此結果計算光譜相關性修復插補誤差進而重建失去的通道。最後為了降低誤差累加效果，我們使用單層神經網路針對上述過程中不同的邊緣區域訓練出不同的縮放插補核心的區域最佳解，以獲得最好的效果品質並減少誤差。
為了減少數位影像裝置的成本及體積並因應高品質的需求，我們希望在本論文中提出有複雜度效益的演算法，以硬體實現的思維進行設計與考量，在提升成果品質的同時，也考慮運算複雜度以及資料存取量，以適合未來能夠硬體實現。

In the process of reconstructing digital camera images, the color filter array (CFA) interpolation or demosaicking is one of the most critical technology. And the image scaling algorithm is also a key part in the field of image processing. Plenty of electronic devices using these techniques to create or display digital images in the variety of devices in different resolution.
Considering demosaicking process, the lost color channels are generally having higher spectral correlation. To reduce color difference error, the interpolation process in a relative flat plane is needed. As for scaling process, the interpolation of lost details, in other way, need different interpolation kernel in different frequency aera. And the higher frequency interpolation may also enlarge the error cause in demosaicking process. In our proposed, the process flow and the parameters are verified in dedicated experiments to find the balance in performance and complexity.
The algorithm is proposed for a novel method to dealing demosaicking and scaling process simultaneously. Using the color difference plane calculated by neighbor channels to extract the edge information and construct the missing channels. Then calculate the spectral correlation of the result to flatten the error and reconstruct the color plane. Finally, it utilizes a novel machine learning based structure for the local optimum interpolation kernels in different edge area to reduce the combination error and improve performance.
We proposed a complexity efficient algorithm in the mind of hardware design conditions. It provides a higher performance result and in consider of the complexity and data access aera for further hardware implementation.

[1] PARKER, J. Anthony; KENYON, Robert V.; TROXEL, Donald E. Comparison of interpolating methods for image resampling. IEEE Transactions on medical imaging, 1983, 2.1: 31-39.
[2] KIM, Chun-Ho, et al. Winscale: An image-scaling algorithm using an area pixel model. IEEE Transactions on circuits and systems for video technology, 2003, 13.6: 549-553.
[3] KEYS, Robert. Cubic convolution interpolation for digital image processing. IEEE transactions on acoustics, speech, and signal processing, 1981, 29.6: 1153-1160.
[4] CHOI, Jae-Seok; KIM, Munchurl. Super-Interpolation With Edge-Orientation-Based Mapping Kernels for Low Complex 2x times Upscaling. IEEE Transactions on Image Processing, 2015, 25.1: 469-483.
[5] DONG, Chao, et al. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 2015, 38.2: 295-307.
[6] KIM, Yongwoo; CHOI, Jae-Seok; KIM, Munchurl. 2X super-resolution hardware using edge-orientation-based linear mapping for real-time 4K UHD 60 fps video applications. IEEE Transactions on Circuits and Systems II: Express Briefs, 2018, 65.9: 1274-1278.
[7] CHEN, Pei-Yin; LIEN, Chih-Yuan; LU, Chi-Pin. VLSI implementation of an edge-oriented image scaling processor. IEEE Transactions on very large scale integration (VLSI) systems, 2009, 17.9: 1275-1284.
[8] HUANG, Chien-Chuan; CHEN, Pei-Yin; MA, Ching-Hsuan. A novel interpolation chip for real-time multimedia applications. IEEE Transactions on circuits and systems for video technology, 2012, 22.10: 1512-1525.
[9] LIEN, Chih-Yuan; YANG, Fu-Jhong; CHEN, Pei-Yin. An efficient edge-based technique for color filter array demosaicking. IEEE Sensors Journal, 2017, 17.13: 4067-4074.
[10] LIEN, Chih-Yuan, et al. Efficient VLSI architecture for edge-oriented demosaicking. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 28.8: 2038-2047.
[11] PEI, Soo-Chang; TAM, Io-Kuong. Effective color interpolation in CCD color filter arrays using signal correlation. IEEE Transactions on Circuits and Systems for video technology, 2003, 13.6: 503-513.
[12] CHUNG, Kuo-Liang, et al. Novel quality-effective zooming algorithm for color filter array. Journal of Electronic Imaging, 2010, 19.1: 013005.
[13] LUKAC, Rastislav; MARTIN, Karl; PLATANOITIS, K. N. Digital camera zooming based on unified CFA image processing steps. IEEE Transactions on Consumer Electronics, 2004, 50.1: 15-24.
[14] CHUNG, King-Hong; CHAN, Yuk-Hee. A low-complexity joint color demosaicking and zooming algorithm for digital camera. IEEE Transactions on Image Processing, 2007, 16.7: 1705-1715.
[15] ZHANG, Lei; ZHANG, David. A joint demosaicking–zooming scheme for single chip digital color cameras. Computer Vision and Image Understanding, 2007, 107.1-2: 14-25.
[16] CHUNG, Kuo-Liang, et al. New joint demosaicking and zooming algorithm for color filter array. IEEE Transactions on Consumer electronics, 2009, 55.3: 1477-1486.
[17] CHUNGA, Kuo-Liang, et al. Joint Demosaicking and Arbitrary-Ratio Resizing Algorithm for Bayer Color Filter Array Based on DCT Approach.
[18] HUANG, Wen-Tsung; CHEN, Wen-Jan; TAI, Shen-Chuan. A sharp edge-preserving joint color demosaicking and zooming algorithm using integrated gradients and an iterative back-projection technique. Digital Signal Processing, 2014, 27: 79-94.
[19] ZHOU, Dengwen; DONG, Weiming; CHEN, Wengang. Joint demosaicking and zooming using moderate spectral correlation and consistent edge map. Journal of Electronic Imaging, 2014, 23.4: 043010.
[20] ADAMS JR, James E. Design of practical color filter array interpolation algorithms for digital cameras. In: Real-Time Imaging II. International Society for Optics and Photonics, 1997. p. 117-125.
[21] L. Zhang, “18 McMaster Color images,” http://www.comp.polyu.edu.hk/~cslzhang/CDM_Dataset.htm
[22] “KodakPhotoCD,” http://r0k.us/graphics/kodak/
[23] ZHANG, Junkang; AN, Cheolhong; NGUYEN, Truong. DEEP JOINT DEMOSAICKING AND SUPER RESOLUTION ON HIGH RESOLUTION BAYER SENSOR DATA. In: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, 2018. p. 619-623.
[24] ZHOU, Ruofan; ACHANTA, Radhakrishna; SÜSSTRUNK, Sabine. Deep residual network for joint demosaicking and super-resolution. In: Color and Imaging Conference. Society for Imaging Science and Technology, 2018. p. 75-80.