大部分的數位相機均是採用單一影像感測器(single-sensor)的技術來拍攝景像。這類型的相機藉著覆蓋貝爾色彩濾波陣列(Bayer color filter array)於單一影像感測器上同時截取紅、綠及藍色彩資訊來降低製造成本。其中，在每一個像素位置只記錄一種色彩強度值。而全彩影像則是將擷取到的貝爾碼賽克影像經由一個針對遺失色彩成份的重建處理程序所得到。此色彩重建處理步驟一般稱為解碼賽克(demosaicking)或色彩濾波陣列內插(CFA interpolation)。
色彩解碼賽克包含了對於影像內容結構及色彩的分析，並且對於最後的影像品質有相當關鍵性的影響。它通常也是數位相機內所有影像處理流程計算複雜度較高的一個步驟。這樣的原因造成了一個可以在較少的運算成本下重建出高度逼真全彩影像的需求。在這個動機下，我們提出兩種具有高效率及高實用性的軟性決定(soft-decision)解碼賽克演算法。第一個演算法是以方向性內插及嵌入式修正為基礎。此方法的新穎性有兩點。第一，我們突破了在貝爾碼賽克影像之下無法有效的進行對角斜邊重建的限制。演算法中所採用的方向性內插技術對於垂直、水平、四十五度斜角及一百三十五度斜角的邊緣特徵都能有效的重建。此外，所提出的嵌入式修正技術可應用於其它以軟性決定為基礎的解碼賽克方法中，達到能以較少的計算來改善影像品質目的。第一個解碼賽克演算法已經被驗證可以在不需要高度的運算成本之下，顯著的保留較多邊緣細節和處理細微紋理。
我們提出的第二個解碼賽演算法則是利用解交錯與中值濾波技術的優點。首先將取樣過的綠色平面視為一個對角交錯的圖樣，並且使用一些有關解交錯的觀念來幫助提高邊緣預測的準確性及多向性。此外，我們也提出了一種可以偵測只有一個像素寬度的邊緣特徵技術。接著我們再利用以中值濾波為基礎的技術去消除大部分可察覺的色彩瑕疵。實驗結果顯示，第二個解碼賽克演算法可以有效的降低重建錯誤、保留較銳利的影像邊緣特徵及有較佳視覺審視品質，同時維持著良好的計算效率。
在消費性的數位相機設計上，拍攝的影像通常會被經過內插放大以彌補光學性能的限制。幾乎所有最近提出與解碼賽克順序一起考慮的方法都專注在內插方向錯誤的問題。然而，這些方法在銳利邊緣及細微結構的影像區域仍然容易產生模糊效應。本論文也提出一個新的結合演算法來解決上述有關解碼賽克與影像放大處理會遭遇的問題。利用軟性決定的架構，我們計算整合的梯度參數來估計邊緣特徵，包含同時在空間域及譜頻域來估計梯度資訊。接著，為了保持邊緣一致性及提升計算效率，我們繼續在整個演算法中不同的內插步驟利用這些邊緣指示參數。此外，我們發展出一個具邊緣適應性的迭代反投影技術來補償影像模糊以及更進一步的減少重建誤差造成的色彩瑕疵。實驗結果顯示，這個新的結合解碼賽克及放大的演算法不但在客觀的測量準則上有顯著的效能提升，放大的影像也有更清楚輪廓。

Most digital cameras adopt a single-sensor technique to capture visual scenes. This type of cameras reduces costs by placing a Bayer color filter array (CFA) in front of the image sensor, usually a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS), to simultaneously capture primary colors (red, green, and blue). At each pixel location, only one color intensity magnitude is record. To obtain a full-color image from the mosaicked sensor data, a reconstruction step of missing color components must be undertaken. This processing operation is called demosaicking or CFA interpolation.
Color demosaicking involves both structural content and color analysis and is critical to the image quality of single-sensor cameras. Accordingly, it is usually the most computational demanding procedure of the camera image processing pipeline. This drives the demands for achieving high-quality images with a reduced computational cost. Motivated by this, we propose two efficient and effective soft-decision demosaicking algorithms to meet this condition. The first algorithm is based on directional interpolation and embedded artifact refinement. The novelty of this method is twofold. First, we lift the constraint of the Bayer CFA that results in the absence of diagonal neighboring green color values for directionally recovering diagonal edges. The developed directional interpolation method is fairly robust in dealing with the four edge features, namely, vertical, horizontal, 45 diagonal, and 135 diagonal. In addition, the proposed embedded refinement technique provides an efficient way for soft-decision-based algorithms to achieve improved results with fewer computations. The first algorithm have been demonstrated that it can outstandingly preserve more edge details and handle fine textures well, without requiring a high computational cost.
Instead, the second demosaicking algorithm takes advantage of deinterlacing and median-based filtering techniques. We treat the sampled green data of Bayer CFA as a form of diagonal interlaced green planes and make use of some key concepts about spatial deinterlacing to help the edge estimation in terms of both various directions and accuracy. In addition, a specific edge feature, sharp line edge of width 1 pixel, can also be handed well by the proposed method. The median-based filtering techniques are developed for suppressing most visual demosaicking artifacts, such as zipper effect, false color artifact, and interpolation artifact. Experimental results show that our second algorithm is effective in suppressing visual artifacts, preserving the edges of image with sharpness and satisfying visual inspection, while keeping computational efficiency.
In the consumer-class digital cameras, the captured images are upsampled to overcome the limitations of optical capabilities with an interpolation solution. Almost all of the recent approaches identified, with respect to the demosaicking step in the imaging pipeline, have chiefly focused on misguidance problems. However, in regions consisting of sharp edges or fine textures, these approaches are prone to large blurring effects. We propose a new joint solution to overcome the above problems associated with demosaicking and zooming operations. On the basis of a soft-decision framework, we estimate the edge features by computing the integrated gradients. This allows the extraction of gradient information from both color intensity and color difference domains, simultaneously. Then, the edge guidance is incorporated in the interpolation of various stages to preserve edge consistency and improve computational efficiency. In addition, an edge-adaptive, iterative, back-projection technique is developed to compensate for image blurring as well as to further suppress color artifacts. Experimental results indicate that the new joint demosaicking and upsampling algorithm produces outstanding objective performances and sharp, visually pleasing color outputs, when compared to numerous other single-sensor image zooming solutions.

[1] B. E. Bayer, “Color imaging array,” U.S. Patent 3 971 065, July 20, 1976.
[2] R. Lukac, Ed., Single-Sensor Imaging: Methods and applications for digital cameras.
Atlanta, GA: CRC Press, 2008.
[3] B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: Color filter array interpolation in single chip digital cameras,” IEEE Processing Magazine, no. 9, pp. 44-54, Sep. 2004.
[4] X. Li, B. K. Gunturk, and L. Zhang, “Image demosaicing: A systematic survey,” Proc. SPIE-IS&T Electronic Imaging, Visual Comm. and Image Proc., vol. 6822, no. 1, pp. 68221J-68221J-15, Jan. 2008.
[5] S. Battiato, M. Guarnera, G. Messina, and V. Tomaselli, “Recent patents on color demosaicing”, Recent Patents on Computer Science, vol. 1, no. 3, pp. 194–207, Nov. 2008.
[6] D. Cok, “Signal processing method and apparatus for producing interpolated chrominance values in a sampled color image signal,” U.S. Patent No. 4,642,678, 1987.
[7] S. C. Pei and I. K. Tam, “Effective color interpolation in CCD color filter arrays using signal correlation,” IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.6, pp. 503-513, June 2003.
[8] R. Lukac and K.N. Plataniotis, “Normalized Color-Ratio Modeling for CFA Interpolation,” IEEE Transactions on Consumer Electronics, vol. 50, no. 2, pp. 737-745, May 2004.
[9] J. E. Adams, “Design of practical color filter array interpolation algorithms for digital cameras,” Proc. SPIE, vol. 3028, pp. 117–125, Feb. 1997.
[10] R. Kimmel, “Demosaicing: Image reconstruction from color CCD samples,” IEEE Transaction on Image Processing, vol. 8, pp. 1221–1228, Sept. 1999.
[11] E. Chang, S. Cheung, and D. Pan, “Color filter array recovery using a threshold-based variable number of gradients,” Proc. SPIE, vol. 3650, pp. 36-43, 1999.
[12] N. Kehtarnavaz, H.J Oh, and Y. Yoo, “Color filter array interpolation using color correlations and directional derivatives,” Journal of Electronic Imaging, vol. 12, no. 4, pp. 621-632 October 2003.
[13] Lukac, K.N. Plataniotis, D. Hatzinakos, and M. Aleksic, “A novel cost effective demosaicing approach,” IEEE Transactions on Consumer Electronics, vol. 50, no. 1, pp. 256-261, February 2004.
[14] Lu and Y. P. Tan, “Color filter array demosaicing: New method and performance measures”, IEEE Transactions on Image Processing, vol. 12, no. 10, pp. 1194-1210, Oct., 2003.
[15] L. Chang and Y. P. Tan, “Effective use of spatial and spectral correlations for color filter array demosaicing,” IEEE Transactions on Consumer Electronics, vol. 50, no. 1, pp. 355–365, Feb. 2004.
[16] K. H. Chung and Y. H. Chan, “Color demosaicing using variance of color differences,” IEEE Transactions on Image Processing, vol. 15, no. 10, pp. 2944-2955, Oct. 2006.
[17] C.Y. Tsai and K. T. Song, “Heterogeneity-projection hard-decision color interpolation using spectral-spatial correlation,” IEEE Transactions on Image Processing, vol. 16, no. 1, pp. 78-91, 2007.
[18] K. H. Chung and Y. H. Chan, “Low-complexity color demosaicing algorithm based on integrated gradients,” Journal of Electronic Imaging, vol. 19, no. 2, pp. 021104-021104-15, June, 2010.
[19] H. Siddiqui and H. Hwang, “Training-based demosaicing,” IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 1034-1037, Mar., 2010.
[20] B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Color plane interpolation using alternating projections,” IEEE Transactions on Image Processing, vol.11, no.9, pp.997-1013, 2002.
[21] X. Li, “Demosaicing by successive approximation,” IEEE Transactions on Image Processing, vol. 14, no. 3, pp. 370–379, Mar. 2005.
[22] D. Alleysson, S. Susstrunk, and J. Herault, “Linear demosaicking inspired by human visual system,” IEEE Transactions on Image Processing, vol. 14, no. 4, pp. 439–449, Apr. 2005.
[23] N. X. Lian, L. Chang, Y. P. Tan, and V. Zagorodnov, “Adaptive filtering for color filter array demosaicking,” IEEE Transactions on Image Processing, vol. 16, no. 10, pp. 2515-2525, Oct. 2007.
[24] K. Hirakawa and T. W. Parks, “Adaptive homogeneity-directed demosaicing algorithm,” IEEE Transactions on Image Processing, vol. 14, no. 3, pp. 360–369, Mar. 2005.
[25] X. Wu and N. Zhang, “Primary-consistent soft-decision color demosaicking for digital cameras (patent pending), ” IEEE Transactions on Image Processing, vol. 13, no. 9, pp. 1263–1274, Sep. 2004.
[26] D. Menon, S. Andriani and G. Calvagno, “Demosaicing with directional Filtering and a posteriori decision,” IEEE Transactions on Image Processing, vol. 16, no. 1, pp.132-141 Jan. 2007.
[27] L. Zhang and X. Wu, “Color demosaicking via directional linear minimum mean square-error estimation”, IEEE Transactions on Image Processing, vol. 14, no. 12, pp. 2167-2178, Dec. 2005.
[28] S. Ferradans, M. Bertalmio, and V. Caselles, “Geometry-based demosaicking,” IEEE Transactions on Image Processing, vol. 18, no. 3, pp. 665-670, 2009.
[29] A. Buades, B. Coll, J.-M. Morel, and C. Sbert, Self-similarity driven color demosaicking, IEEE Transactions on Image Processing, vol. 18, no. 6, pp. 1192-1202, June 2009.
[30] X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Transactions on Image Processing, vol. 10, no. 410, pp. 1521-1527, Oct. 2001.
[31] S. Battiato, G. Gallo and F. Stanco, “A locally adaptive zooming algorithm for digital images”, Image and Vision Computing, vol. 20, no. 11, pp.805-812, Sep. 2002.
[32] L. Zhang and X. Wu, “An edge-guided image interpolation algorithm via directional filtering and data fusion”, IEEE Transactions on Image Processing, vol.15, no. 8, pp.2226-2238, Aug. 2006.
[33] N. Herodotou and A. N. Venetsanopoulos, “Colour image interpolation for high resolution acquisition and display devices”, IEEE Transactions on Consumer Electronics, vol. 41, no. 4, pp.1118-1126, Nov. 1995.
[34] R. Lukac, B. Smolka, K. Martin and K. N. Plataniotis, “Vector filtering for color imaging”, IEEE Signal Processing Magazine, vol. 22, no. 1, pp.74-86, Jan. 2005.
[35] R. Lukac, K. N. Plataniotis, B. Smolka and A. N. Venetsanopoulos, “Vector operators for color image zooming”, Proc. IEEE International Symposium on Industrial Electronics ISIE’05, vol. 3, pp.1273-1277, Jun. 2005.
[36] K. H. Chung and Y. H. Chan, “A low-complexity joint color demosaicking and zooming algorithm for digital camera,” IEEE Transactions on Image Processing, vol. 16, no. 7, pp. 1705–1715, 2007.
[37] R. Lukac, K. Martin and K. N. Plataniotis, “Digital camera zooming based on unified CFA image processing steps”, IEEE Transactions on Consumer Electronics, vol. 50, no. 1, pp.15-24, Feb. 2004.
[38] R. Lukac, K. N. Plataniotis and D. Hatzinakos, “Color image zooming on the Bayer pattern”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 15, no. 11, pp.1475-1492, Nov. 2005.
[39] K. L. Chung, W. J. Yang, J. H. Yu, W. M. Yan, and C. S. Fuh, “Novel quality-effective zooming algorithm for color filter array,” Journal of Electronic Imaging, Vol. 19, No. 1, pp. 013005-013005-15, 2009.
[40] L. Zhang and D. Zhang, “A joint demosaicking-zooming scheme for single chip digital color cameras,” Computer Vision and Image Understanding, vol. 107, no. 1–2, pp. 14–25, 2007.
[41] K. L. Chung, W. J. Yang, P. Y. Chen, W. M. Yan, and C. S. Fuh, “New joint demosaicing and zooming algorithm for color filter array,” IEEE Transactions on Consumer Electronics, Vol. 55, No. 3, pp. 1477-1486, 2009.
[42] R. C. Gonzalez and R. E.Woods, Digital Image Processing, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 2002.
[43] I. Omer and M. Werman, “Using natural image properties as demosaicing hints,” Proc. IEEE Int. Conf. Image Processing, pp. 1665–1670, 2004.
[44] Kodak photoCD. [Available online: http://r0k.us/graphics/kodak/].
[45] H. A. Chang and H. Chen, “Stochastic color interpolation for digital cameras” IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, no. 8, pp. 964-973 Aug. 2007.
[46] M. Mahy, E. V. Eyckden, and O. Oosterlinck, “Evaluation of uniform color spaces developed after the adoption of CIELAB and CIELUV,” Color research and application, vol. 19, no. 2, pp. 105–121, 1994.
[47] Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.
[48] Y. Hoon, and J. Jechang, “Direction-oriented interpolation and its application to de-interlacing,” IEEE Transactions on Consumer Electronics, vol. 48, no. 4, pp. 954-962, 2002.
[49] M. Q. Phu, P. E. Tischer, and H. R. Wu, “A median based interpolation algorithm for deinterlacing,” Proc. International Symposium on Intelligent Signal Processing and Communication Systems, pp. 390-397, 2004.
[50] M. K. Park, M. Byun, J. H. Park and M. G. Kang, “Edge-dependent interpolation-based deinterlacing using edge patterns,” Journal of Electronic Imaging, vol. 15, no. 4, pp. 043003- 04303-8, 2006.
[51] Y. R. Chen and S. C. Tai, “True motion-compensated de-interlacing algorithm,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 19, no. 10, pp. 1489 -1498, Oct. 2009.
[52] T. Doyle, “Interlaced to sequential conversion for EDTV applications,” Proc. 2nd Int. Workshop Signal Processing of HDTV, pp.412-430, Feb. 1998.
[53] W. T. Freeman, “Median filter for reconstructing missing color samples”, U. S. Patent, no. 4,724,395, 1998.
[54] Lanlan Chang and Yap-Peng Tan, “Hybrid color filter array demosaicking for effective artifact suppression”, Journal of Electronic Imaging, vol. 15, Mar. 2006.
[55] S. J. Ko and Y. H. Lee, “Center weighted median filters and their applications to image enhancement,” IEEE Transactions on Circuits and Systems for video technology, vol. 38, no.9, pp. 984-993, 1991.
[56] L. Yin, R. Yang, M. Gabbouj, Y. Neuvo. “Weighted median filters: A tutorial”, IEEE Transactions on Circuits and Systems, no. 43, vol. 3, pp. 157-192, Mar. 1996.
[57] R. Lukac and K.N. Plataniotis, “A new image sharpening approach for single-sensor digital cameras”, International Journal of Imaging Systems and Technology, vol. 17, no.3, pp. 123-131, June 2007.
[58] M. Irani and S. Peleg, “Motion analysis for image enhancement: resolution, occlusion and transparency,” JVCIP, 1993.
[59] J. S. Jimmy Li, Sharmil Randhawa, “Color filter array demosaicking using high-order interpolation techniques with a weighted median filter for sharp color edge preservation”, IEEE Transactions on Image Processing, vol. 18, no. 9, pp.1946-1957, 2009.
[60] S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Magazine, vol.20, no. 3, pp. 21-36, May 2003.
[61] S. Dai, M. Han, Y. Wu, and Y. Gong, “Bilateral back-projection for single image super resolution”, IEEE International Conference on Multimedia & Expo (ICME'07), Beijing, China, July 2-5, pp. 1039-1042, 2007.
[62] J.Mukherjee, R. Parthasarathi, and S. Goyal, “Markov random field processing for color demosaicing,” Pattern Recognition Letters, vol. 22, no. 3-4, pp. 339–351, March 2001.
[63] D. D. Muresan and T. W. Parks, “Demosaicing using optimal recovery,” IEEE Transactions on Image Processing, vol. 14, no. 2, pp. 267–278, Feb. 2005.
[64] K. L. Chung, W. J. Yang, W. M. Yan, and C. C. Wang, “Demosaicing of color filter array captured images using gradient edge detection masks and adaptive heterogeneity-projection”, IEEE Transactions on Image Processing, vol. 17, no. 12, pp. 2356 -2367, Dec. 2008.
[65] K. L. Chung, W. J. Yang, W. M. Yan, and C. S. Fuh, “New joint demosaicing and arbitrary-ratio resizing algorithm for color filter array using DCT approach,” IEEE Trans. Consumer Electronics, vol. 56, no. 2, pp. 783-791, May 2010.
[66] F. Zhang, X. Wu, X. Yang, W. Zhang, L. Zhang, “Robust color demosaicking with adaptation to varying spectral correlations,” IEEE Transactions on Image Processing, vol. 18, no. 12, pp. 2706-2717, Dec. 2009.