影像在擷取、傳輸與處理的過程中總是不可避免的會受到雜訊的干擾或破壞。特定的雜訊來源本來就已經固定在攝影機的硬體中，在低照度的情況下拍攝影像時，雜訊會跟著影像本身一起被放大。有些雜訊則是因為影像透過類比信號的途徑傳送，容易被頻道的雜訊干擾，例如衛星廣播。更進一步的探討就會發現，有些影像處理的技術本身就容易伴隨新的雜訊發生，例如：影像的量化和強化或銳利化。

影像雜訊去除的演算法往往需要作參數的最佳化調整，而這些調整如果可以根據雜訊的高低作因應，就可以得到比較好的效果。這使得影像雜訊的預估變成一個很重要的研究課題。在這本博士論文的第一個部分，我們首先提出一個快速又可靠的混合式策略來進行雜訊的預估。我們假設的雜訊數學模型是additive white Gaussian noise (AWGN)。

首先，將輸入的雜訊影像切割成小方塊，並對每個小方塊實施Sobel運算子的運算。利用Sobel運算子邊緣偵測的能力，我們可以找出具邊緣的方塊，並將它排除在雜訊預估的過程之外。判斷是否具邊緣所依據的臨界標準則可以從Sobel運算結果的統計直方圖中找到。這樣子找出來的相對平滑區域再施以濾波運算，藉由濾波結果的統計平均就可以提供一個準確的雜訊預估。我們使用各種不同類型的測試圖做為實驗的對象，加入的雜訊值則由低到高，結果顯示這樣的方法既快速又有效。

基於上述演算法的成功經驗，我們進一步考慮時間域的資訊，提出適合視訊資料使用的雜訊預估方法；並且提出一個結合雜訊預估與去除的共通性架構，希望將預估的結果成功運用在去除雜訊上。首先我們探討使用Sobel以外其他運算子的可行性，基本上只要可以提供影像平滑度量測的運算子都可以考慮。不同運算子對於預估的準確度以及可以提供的額外資訊都被詳細討論。例如：Sobel運算子可以告訴我們邊緣的方向，其他二階梯度運算子則可以告訴我們位於邊緣上的像素是處於比較亮的一邊或者比較暗的一邊。接著我們回顧兩個已經被廣泛應用的非線性濾波器，雙邊濾波器(bilateral filter)及非區域平均濾波器(non-local mean filter)。兩個濾波器分別有二到四個不等的參數需要調整設定，我們透過廣泛的實驗試著找出最佳的經驗法則。結果顯示，最佳的設定值都與雜訊的高低有關。我們更進一步利用邊緣的資訊來改善這兩個濾波器，讓雙邊濾波器可以有邊緣強化的輸出，並且讓非區域平均濾波器可以降低高達75%的運算複雜度，同時還保有相同的影像品質。結論就是我們提出一個可以成功結合影像雜訊預估與去除的架構，並且可以有效達到去除雜訊的效果。

Noise can always be introduced into digital images and videos during acquisition, transmission and processing. Certain noise sources are located in the camera hardware and become amplified under low light conditions. Other noise sources are due to interference during transmission. For example, images transmitted over wireless network might be corrupted by lightning or atmospheric disturbance. Further noise can be introduced by image processing, e.g., quantization and enhancement.

Image denoising algorithms often require their parameters to be adjusted according to the noise level. In this work, we first propose a hybrid approach which is shown to be fast and reliable for estimating image noise. The input image is assumed to be contaminated by an additive white Gaussian noise (AWGN) process. To exclude structures or details from contributing to the estimation of noise variance, a Sobel edge detection operator with a self-determined threshold is first applied to each image block. Then a filter operation, followed by an averaging of the convolutions over the selected blocks, will provide a very accurate estimation of noise variance. We successfully combine the effectiveness of filter-based approaches with the efficiency of block-based approaches, and the simulated results demonstrate that the proposed method performs well for a variety of images over a large range of noise variances. Performance comparisons against other approaches are also provided.

Based on the success of the hybrid approach, a video noise estimator is proposed where both spatial and temporal information are considered. Then, a common framework for combining image noise estimation and reduction is presented. Different operators for identifying the homogeneous blocks are discussed and their performances on noise estimation are compared. Then, two well-known filters, the bilateral and the non-local mean, are reviewed. There are two and four parameters to be decided, respectively. The best settings are suggested by empirical results. It is shown that the rules of thumb really depend on the estimated noise level. A new bilateral filer with edge enhancement is proposed where sharpening is made possible by the information provided by the homogeneity analyzer. In addition to that, a modified non-local mean filter with less complexity is also present. Compared to the original non-local mean filter, the complexity is dramatically reduced by 75% and yet the visual quality of the recovered image is maintained. At the end, a complete and effective image restoration system is formed by combing a noise estimator and a non-linear filter.

[1]S. Aja-Fernandez, G. Vegas-Sanchez-Ferrero, M. Martin-Fernandez and C. Alberola- Lopez, “Automatic noise estimation in images using local statistics. Additive and multiplicative cases,” Image and Vision Computing 27(6), 756-770 (2009).
[2]M. Aleksic, M. Smirnov, and S. Goma, “Novel Bilateral Filter Approach: Image Noise Reduction with Sharpening,” in Proceedings of the Digital Photography II Conference, Volume 6069, SPIE, 2006.
[3]A. Amer and E. Dubois, “Fast and Reliable Structure-Oriented Video Noise Estimation,” IEEE Transactions on Circuits & Systems for Video Technology, 15(1), 113-118 (2005).
[4]V. Aurich and J. Weule, “Non-linear Gaussian Filters Performing Edge Preserving Diffusion,” In Proceedings of the DAGM Symposium, 1995.
[5]N. Azzabou, N. Paragios and F. Guichard, “Random walks, constrained multiple hypothesis testing and image enhancement,” in ECCV, Vol. 1, pp. 379-390.
[6]D. Barash, “A fundamental relationship between bilateral filtering, adaptive smoothing and the nonlinear diffusion equation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6):844, 2002.
[7]D. Barash and D. Comaniciu, “A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift,” Image and Video Computing, 22(1):73–81, 2004.
[8]R. C. Bilcu and M. Vehvilainen, “New method for noise estimation in images,” Proceedings of IEEE-Eurasip International Workshop on Nonlinear Signal and Image Processing 290-293 (2005).
[9]R. C. Bilcu and M. Vehvilainen, “Modified Non-Local Averaging for Image Denoising,” WSEAS Transactions on Circuits and Systems, Issue 7, Vol. 5, July 2006.
[10]J. Boulanger, C. Kervrann, and P. Bouthemy, “Adaptive spacetime patch-based method for image sequence restoration,” In Proceedings of the ECCV’06 workshop on statistical methods in multi-image and video processing (SMVP’06), Graz, Austria, May 2006.
[11]A. C. Bovik, “Handbook of Image and Video Processing,” 2nd Ed., Elsevier Academic Press, Amsterdam, 2005.
[12]R. Bracho and A. C. Sanderson, “Segmentation of image based on intensity gradient information,” Proceedings of Conference on Computer Vision Pattern Recognition, 341-347, San Francisco, CA, (1985).
[13]A. Buades, B. Coll, and J. M. Morel, "A Review of Image Denoising Algorithms, with a New One," Multiscale Modeling & Simulation, Vol. 4, No. 2, pp. 490-530, 2005.
[14]A. Buades, B. Coll, and J. M. Morel, "A non-local algorithm for image denoising," IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 60-65, 2005.
[15]A. Buades, B. Coll, and J. M. Morel, "Nonlocal Image and Movie Denoising," International Journal of Computer Vision, Vol. 76, No. 2, pp. 123-139, 2008.
[16]G. Calvagno, F. Fantozzi, R. Rinaldo, and A. Viareggio, “Model-based global and local motion estimation for videoconference sequences,” IEEE Trans. Circuits and Systems for Video Technology, Vol. 14, No. 9, pp. 1156-1161, Sept. 2004.
[17]J. Chen, S. Paris, and F. Durand, “Real-time Edge-Aware Image Processing with the Bilateral Grid,” ACM Transactions on Graphics, 26(3), 2007.
[18]K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, August 2007.
[19]D. L. Donoho and J. M. Johnstone, “Ideal spatial adaptation by wavelet shrinkage,” Biometrika 81(3), 425-455 (1994).
[20]W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, “Learning Low-Level Vision,” International Journal of Computer Vision 40 (1), 25-47 (2000).
[21]M. Ghazal, A. Amer, and A. Ghrayeb, “A real-time technique for spatial-temporal video noise estimation,” IEEE Transactions on Circuits & Systems for Video Technology 17(12), 1690-1699 (2007).
[22]M. Ghazal, A. Amer and A. Ghrayeb, "Structure-Oriented Multidirectional Wiener Filter for Denoising of Image and Video Signals," Circuits and Systems for Video Technology, IEEE Transactions on , Vol.18, No.12, pp.1797-1802, Dec. 2008.
[23]R. C. Gonzalez and R. E. Woods, “Digital Image Processing,” 2nd Ed., Prentice-Hall, Inc., New Jersey, USA, 2002.
[24]J. Immerkær, “Fast Noise Variance Estimation,” Computer Vision and Image Understanding 64(2), 300-302 (1996).
[25]A. K. Jain, “Fundamentals of Digital Image Processing,” Prentice Hall, 1989.
[26]C. Kervrann and J. Boulanger, “Unsupervised patch-based image regularization and representation,” In Proceedings of the European conference on computer vision (ECCV’06), Graz, Austria, May 2006.
[27]H. H. Khalil, R. O. K. Rahmat and W. A. Mahmoud, “Estimation of Noise in Gray-Scale and Colored Images Using Median Absolute Deviation (MAD),” Proceedings of the 2008 3rd International Conference on Geometric Modeling and Imaging, 92-97 (2008).
[28]S. Kindermann, S. Osher, and P. W. Jones, “Deblurring and denoising of images by nonlocal functional,” Multiscale Modeling and Simulation, 4(4), 1091–1115, 2005.
[29]J. S. Lee, “Speckle analysis and smoothing of synthetic aperture radar images,” Computer Graphics and Image Processing, Volume 17, Issue 1, pp. 24-32, Sept. 1981.
[30]J. S. Lee, “Refined filtering of image noise using local statistics,” Computer Graphics and Image Processing, 15:380-389, (1981).
[31]J.-S. Lee, “Digital image smoothing and the sigma filter,” Computer Vision, Graphics, and Image Processing Vol. 24, No. 2, pp. 255-269, 1983.
[32]D. Lowe, “Object Recognition From Local Scale Invariant Features,” Proc. IEEE International Conference Computer Vision, 1150-1157, 1999.
[33]M. Mahmoudi and G. Sapiro, “Fast Image and Video Denoising via Nonlocal Means of Similar Neighborhoods,” IEEE Signal Processing Letters, Vol. 12, No. 2, pp. 839-842, Dec. 2005.
[34]G. A. Mastin, “Adaptive filters for digital image noise smoothing: An evaluation,” Computer Vision, Graphics, and Image Processing Vol. 31, No. 1, pp. 103-121 ,1985.
[35]M. S. Nixon and A. S. Aguado, Feature Extraction and Image Processing, 2nd ed., Academic Press, Boston(2008).
[36]S. I. Olsen, “Estimation of Noise in Images: An Evaluation,” CVGIP: Graphical Models and Image Processing 55(4), 319-323 (1993).
[37]A. Papoulis, “Probability, Random Variables and Stochastic Processes,” McGraw-Hill, 2002.
[38]S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, “A gentle introduction to bilateral filtering and its application,” presented at the ACM SIGGRAPH, 2007.
[39]P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 5, pp. 629–639, May 1990.
[40]I Pitas, “Digital Image Processing Algorithms and Applications,” Wiley, New York, 2000.
[41]C. Poynton, “Digital Video and HDTV, Algorithms and Interfaces,” Morgan Kaufmann Publishers, San Francisco, USA, 2003.
[42]W. H. Press, W. T. Vetterling, S. A. Teukolsky, and B. P. Flannery, “Numerical Recipes in C: The Art of Scientific Computing,” 2dn Ed., Cambridge University Press, Cambridge, England, 1992.
[43]R. Ramanath and W. E. Snyder, “Adaptive Demosaicking,” Journal of Electronic Imaging, 12(4):633-642, 2003.
[44]K. Rank, M. Lendl and R. Unbehauen, “Estimation of image noise variance,” IEE Proceedings on Vision, Image and Signal Processing 146(2), 80-84 (1999).
[45]F. Russo, “A Method Based on Piecewise Linear Models for Accurate Restoration of Images Corrupted by Gaussian Noise,” IEEE Transactions on Instrumentation & Measurement 55(6), 1935-1943 (2006).
[46]F. Russo, “Technique for Image Denoising Based on Adaptive Piecewise Linear Filters and Automatic Parameter Tuning,” IEEE Transactions on Instrumentation & Measurement 55(6), 1362-1367 (2006).
[47]F. Russo, “An Image-Enhancement System Based on Noise Estimation,” in IEEE Transactions on Instrumentation & Measurement 56(4), 1435-1442 (2007).
[48]M. Salmeri, A. Mencattini, E. Ricci and A. Salsano, “Noise estimation in digital images using fuzzy processing,” Proceedings of 2001 International Conference on Image Processing, 517-520 (2001).
[49]G. Sharma, “Digital Color Imaging,” CRC Press LLC, Boca Raton, Florida, USA, 2003.
[50]D.-H. Shin, R.-H. Park, S. Yang and J.-H. Jung, “Block-Based Noise Estimation Using Adaptive Gaussian Filtering,” in IEEE Transactions on Consumer Electronics 51(1), 218-226 (2005).
[51]S. M. Smith and J. M. Brady, “SUSAN – a New Approach to Low Level Image Processing,” International Journal of Computer Vision, 23(1):45-78, May 1997.
[52]B. C. Song and K. W. Chun, "Motion-compensated noise estimation for efficient prefiltering in a video encoder," Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on , Vol.2, pp. II- 211-14, Sept. 2003.
[53]B. C. Song and K. W. Chun, "Noise Power Estimation for Effective De-Noising in a Video Encoder," Acoustics, Speech, and Signal Processing, 2005, Proceedings. (ICASSP '05). IEEE International Conference on , Vol.2, pp. 357- 360, March 18-23, 2005.
[54]S. C. Tai and S. M. Yang, “A fast method for image noise estimation using Laplacian operator and adaptive edge detection,” Proceedings of the 3rd International Symposium on Communications, Control and Signal Processing, ISCCSP 2008, 1077-1081, St Julians (2008).
[55]C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” in Sixth International Conference on Compute, 839-846, New Dclhi, India (1998).
[56]B. Watson, “Perceptual-components architecture for digital video,” Journal of Optical Society, Vol. 7, No. 10, pp. 1943-1954, 1990.
[57]Xiph.Org Foundation. Xiph.org Test Media. Retrieved December 10, 2010, from http://media.xiph.org/video/derf/
[58]S. M. Yang and S. C. Tai, “Fast and reliable image noise estimation using a hybrid approach,” Journal of Electrical Imaging, Vol. 19, No. 3, 033007-1, Jul-Sep 2010.
[59]S. M. Yang and S. C. Tai, “A Fast and Reliable Algorithm for Video Noise Estimation Based on Spatio-Temporal Sobel Gradients,” accepted to be present at the 1st International Conference on Electrical, Control and Computer Engineering, InECCE2011, Pahang, Malaysia, Jun. 21-22, 2011.
[60]M. Zhang, B. Gunturk. “A new image denoising method based on the bilateral filter,” IEEE Int. Conf. on Acoustics, Speech, Signal Processing (ICASSP), 2008, pp.929~932.
[61]B. Zhang and J. P. Allebach, “Adaptive Bilateral Filter for Sharpness Enhancement and Noise Removal,” IEEE Transactions on Image Processing, 17(5), 664-678 (2008).
[62]V. Zlokoloca, A. Pizurica, and W. Philips, “Noise estimation for video processing based on spatio-temporal gradients,” IEEE Signal Process. Lett., 13(6), 373-340, (2006).
[63]V. Zlokolica and W. Philips, “Motion and detail adaptive denoising of video,” Proc. SPIE Electronic Imaging, San Jose, CA, Jan. 2004.