在本論文中，我們提出一個不需要使用棋盤格的魚眼攝影機校正方法，我們將二維的影像透過一個投影方程式投影至三維球體表面上。利用三維球體的物理性質，我們可以在三維球體的中心建立虛擬攝影機，並將球體表面投影至虛擬攝影機的影像平面上，得到一個沒有形變失真的影像結果。
對於飛行時間距離感測器測得之深度資訊的校正，在使用平滑化演算法之前，我們使用一個基於穩定非區域性均值濾波器的離群值估算方法將離群值濾除。為了使結果更好，在平滑化時除了使用深度資訊外，我們可以在深度與色彩影像對準後，將色彩資訊也加入到平滑化演算法之中。與原始演算法相比，我們的方法在不失去精準度的情況下大幅縮小程式運算量。

We propose a real-time fisheye calibration method without using chessboard. We project 2D image back to 3D viewing sphere according to a fisheye distortion function. With the properties of 3D viewing sphere, we set a virtual camera at centroid of viewing sphere; project the sphere surface to image plane of virtual camera and get a non-distorted image result.
For depth data from time-of-flight sensor, we use an outlier estimation method to remove outliers before applying smooth algorithm based on robust non-local means filter. To get a better smooth result, color information can be taken into account as additional weights factor for smoothing filter by aligning depth sensor and color camera. Comparing with original robust non-local means filter, our method reduce computational cost without losing PSNR.

[1] J.A. Bilmes, “A Gentle Tutorial of the EM algorithm and Its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models,” International Computer Science Institute, Vol. 4, No. 510, pp.126, 1998.
[2] A. Buades, B. Coll, and J.-M. Morel, “A Non-Local Algorithm for Image Denoising,” Computer Vision and Pattern Recognition, IEEE Conference on, Vol. 2, pp. 60-65, 2005.
[3] R. Carroll, M. Agrawala, and A. Agarwala, “Optimizing Content-Preserving Projections for Wide-Angle Images,” ACM Transactions on Graphics, Vol. 28, No. 3, pp. 43, 2009.
[4] A.P. Dempster, N.M. Laird, and D.B. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” Journal of the Royal Statistical Society, Vol. 39, pp. 1–38, 1977.
[5] F. Devernay, and O. Faugeras, “Straight Lines Have to Be Straight,” Machine vision and applications, Vol. 13, No. 1, pp.14-24, 2001.
[6] M. Frank, M. Plaue, and F.A. Hamprecht, “Denoising of Continuous-Wave Time-of-Flight Depth Images Using Confidence Measures,” Optical Engineering, Vol. 48, No. 7, pp. 077003-077003, 2009.
[7] H. Fu, and X. Cao, “Forgery Authentication in Extreme Wide-Angle Lens Using Distortion Cue and Fake Saliency Map,” Information Forensics and Security, Vol. 7, No. 4, pp. 1301-1314, 2012.
[8] C. Geyer, and K. Daniilidis, “A Unifying Theory for Central Panoramic Systems and Practical Implications.” Computer Vision—ECCV 2000. Springer Berlin Heidelberg, pp. 445-461, 2000.
[9] C. Hughes, P. Denny, M. Glavin, and E. Jones, “Equidistant Fisheye Calibration and Rectification by Vanishing Point Extraction,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 32, No.12, pp. 2289-2296, 2010.
[10] C. Hughes, P. Denny, E. Jones, and M. Glavin, “Accuracy of Fisheye Lens Models,” Applied optics, Vol. 49, No.17, pp. 3338-3347, 2010.
[11] B. Huhle, T. Schairer, P. Jenke, and W. Straßer, “Robust Non-Local Denoising of Colored Depth Data,” Computer Vision and Pattern Recognition Workshop, IEEE Computer Society Conference on, pp. 1-7, 2008.
[12] J. Kannal, and S.S. Brandt, “A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fisheye Lenses,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 28, No. 8, pp. 1335-1340, 2006.
[13] S.-Y. Kim, J.-H. Cho, A. Koschan, and M.A. Abidi, “Spatial and Temporal Enhancement of Depth Images Captured by a Time-of-Flight Depth Sensor,” Pattern Recognition (ICPR), pp. 2358-2361, 2010.
[14] K, Miyamoto, “Fish eye Lens,” JOSA, Vol. 54, No. 8, pp. 1060-1061, 1964.
[15] O. Schall, A. Belyaev, and H.-P. Seidel, “Adaptive Feature-Preserving Non-Local Denoising of Static and Time-Varying Range Data,” Computer-Aided Design, Vol. 40, No. 6, pp. 701-707, 2008.
[16] D. Schneider, E. Schwalbe, and H.-G. Maas, “Validation of Geometric Models for Fisheye Lenses,” ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 64, pp. 259–266, 2009.
[17] E. Schwalbe, “Geometric Modelling and Calibration of Fisheye Lens Camera Systems,” Proc. 2nd Panoramic Photogrammetry Workshop, Int. Archives of Photogrammetry and Remote Sensing, Vol. 36, No. Part 5, 2005.
[18] S.M. Smith, and J.M. Brady, “SUSAN—a New Approach to Low Level Image Processing,” International journal of computer vision, Vol. 23, No. 1, pp. 45-78, 1997.
[19] C. Tomasi, and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” Computer Vision, IEEE Sixth International Conference on, pp. 539-846, 1998.
[20] R.Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-The-Shelf TV Cameras and Lenses,” IEEE Journal of Robotics and Automation, Vol. 3, No.4, pp. 323-344, 1987.
[21] J. Wei, C.-F. Li, S.-M. Hu, R.R. Martin, and C.-L. Tai, “Fisheye Video Correction,” Visualization and Computer Graphics, IEEE Transactions on, Vol. 18, No. 10, pp. 1771-1783, 2012.
[22] X. Ying, and Z. Hu, “Can We Consider Central Catadioptric Cameras and Fisheye Cameras within a Unified Imaging Model,” Computer Vision-ECCV, Springer Berlin Heidelberg, pp. 442-455, 2004.
[23] Z. Zhang, “A Flexible New Technique for Camera Calibration,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 22, No. 11, pp. 1330-1334, 2000.
[24] J. Zhu, L. Wang, R. Yang, and J. Davis, “Fusion of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps,” Computer Vision and Pattern Recognition, IEEE Conference on, pp. 1-8, 2008.