在本篇論文中，提出深度圖增強與補洞演算法，此演算法利用時間上的參考資訊來提升DIBR系統中合成影像的視覺品質。深度圖增強演算法藉由維持深度值與物體的尺度變化的一致性以及深度值在不同時間點上的一致性來提升3D效果並減少影像閃爍的問題。此演算法藉由尺度變化來估算物體在三維空間中Z軸上的位移量以及位置來維持深度值與物體的尺度變化的一致性，是目前其他的深度圖增強演算法無法達成的。因為大部分的深度圖增強演算法個別處理每張深度圖，在不同時間點上物體深度值的一致性是非常薄弱的，而一些利用時間上的資訊來增強深度圖的演算法，其維持物體深度值在不同時間點上的一致性的能力也是不足的,尤其是在移動的物體。因此，在本論文中，提出一個動態補償無限脈衝響應濾波器，沿著物體移動的軌跡來對深度圖濾波並同時保留邊緣的資訊來降低不同時間點上的不一致性。本論文提出的補洞演算法加入了時間上的資訊，利用不同時間點上影像中沒有被物體遮蔽的區域來修補在合成影像中的洞，來提升合成影像的視覺品質。從實驗結果中可以看出本論文提出的深度圖增強與補洞演算法可以明顯的提升合成影像的視覺品質。

This thesis proposes a depth map enhancement and hole filling algorithm that uses temporal reference information to improve the visual quality of synthesized virtual views in a DIBR system. The depth map enhancement algorithm maintains the scaling ratio and depth value consistencies and temporal consistencies in difference time instant to enhance the three-dimensional (3D) effect and reduce the flicker problem. This algorithm estimates the Z-displacement and position in 3D space based on the scaling ratio of an object in order to maintain the consistency of the scaling ratio and the depth value, which cannot be achieved by existing depth map improvement algorithms. Many depth map enhancement algorithms process each frame individually, and therefore, the temporal consistency of the resulting depth map is weak. Some enhancement algorithms include temporal information; however, their ability to reduce temporal inconsistencies is poor, especially when the object is moving. Therefore, this thesis also proposes a motion-compensated IIR depth filter that smoothes the depth value along the moving trajectory of the object and preserves edges to reduce temporal inconsistency. To fill a hole in a synthesized virtual view, the proposed algorithm adds temporal information, which is the uncovered region in a temporal reference frame, to improve visual quality. The results show that the visual quality of the synthesized virtual view is enhanced by the proposed depth map enhancement and hole filling algorithm.

[1] C. Fehn, "Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV", Proc. SPIE Conf. Stereoscopic Displays and Virtual Reality Systems XI, vol. 5291, pp.93 -104 2004.
[2] A. Redert, M. Op de Beeck, C. Fehn, W. IJsselsteijn, M. Pollefeys, L. Van Gool, E. Ofek, I. Sexton, and P. Surman, "ATTEST—advanced three-dimensional television system techniques", Proc. 3DPVT" 02, pp.313 -319 2002.
[3] C. Fehn, K. Hopf and Q. Quante, "Key Technologies for an Advanced 3D-TV System" In Proceedings of SPIE Three-Dimensional TV, Video and Display III, Philadephia, PA, USA, October 2004.pp. 66-80.
[4] C. Fehn, “A 3D-TV approach using depth-image-based-rendering (DIBR),” in Proc. Visualization, Imaging, and Image Processing, Benalmadena, Spain, Sep. 2003, pp. 482–487.
[5] G. Iddan and G. Yahav, "3D imaging in the studio (and elsewhere …)", Proc. SPIE, vol. 4298, pp.48 -55 2001.
[6] D. Kim, D. Min, and K. Sohn, “Stereoscopic video generation method using motion analysis,” in Proc. 3DTV Conf., 2007, pp. 1–4.
[7] I. Ideses, L. P. Yaroslavsky, and B. Fishbain, “Real-time 2D to 3D video conversion,” J. Real-Time Image Proc., vol. 2, no. 1, pp. 3–9, 2007.
[8] W. J. Tam and L. Zhang, “3D-TV content generation: 2D-to-3D conversion,” in Proc. ICME, 2006, pp. 1869–1872.
[9] J. Kim, A. Baik, and Y. J. Jung, “2D-to-3D conversion by using visual attention analysis,” in Proc. SPIE, 2010, vol. 7524, etc.
[10] W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, "Smoothing depth maps for improved stereoscopic image quality", Proc. SPIE Conf. Three-Dimensional TV, Video, and Display III, vol. 5599, pp.162 -172 2004.
[11] L. Zhang & W. J. Tam, “Stereoscopic image generation based on depth images for 3D TV,” IEEE Transactions on Broadcasting, Vol. 51, pp. 191-199, 2005.
[12] W. J. Tam & L. Zhang, "Non-uniform smoothing of depth maps before image-based rendering," Proceedings of Three-Dimensional TV, Video and Display III (ITCOM'04), Vol. 5599, pp. 173-183, Philadelphia, USA, Oct. 25-28, 2004.
[13] L. Zhang, J. Tam, & D. Wang, “Stereoscopic image generation based on depth images,” Proceedings of the IEEE Conference on Image Processing, pp. 2993-2996, Singapore, Oct. 2004.
[14] W. Chen, Y. Chang, S. Lin, L. Ding, L. Chen, Efficient depth image based rendering with edge dependent depth filter and interpolation, in: Proceedings of International Conference on Multimedia and Expo, 2005, pp. 1314–1317.
[15] Sang-Beom Lee, Sung-Yeol Kim, and Yo-Sung Ho, "Multi-view image generation from depth maps and texture images using adaptive smoothing filter," International Conference on Embedded Systems and Intelligent Technology (ICESIT), Bangkok, Thailand, pp. 353-358, 2008.
[16] Johannes Kopf , Michael F. Cohen , Dani Lischinski , Matt Uyttendaele, Joint bilateral upsampling, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007.
[17] S. Liu, P. Lai, D. Tian, C. Gomila, and C. Chen, “Joint trilateral filtering for depth map compression,” in Proceedings of SPIE, vol. 7744, 2010, p. 77440F.
[18] Sung-Yeol Kim, Ji-Ho Cho, Andreas Koschan, and Mongi A. Abidi, "Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor", in Proc. of the IEEE International Conference on Pattern Recognition (ICPR), pp. 2358-2361, 2010.
[19] C. M. Cheng, S. J. Lin, and S. H. Lai, “Spatio-temporally consistent novel view synthesis algorithm from video-plus-depth sequences for autostereoscopic displays,” IEEE Trans. Broadcast., vol. 57, no. 2, pp. 523–532, Jun. 2011.
[20] W.-Y. Chen, Y.-L. Chang, S.-F. Lin, L.-F. Ding, and L.-G. Chen, "Efficient depth image based rendering with edge dependent depth filter and interpolation," in ICME 2005 : IEEE International Conference on Multimedia and Expo, 6-8 2005, pp. 1314-1317.
[21] Cheon Lee, Yo-Sung Ho. "Temporally Consistent Hole Filling for View Extrapolation". ISO/IEC JTC1/SC29/WG11, m21282, pp. 1-7, July 2011.
[22] B. S. Reddy and B. N. Chatterji, "An FFT-based technique for translation, rotation, and scale-invariant image registration," Image Processing, IEEE Transactions on, vol. 5, pp. 1266-1271, 1996.
[23] C.D. Kuglin and D.C. Hines, “The Phase Correlation Image Alignment Method,” Proc. IEEE Conf. Cybernetics and Soc., pp. 163-165, 1975.
[24] G. Tzimiropoulos, V. Argyriou, S. Zafeiriou, and T. Stathaki, "Robust FFT-Based Scale-Invariant Image Registration with Image Gradients," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 32, pp. 1899-1906, 2010.
[25] Gwo Giun (Chris) Lee, Yen-Kuang Chen, Marco Mattavelli and Euee S. Jang, “Algorithm/Architecture Co-Exploration of Visual Computing on Emergent Platforms: Overview and Future Prospects,” IEEE Trans. on Circuits and Systems for Video Technology, Vol.19, Iss. 11, pp. 1576-1587, Nov. 2009.
[26] A. V. Oppenheim, R. W. Schafer, and J. R. Buck, Discrete-Time Signal Processing: Prentice-Hall, 1999.
[27] Chun-Fu Chen, Gwo Giun (Chris) Lee, Jui-Che Wu, Ching-Jui Hsiao, Jun-Yuan Ke, “Variable Block Size Motion Estimator design for Scan Rate Up-convertor”, Proc. IEEE Workshop Signal Processing Systems Design and Implementation (SIPS "12), Oct. 2012.
[28] G. G. Lee, M.-J. Wang, H.-Y. Lin, D. W.-C. Su, and B.-Y. Lin, "Algorithm/architecture co-design of 3-D spatio-temporal motion estimation for video coding," IEEE Transactions on Multimedia, vol. 9, no. 3, pp. 455-465, 2007.