3D 影像已經是現今科技熱門的趨勢之一。近年來娛樂產業、醫療體系等等都將3D應用在生活上，可惜因多視角攝影機拍攝成本昂貴，導致內容缺乏。因此，我們提出精確基於深度影像生成技術(DIBR)，可將2D影像轉成精緻多視角3D影像。其使用單張彩圖及對應深度圖以合成多視角的立體影像。本論文提出次像素曲移(Warping)及深度圖層補洞(Hole filling)演算法以完成高精確度深度影像生成系統。次像素曲移演算法可將像素點曲移到精確的小數點位置，以解決原位移量僅能取整數點不精準的問題。深度圖層補洞演算法利用深度資訊將曲移彩圖(Warped View)分離出數層，各層彩圖依內彩修復(Inpainting)概念尋找周圍相似區塊以填補缺失的資訊優於傳統的背景延伸補洞作法。實驗證明我們的基於深度影像生成系統比較現有方法，可獲得更自然且合理的合成虛擬視角影像。

The 3D related techniques have become one of the hottest topics in multimedia applications nowadays. In recent years, entertainment industry, medical system, etc. have applied 3D techniques for many living applications. Unfortunately, the cost of multi-view shooting is so high that it results in lack of contents. Therefore, the depth image-based rendering (DIBR) algorithm, which can convert 2D image to multiple 3D view images. The DIBR system requires a texture image and its corresponding depth image to generate synthesized views. In the thesis, we propose a precision DIBR system by using sub-pixel warping and depth layer inpainting hole filling algorithms. The sub-pixel warping method can help to warp the pixels to the precise fractional positions to avoid the rounding errors of fractional disparity. The suggested sub-pixel warping method can increase warping performance with fractional disparity accuracy. In the proposed hole filling process, we separate the warped view into several depth layers and fill the missing pixels with the best patches of known texture information by using the inpainting concept. The proposed hole filling method can avoid the foreground objects propagating to the holes and achieve better performances than the traditional background extension method. Simulations shows that the proposed DIBR system attains more natural and reasonable synthetic virtual views than the existed DIBR methods.

[1]I. Sexton and P. Surman, "Stereoscopic and autostereoscopic display systems," IEEE Signal Processing Magazine, vol. 16, no. 3, pp. 85-99, 1999.
[2]S. Kejian and W. Fei, "The development of stereoscopic display technology," in 2010 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), 2010, vol. 4: IEEE, pp. V4-276-V4-280.
[3]C. Fehn, K. Hopf, and B. Quante, "Key technologies for an advanced 3D TV system," in Three-Dimensional TV, Video, and Display III, 2004, vol. 5599: International Society for Optics and Photonics, pp. 66-81.
[4]C. Fehn, R. De La Barré, and S. Pastoor, "Interactive 3-DTV-concepts and key technologies," Proceedings of the IEEE, vol. 94, no. 3, pp. 524-538, 2006.
[5]W.-Y. Chen, Y.-L. Chang, S.-F. Lin, L.-F. Ding, and L.-G. Chen, "Efficient depth image based rendering with edge dependent depth filter and interpolation," in 2005 IEEE International Conference on Multimedia and Expo, 2005: IEEE, pp. 1314-1317.
[6]C.-H. Hsia, "Improved depth image-based rendering using an adaptive compensation method on an autostereoscopic 3-D display for a Kinect sensor," IEEE Sensors Journal, vol. 15, no. 2, pp. 994-1002, 2015.
[7]W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, "Smoothing depth maps for improved steroscopic image quality," in Three-Dimensional TV, Video, and Display III, 2004, vol. 5599: International Society for Optics and Photonics, pp. 162-173.
[8]L. Zhang and W. J. Tam, "Stereoscopic image generation based on depth images for 3D TV," IEEE Transactions on broadcasting, vol. 51, no. 2, pp. 191-199, 2005.
[9]Y.-m. Feng, D.-X. Li, K. Luo, and M. Zhang, "Asymmetric bidirectional view synthesis for free viewpoint and three-dimensional video," IEEE Transactions on consumer electronics, vol. 55, no. 4, pp. 2349-2355, 2009.
[10]Z.-w. Liu, P. An, S.-x. Liu, and Z.-y. Zhang, "Arbitrary view generation based on DIBR," in 2007 International Symposium on Intelligent Signal Processing and Communication Systems, 2007: IEEE, pp. 168-171.
[11]W. R. Mark, L. McMillan, and G. Bishop, "Post-rendering 3d warping," SI3D, vol. 97, pp. 7-16, 1997.
[12]Y. Mao, G. Cheung, and Y. Ji, "Graph-based interpolation for dibr-synthesized images with nonlocal means," in 2013 IEEE Global Conference on Signal and Information Processing, 2013: IEEE, pp. 451-454.
[13]Y. Mao, G. Cheung, A. Ortega, and Y. Ji, "Expansion hole filling in depth-image-based rendering using graph-based interpolation," in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013: IEEE, pp. 1859-1863.
[14]L.-M. Po, S. Zhang, X. Xu, and Y. Zhu, "A new multidirectional extrapolation hole-filling method for depth-image-based rendering," in 2011 18th IEEE International Conference on Image Processing, 2011: IEEE, pp. 2589-2592.
[15]C. Vázquez, W. J. Tam, and F. Speranza, "Stereoscopic imaging: filling disoccluded areas in depth image-based rendering," in Three-Dimensional TV, Video, and Display V, 2006, vol. 6392: International Society for Optics and Photonics, p. 63920D.
[16]C.-M. Cheng, S.-J. Lin, S.-H. Lai, and J.-C. Yang, "Improved novel view synthesis from depth image with large baseline," in 2008 19th International Conference on Pattern Recognition, 2008: IEEE, pp. 1-4.
[17]K.-J. Oh, S. Yea, and Y.-S. Ho, "Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3-d video," in 2009 Picture Coding Symposium, 2009: IEEE, pp. 1-4.
[18]I. Daribo and B. Pesquet-Popescu, "Depth-aided image inpainting for novel view synthesis," in 2010 IEEE International Workshop on Multimedia Signal Processing, 2010: IEEE, pp. 167-170.
[19]A. Criminisi, P. Pérez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Transactions on image processing, vol. 13, no. 9, pp. 1200-1212, 2004.
[20]C. Yao, T. Tillo, Y. Zhao, J. Xiao, H. Bai, and C. Lin, "Depth map driven hole filling algorithm exploiting temporal correlation information," IEEE Transactions on Broadcasting, vol. 60, no. 2, pp. 394-404, 2014.
[21]H. K. Tseng, "A Patch-matched Hole Filling Algorithm for Multiview Synthesis and its VLSI Implementation," Master of Science, Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan, R.O.C., 2015.
[22]K. J. Hsu, "GPU Implementation for Centralized Texture Depth Depacking and Depth Image-based Rendering," Master of Science, Institute of Computer and Communication Engineering National Cheng Kung University, Tainan, Taiwan, R.O.C., 2017.
[23]K.-T. Lee, "Depth Map Preprocessing Based on Inflection of Gradient for Virtual View Synthesis," presented at the 3DSA IMID, 2017.
[24]劉楷哲;吳其霖, "基於3D顯示器格式之即時3D內容合成技術," 影像與識別, 電子與光電研究所, 工業技術研究院, vol. 16 no. 2, 2010.
[25]A. I. Purica, E. G. Mora, B. Pesquet-Popescu, M. Cagnazzo, and B. Ionescu, "Improved view synthesis by motion warping and temporal hole filling," in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015: IEEE, pp. 1191-1195.
[26]A. Oliveira, G. Fickel, M. Walter, and C. Jung, "Selective hole-filling for depth-image based rendering," in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015: IEEE, pp. 1186-1190.
[27]Y. Mori, N. Fukushima, T. Yendo, T. Fujii, and M. Tanimoto, "View generation with 3D warping using depth information for FTV," Signal Processing: Image Communication, vol. 24, no. 1-2, pp. 65-72, 2009.
[28]P. Ndjiki-Nya et al., "Depth image-based rendering with advanced texture synthesis for 3-D video," IEEE Transactions on Multimedia, vol. 13, no. 3, pp. 453-465, 2011.
[29]J.-H. Cho, W. Song, H. Choi, and T. Kim, "Hole filling method for depth image based rendering based on boundary decision," IEEE Signal Processing Letters, vol. 24, no. 3, pp. 329-333, 2017.
[30]S. Huq, A. Koschan, and M. Abidi, "Occlusion filling in stereo: Theory and experiments," Computer Vision and Image Understanding, vol. 117, no. 6, pp. 688-704, 2013.
[31]Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, vol. 13, no. 4, pp. 600-612, 2004.
[32]Middlebury Stereo Vision [Online] Available: http://vision.middlebury.edu/stereo/