3D影像或是影片是現在最接近人類生活的視覺科技，尤其是智慧型手機的盛行，讓人類對於追求視覺享受的慾望逐年提升，不論在電影產業,遊戲產業抑或是醫療需求都可以看見3D應用的蹤跡。近年來，由於戴眼鏡式3D顯示器的不方便以及內容的缺乏導致3D影像在應用上遇到瓶頸。因此，我們必須藉由基於深度影像生成渲染技術，透過平面的2D彩色影像以及其對應深度影像轉換為精緻的多視角3D影像。在深度影像生成渲染技術中，缺洞填補是生成3D影像的成敗關鍵，由於傳統缺洞填補演算法有耗費太多時間以及填補資訊不夠精確的缺點。所以，本論文提出透過深度學習闕漏資訊填補神經網路以合成高精確度的立體影像，且可加快生成系統的運行速度。利用搜尋周圍相似區塊填補缺洞與比較深度影像相似度的概念，有效應用在所提網路模型。實驗證明，本論文所提出基於深度圖多視角渲染法之缺洞填補神經網路，相較現行的方法，可以改善大幅精確度不足之問題，並且達到更好的立體影像之效果。

3D images or videos are the closest visual technology to human daily life. In particular, the prevalence of smart phones has increased human desire for pursuing visual enjoyment year by year. Whether in the movie industry, game industry or medical needs, 3D applications can be seen everywhere. In recent years, due to the inconvenience of wearing glasses and the lack of 3D content has led to the bottlenecks in the applications of 3D images. Therefore, through the depth image-based rendering (DIBR) algorithm, the multi-view 3D contents it can be produced from the 2D color image and its corresponding depth image. In the DIBR system, hole filling is the key to the success of generating 3D images. Due to the shortcomings of the traditional hole filling algorithms by taking too much time and exhibiting poor 3D quality, in this thesis, we propose a deep learning hole filling neural network to improve its applications. The proposed hole filling neural networks not only could synthesize high-precision 3D images but also speed-up the generation system where hole filling uses the concepts of comparing the depth similarity in the feature domain and searching the similar patches around the background regions. Experimental results prove that the proposed hole filling algorithm can improve the problem of insufficient accuracy and achieve a better 3D effect.

[1] P.B enzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey and C. V. Kopylow, “A survey of 3DTV displays: techniques and technologies. IEEE Trans. on Circuits and Systems for Video Technology, vol.17, no. 11, pp. 1647-1658, 2007.
[2] N. S. Holliman, N. A. Dodgson, G. E. Favalora and L. Pockett, “Three-dimensional displays: a review and applications analysis, “ IEEE Trans. on Broadcasting, vol.7, no. 2, pp.362-371, 2011.
[3] S. Kejian and W. Fei, "The development of stereoscopic display technology," in 2010 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), 2010, vol. 4: IEEE, pp. V4-276-V4-280.
[4] Fehn, C. “A 3D-TV approach using depth-image-based rendering (DIBR),” Proc. of VIIP , vol. 3, no. 3. , Sept. 2003
[5] M. Tanimoto, T. Fujii, K. Suzuki, N. Fukushima and Y. Mori, “Reference softwares for depth estimation and view synthesis” ISO/IEC JTC1/SC29/WG11 MPEG, 20081, M15377, 2008.
[6] Y. J. Jeong, Y. J. Jung and D. S. Park, “Depth image‐based rendering for multiview generation”, Journal of the Society for Information Display, vol.18, no. 4, pp.310-316, 2010.
[7] L. C. Tran, C. J. Pal and T. Q. Nguyen, “View synthesis based on conditional random fields and graph cuts,” Proc. of IEEE International Conference on Image Processing, pp. 433-436, September 2010.
[8] P. Ndjiki-Nya, M. Koppel, D. Doshkov, H. Lakshman, P. Merkle, K. Muller and T. Wiegand, “Depth image-based rendering with advanced texture synthesis for 3D video”, IEEE Transactions on Multimedia, vol.13, no. 3, pp.453-465, 2011.
[9] C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3DTV,” Proc. of SPIE Stereoscopic Displays and Virtual Reality Systems, 5291, pp. 93–104, May 2004.
[10] K. Wegner, O. Stankiewicz, M. Tanimoto and M. Domanski, “Enhanced view synthesis reference software (VSRS) for free-viewpoint television”, ISO/IEC JTC1/SC29/WG11 MPEG2013 M31520, 2013.
[11] A. Criminisi, P. Pérez and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Trans. on Image Processing, vol.13, no.9, pp.1200-121, 2004.
[12] J. Habigt, K. Diepold, “Image completion for view synthesis using markov random fields and efficient belief propagation,” Proc. of IEEE International Conference on Image Processing, pp. 2131-2134, September 2013.
[13] I. Daribo and B. Pesquet-Popescu, “Depth-aided image inpainting for novel view synthesis,” Proc. of IEEE International Workshop on Multimedia Signal Processing, pp. 167-170, October 2010.
[14] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair and Y. Bengio, “Generative adversarial nets. Proc. of Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
[15] S. ,Iizuka,E. Simo-Serra, & H. Ishikawa. (2017). Globally and locally consistent image completion. ACM Transactions on Graphics (ToG), 36(4), 107..
[16] J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, and W. Liu, “Deep high-resolution representation learning for visual recognition,” arXiv preprint arXiv:1908.07919, 2019.
[17] J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. S. Huang "Free-form image inpainting with gated convolution," Proc. of the IEEE International Conference on Computer Vision, 2019.
[18] D.Simakov, Y. Caspi, E. Shechtman and M. Irani, “Summarizing visual data using bidirectional similarity,” Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[19] C. Barnes, E. Shechtman, A. Finkelstein and D. B. Goldman, “PatchMatch: A randomized correspondence algorithm for structural image editing,” Proc. of ACM Trans. on Graphics vol. 28, no. 3, p. 24, July 2009.
[20] D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell and A. A. Efros, “Context encoders: Feature learning by inpainting,” Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536-2544, 2016.
[21] J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu and T. S. Huang, “Generative image inpainting with contextual attention,” Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 5505-5514, 2018.
[22] Y. C. Li, Precision Depth Image-based Rendering by Sub-pixel Warping and Depth Layer Inpainting Hole Filling, M. S. Thesis, National Cheng Kung University, Tainan, Taiwan, July (2019)