隨著3D影像技術的趨勢以及人們對於3D視覺感官的追求，愈來愈多的3D顯示產品或解決方案推層出新。在多視角3D電視廣播系統中配合裸眼3D顯示，框兼容3D格式被設計作為傳輸媒介，優點為降低頻寬需求更能在現有的2D電視廣播系統中傳輸3D影像。影像置中深度包裝格式(CTDP)不只具備框兼容特性，也優化解包裝後的影像品質；基於深度影像生成技術(DIBR) 利用了光學投射原理，給定一張2D影像與其他對應之深度資訊，將可以生成出看似從虛擬視角觀看之影像。本論文提出之系統結合CTDP格式解包裝端與DIBR處理下，提出的三個改善方法，含基於CTDP格式之深度中值內插、基於變異數之深度圖上採樣、及多視角深成的雙徑補洞技術。實驗結果顯示，深度圖深度中值內插已獲得不錯結果，基於變異數圖上採樣將深度標記，可透過適應性三層濾波器加以強化。最後，經雙徑補洞以強化虛擬視角影像後，本論文能將CTDP影像格式簡單地解出舒適的3D視覺體驗。

With the trend of 3D video technology and the pursuit of 3D visual perception, more and more related 3D image products and solutions are released in the market. In the multi-views 3D TV broadcasting system, several frame-compatible 3D formats are designed to be the delivering 3D contents for naked-eye displays. It not only reduces the bandwidth requirement but also can be utilized in an existing 2D broadcasting channel. The centralized texture depth packing (CTDP) formats, which are frame-compatible 3D formats, can preserve better 3D quality. Depth image-based rendering (DIBR) based on the optic projection concept is a multi-views generation algorithm that takes a texture image frame and its associated depth map to synthesize images observed at the selected viewpoints. In this thesis, for the CTDP format depacker and its DIBR engine, we proposed three enhanced methods including depth central interpolation for depacking CTDP color depth, variance-based depth upsampling, and two-pass hole filling for DIBR process. Experimental results show that the depth central interpolation by using mid-average method achieve good quality. Variance-based depth upsampling improves the joint trilateral filter by adaptively applying the variance-based map. Finally, two-pass hole filling method can enhance the virtual view images. These researches achieve good 3D visual experiences depacked from CTDP video formats.

[1] K. C. Chen, "New RGB Color Depth and Upsampling Methods for Centralized Texture Depth Packing Formats," Master of Science, Institute of Computer and Communication Engineering National Cheng Kung University, Tainan, Taiwan, R.O.C., 2018.
[2] Keys, Robert. "Cubic convolution interpolation for digital image processing." IEEE transactions on acoustics, speech, and signal processing 29.6 (1981): 1153-1160.
[3] Mohler, Ronald R. Bilinear control processes: with applications to engineering, ecology and medicine. Academic Press, Inc., 1973.
[4] Angel, Edward, and Anil Jain. "A nearest neighbors approach to multidimensional filtering." Proceedings of the 1972 IEEE Conference on Decision and Control and 11th Symposium on Adaptive Processes. IEEE, 1972.
[5] Kopf, Johannes, et al. "Joint bilateral upsampling." ACM Transactions on Graphics (ToG). Vol. 26. No. 3. ACM, 2007.
[6] Garcia, Frederic, et al. "Real-time depth enhancement by fusion for RGB-D cameras." IET Computer Vision 7.5 (2013): 335-345.
[7] Song, Yibing, and Lijun Gong. "Analysis and improvement of joint bilateral upsampling for depth image super-resolution." 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP). IEEE, 2016.
[8] Tomasi, Carlo, and Roberto Manduchi. "Bilateral filtering for gray and color images." Iccv. Vol. 98. No. 1. 1998.
[9] Yang, Qingxiong, Kar-Han Tan, and Narendra Ahuja. "Real-time O (1) bilateral filtering." 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2009.
[10] Hong, Su-Min, and Yo-Sung Ho. "Depth map refinement using superpixel label information." 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). IEEE, 2016.
[11] Chang, Ting-An, Yang-Ting Chou, and Jar-Ferr Yang. "Robust depth enhancement and optimization based on advanced multilateral filters." EURASIP Journal on Advances in Signal Processing 2017.1 (2017): 51.
[12] K. H. Lee, "Multiview Synthesis Algorithms Based on Depth and Texture Consistency," Master of Science, Institute of Computer and Communication Engineering National Cheng Kung University, Tainan, Taiwan, R.O.C., 2011.
[13] Tam, Wa James, et al. "Smoothing depth maps for improved steroscopic image quality." Three-Dimensional TV, Video, and Display III. Vol. 5599. International Society for Optics and Photonics, 2004.
[14] Lee, Pei-Jun. "Nongeometric distortion smoothing approach for depth map preprocessing." IEEE Transactions on Multimedia 13.2 (2010): 246-254.
[15] Xu, Xuyuan, et al. "Depth map misalignment correction and dilation for DIBR view synthesis." Signal Processing: Image Communication 28.9 (2013): 1023-1045.
[16] Xu, Xuyuan, et al. "A foreground biased depth map refinement method for DIBR view synthesis." 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2012.
[17] Kuan-Ting Lee, Bin-Da Liu, and Jar-Ferr Yang. “Depth Map Preprocessing Based on Inflection of Gradient for Virtual View Synthesis,” 3DSA IMID 2017.
[18] K. J. Hsu, "GPU Implementation for Centralized Texture Depth Depacking and Depth Image-based Rendering," Master of Science, Institute of Computer and Communication Engineering National Cheng Kung University, Tainan, Taiwan, R.O.C., 2017.
[19] Criminisi, Antonio, Patrick Pérez, and Kentaro Toyama. "Region filling and object removal by exemplar-based image inpainting." IEEE Transactions on image processing 13.9 (2004): 1200-1212.
[20] Ma, Lingni, Luat Do, and Peter HN de With. "Depth-guided inpainting algorithm for free-viewpoint video." 2012 19th IEEE International Conference on Image Processing. IEEE, 2012.
[21] Lie, Wen-Nung, Chun-Cheng Yeh, and Guo-Shiang Lin. "Improving DIBR technique to resolve foreground color/depth edge misalignment." 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 2015.
[22] C. Y. Chen, "A New YCbCr Representation Design for Centralized Texture Depth Packing Formats," Master of Science, Institute of Computer and Communication Engineering National Cheng Kung University, Tainan, Taiwan, R.O.C., 2016.
[23] Scharr, Hanno. Optimal operators in digital image processing. Diss. 2000.
[24] Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods." IEEE transactions on pattern analysis and machine intelligence 34.11 (2012): 2274-2282.
[25] Wang, Zhou, et al. "Image quality assessment: from error visibility to structural similarity." IEEE transactions on image processing 13.4 (2004): 600-612.
[26] Otsu, Nobuyuki. "A threshold selection method from gray-level histograms." IEEE transactions on systems, man, and cybernetics 9.1 (1979): 62-66.