基於較有效率的2D影像搭配其對應之深度的3D影像格式，深度影像繪圖法(DIBR)是一種常被使用搭配撥放3D影像的方式。一般的深度平滑演算法用來減少因深度影像繪圖而產生的空洞大小，但此種作法經常伴隨著深度品質的破壞。有一些基於空間相似性做法例如填充、鏡射、以及平均濾波器被用來填補影像空洞處。但這些基礎的做法時常伴隨著影像失真的產生。
本篇論文針對深度影像繪圖系統提出了一個方向適應性深度平滑及結構基礎空洞填補演算法及其架構的設計。本論文提出的深度平滑演算法是基於物件邊緣局部性的方向來選擇不同的平滑濾波器應用在影像空洞可能產生的地方，對於沒有影像空洞產生處則維持原始深度資訊來避免嚴重的影像失真。而所提出的結構基礎空洞填補法同時使用了時間以及空間上的資訊，透過一個參考結構的建立以及內插補點法來填補影像空洞資訊。從主觀及客觀的實驗結果比較證實所提出之演算法可以產生較高品質的虛擬影像。
本篇論文也示範了一個基於由上而下的架構設計流程以及設計空間探索方法。我們探討了演算法計算量、記憶體使用量、時脈速度、以及頻寬大小。根據上述數據選擇了每次處理8x1個點當作單次處理長度，由此選擇產生的設計能夠有較低的頻寬以及可接受的記憶體大小及時脈速度。此架構的生產量在時脈108 百萬赫茲下足夠支援規格1920x1080的視訊，每秒顯示30張圖的3D影片即時撥放。

Due to efficiency of video plus depth representation of the 3D video format, depth-image-based rendering (DIBR) is a common approach used by products to display 3D content. General depth smoothing can be used to reduce the hole size, but it is usually accompanied by a reduction in depth quality. Some simple methods like padding, mirroring, and averaging have been used for hole filling based on spatial consistency. However, this method also is accompanied by serious image distortion.
This thesis presents an adaptive directional depth smoothing and structure based hole filling for using a DIBR system algorithm and its hardware implementation. The proposed depth map smoothing algorithm is based on the edge direction in local region adaptively smooth sharp depth transition region, and preserves the depth quality at non-hole regions to prevent serious image distortion. The proposed structure based hole filling method uses both spatial and temporal information by building and interpolating a reference structure. Our experimental results show that the proposed algorithm creates a better visual quality for a generated virtual view.
This thesis also demonstrates a top-down design methodology with design space exploration. We analyzed the design conditions related to the number of operations, memory usage, clock rate, and bandwidth, and finally we selected a data granularity of 8x1, which uses a lower bandwidth with acceptable memory size and clock speed. The throughput supports the architecture to work at a clock speed of 108MHz for display 1920x1080@30fps 3D video sequence in real time.

[1] B. Javidi and F. Okano, Three-Dimensional Television, Video, and Display Technologies, Springer, Oct. 2002.
[2] nVidia, “nVidia 3D Vision Pro and Stereoscopic 3D,” Oct. 2010. Available: http://www.nvidia.pl/docs/IO/40505/WP-05482-001_v01-final.pdf
[3] A. Smolic, K. Mueller, P. Merkle, P. Kauff, and T. Wiegand, “An Overview of Available and Emerging 3D Video Formats and Depth Enhanced Stereo as Efficient Generic Solution,” IEEE Conf. on Picture Coding Symposium (PCS), pp. 1-4, 2009.
[4] C. Fehn, “Depth-Image-Based Rendering (DIBR), Compression and Transmission for a New Approach on 3D-TV,” in Proceeding of SPIE Stereoscopic Displays and Virtual Reality System XI, San Jose, CA, USA, vol. 5291, pp. 93-104, January 2004.
[5] C. Fehn, K. Hopf, and Q. Quante, “Key Technologies for an Advanced 3D-TV System,” in Proceedings of SPIE Three-Dimensional TV Video and Display III, pp. 66-80, Oct. 2004.
[6] J.-X. Chai, S. C. Chan, H.-Y. Shum, and X. Tong, “Plenoptic Sampling,” in Proceedings of ACM SIG-GRAPH ’00, pp. 307–318, (New Orleans, LA, USA), July 2000.
[7] T. Docherty, R. Koch, and A. Woods, “Image Distortions in Stereo-Scopic Video Systems,” in Proc. SPIE Stereoscopic Displays and Applications, San Jose, CA, pp. 36-48, Feb. 1993.
[8] C. Fehn, “A 3D-TV Approach Using Depth-Image-Based Rendering (DIBR),” Proc. of VIIP 03, Benalmadena, Spain, Sept. 2003.
[9] J. Tam, L. Zhang, and D. Wang, "Stereoscopic Image Generation Based on Depth Images," in IEEE Int. Conf. on Image Processing (ICIP), Singapore, pp.2993-2996, 2004.
[10] N. Hur, K. Jung, J. K. Kim, J. Kim, Y. Oh, S. Lee, H. Lee, G. Lee, Y. K. Park, and K. Yun, “Depth-Image-Based Rendering for 3DTV Service Over T-DMB,” in Signal Processing: Image Communication, Vol. 24, pp. 122-136, Jan. 2009.
[11] S. Lin, S. Lai, and J. Yang, “Improved Novel View Synthesis from Depth Image with Large Baseline,” in Pattern Recognition, ICPR 9th, pp. 8-11 Dec. 2008.
[12] P. Kauff, M. Mueller, and F. Zilly "Adaptive cross-trilateral median filter," in Proc. IEEE 3DTV Conf.: True Vis. Capture, Transmiss. Display 3D Video, pp. 1-4 2010.
[13] L. G. Chen, W. Y. Chen, Y. L. Chang, L. F. Ding, and S. F. Lin, “Efficient Depth Image Based Rendering with Edge Dependent Depth Filter and Interpolation,” in Proc. IEEE Int. Conf. Multimedia and Expo, Jul. 2005, pp. 1314-1317.
[14] P. J. Lee, and Effendi, "Nongeometric Distortion Smoothing Approach for Depth Map Preprocessing" in IEEE Transitions on Multimedia, Vol. 13, Iss. 2, pp. 246-254, April 2011.
[15] T. S. Chang, Y. R. Horng, and Y. C. Tseng, "Stereoscopic images generation with directional Gaussian filter," in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2650-2653, May 2010.
[16] Y. Chang, W. Guo, B. Liu, J. Wei, and S. Shi, "An Optimized Hardware Design for Stereoscopic Image Generation Based on Depth Image," in Circuits and Systems (MWSCAS), 2011 on IEEE 54th International Midwest Symposium, pp. 1-4, Aug. 2011.
[17] T. Doyle, “Interlaced to Sequential Conversion for EDTV Applications,” in Proc. 2nd International Workshop Signal Processing for HDTV, pp. 412-430, Feb. 1988.
[18] ISO/IEC JTC1/SC29/WG11, “Temporally Consistent Hole Filling for View Extrapolation,” in MEPG input document M21282, July 2011.
[19] F. H. Cheng, Y. W. Chang, and Y. S. Huang, “A Hardware Architecture for Real-Time Stereoscopic Image Generation from Depth Map,” in IEEE International Conference on Machine Learning and Cybernetics (ICMLC), 2011, vol. 4, pp. 1622-1627, July 2011.
[20] C. L. Chen, S. Y. Chien, and J. H. Lai, “Architecture Design and Analysis of Image-Based Rendering Engine,” in IEEE International Conference on Multimedia and Expo (ICME), 2011, pp. 1-6, July 2011.
[21] S. Shi, Y. Chang, W. Guo, B. Liu, and J. Wei, “An Optimized Hardware Design for Stereoscopic Image Generation Based on Depth Image,” in IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS), 2011, pp. 1-4, Aug. 2011.
[22] W. Y. Chen, Y. L. Chang, and L. G. Chen, “Real-time depth image based rendering hardware accelerator for advanced three dimensional television system,” in IEEE International Conference on Multimedia and Expo (ICME), 2006.