本論文提出一種用於3D彩圖置中景深包裝之雙優化立體匹配法的系統。傳統的深度後處理方法只進行了左右檢測，因此無法有效地檢測出所有的匹配錯誤，為了提升深度資訊的精確度，本論文提出一優化檢測法，藉此標記出大量的匹配錯誤，以提升深度資訊的準確性；此外，由於近年來最新提出的彩圖置中景深包裝格式目前尚未發展出與其相對應的硬體架構，因此，本論文針對彩圖置中景深包裝格式進行硬體架構的設計，同時將其與本文提出的雙優化立體匹配法之硬體架構進行結合。
在所提出的雙優化立體匹配法中，本論文首先改良普查式轉換的參考點選取方法，並採用新的參考點計算位元串之間的漢明距離作為初始匹配成本，再配合適用於硬體實現之適應性權重進行疊代式成本聚合，最後選取最佳視差進而得到初始深度圖，在深度圖後處理的部分，本論文將邊緣及遭遮擋之區域使用雙優化後處理進行修補，並且獲得更精確之深度資訊。在彩圖置中景深包裝格式中，經分析後，本論文採用蘭克斯濾波器針對深度以及彩圖資訊進行影像縮小以及包裝的工作。
本論文同時設計了雙優化立體匹配演算法以及彩圖置中景深包裝解包裝之硬體架構，並以Altera之 FPGA實現；在雙優化立體匹配之硬體架構中共需約27k個邏輯單元、30k個暫存器及約4.8Mb的記憶體儲存量，時脈最快可達160 MHz，提供解析度1920 × 1080、每秒60張且深度搜尋範圍為64層之處理速度。在3D彩圖置中景深包裝解包裝之硬體架構則需5 776個邏輯單元、7 791個暫存器及約1.43Mb的記憶體儲存量，時脈最快可達166.56 MHz，於每秒60張1920 × 1080解析度之影像處理速度。

In this thesis, a stereo matching with double refinements which combines centralized texture depth packing (CTDP) system is proposed. In the traditional stereo matching algorithm, left-right check in the refinement process is unable to detect mismatching errors usefully. In order to raise the accuracy, texture-based detection is proposed in this thesis to flag mismatching errors for achieving high accuracy. In addition, the CTDP format which has no corresponding hardware implementation is proposed, recently. This thesis proposed the CTDP system combined with a stereo matching with double refinements.
The proposed stereo matching algorithm changes the reference pixels in the census transform. Then, computes initial matching costs by Hamming distances of bit-streams resulted with new selected reference pixels. In the cost aggregation, the proposed stereo matching algorithm, a hardware-friendly adaptive support weights are adopted to perform iterative aggregations for generating better disparities. Finally, double refinements are used to repair the error disparities in boundary and occluded regions. For the CTDP format, both texture and depth are down-scaled by Lanczos filter for packing with CTDP format.
The corresponding VLSI architecture of stereo matching and CTDP format are provided and realized in Altera FPGA. The design of stereo matching requires 27k logic elements, 30k registers and 4.8Mb RAM, and the process speed achieves 60 frames per second with the resolution of 1920 × 1080 and 64 disparity levels in 160 MHz. The design of CTDP architecture requires 5 776 logic elements, 7 791 registers and 1.43 Mb RAM, and the process speed achieves 60 frames per second with the resolution of 1920 × 1080.

[1] C. Fehn, K. Hopf, and B. Quante, "Key technologies for an advanced 3D TV system," Proc. SPIE, Three-dimentional TV, Video, and Display III, vol. 5599, pp. 66–80, Dec. 2004.
[2] C. Fehn, R. De La Barré, and S. Pastoor, "Interactive 3-DTV-concepts and key technologies," Proc. IEEE, vol. 94, pp. 524–538, Mar. 2006.
[3] J. F. Yang, H. M. Wang, and A. T. Chiang, "2D backwards compatible centralized color-depth packing," Joint Collaborative Team on 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, the 6th Meeting: Document: JCT3V-F0087, Geneva, Nov. 2013.
[4] A. F. Bobick and S. S. Intille, "Large occlusion stereo," Int. J. Comput. Vis., vol. 33, pp. 181–200, Sep. 1999.
[5] Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, pp. 1222–1239, Nov. 2001.
[6] P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient belief propagation for early vision," Int. J. Comput. Vis., vol. 70, pp. 41–54, May 2006.
[7] K. J. Yoon and I. S. Kweon, "Adaptive support-weight approach for correspondence search," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, pp. 650–656, Apr Apr. 2006.
[8] K. Ambrosch, M. Humenberger, W. Kubinger, and A. Steininger, "SAD-based stereo matching using FPGAs," in Proc. Conf. Embed. Comput. Vis., Dec. 2009, pp. 121–138.
[9] A. Darabiha, W. J. MacLean, and J. Rose, "Reconfigurable hardware implementation of a phase-correlation stereoalgorithm," Mach. Vision Appl., vol. 17, pp. 116–132, Apr. 2006.
[10] I. P. Howard and B. J. Rogers, Binocular vision and stereopsis: Oxford University Press, USA, 1995.
[11] N. Y.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang, "Algorithm and architecture of disparity estimation with mini-census adaptive support weight," IEEE Trans. Circuits Syst. Video Technol., vol. 20, pp. 792–805, Jun. 2010.
[12] T. Kanade and M. Okutomi, "A stereo matching algorithm with an adaptive window: Theory and experiment," IEEE Trans. Pattern Anal. Mach. Intell., vol. 16, pp. 920–932, Sep. 1994.
[13] K. Zhang, J. Lu, and G. Lafruit, "Cross-based local stereo matching using orthogonal integral images," IEEE Trans. Circuits Syst. Video Technol., vol. 19, pp. 1073–1079, Jul. 2009.
[14] O. Veksler, "Stereo correspondence with compact windows via minimum ratio cycle," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, pp. 1654–1660, Dec. 2002.
[15] Y. Boykov, O. Veksler, and R. Zabih, "A variable window approach to early vision," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, pp. 1283–1294, Dec. 1998.
[16] Y. Wei and L. Quan, "Region-based progressive stereo matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., Jun. 2004, pp. 106–113.
[17] X. Sun, X. Mei, S. Jiao, M. Zhou, and H. Wang, "Stereo matching with reliable disparity propagation," in Proc. IEEE Int. Conf. 3D Imaging, Model., Process., Vis. Transm., May 2011, pp. 132–139.
[18] P. C. Kuo, A. J. Lin, B. D. Liu, and J. F. Yang, "An advanced video and depth depacking architecture for 3D applications," J. Inform. Sci. Eng., vol. 31, pp. 1537–1555, Sep. 2015.
[19] K. L. Lo, "A real-time stereo matching algorithm with iterative aggregation and its VLSI implementation " M.S. thesis, National Cheng Kung University, Tainan, Taiwan, Jul. 2015.
[20] X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang, and X. Zhang, "On building an accurate stereo matching system on graphics hardware," in Proc. IEEE Conf. Comput. Vis. Workshops, Nov. 2011, pp. 467-474.
[21] W. Wang, J. Yan, N. Xu, Y. Wang, and F.-H. Hsu, "Real-time high-quality stereo vision system in FPGA," IEEE Trans. Circuits Syst. Video Technol., vol. 25, pp. 1696–1708, Jan. 2015.
[22] C. C. Pham and J. W. Jeon, "Domain transformation-based efficient cost aggregation for local stereo matching," IEEE Trans. Circuits Syst. Video Technol., vol. 23, pp. 1119–1130, Jul. 2013.
[23] W. S. Fife and J. K. Archibald, "Improved census transforms for resource-optimized stereo vision," IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, pp. 60–73, Jan. 2013.
[24] Q. Yang, D. Li, L. Wang, and M. Zhang, "Full-image guided filtering for fast stereo matching," IEEE Signal Process. Lett., vol. 20, pp. 237–240, Mar. 2013.
[25] A. S. Glassner, Graphic Gem. Cambrige, MA: Academic Press, 1990, pp. 147–165.
[26] S. E. Reichenbach and F. Geng, "Two-dimensional cubic convolution," IEEE Trans. Image Process., vol. 12, pp. 857–865, Aug. 2003.
[27] E. Angel and A. Jain, "A nearest neighbors approach to multidimensional filtering," in Proc. IEEE conf. Decision Contr. (ICDC), Louisiana, U.S.A., Dec. 1972, pp. 13–15.
[28] H. S. Hou and H. C. Andrews, "Cubic-splines for image interpolation and digital filtering," IEEE Trans. Acoust. Speech Signal Process., vol. 26, pp. 508–517, Dec. 1978.
[29] R. G. Keys, "Cubic convolution interpolation for digital image-processing," IEEE Trans. Acoust. Speech Signal Process., vol. 29, pp. 1153–1160, Dec. 1981.
[30] H. C. Andrews and C. L. Patterson, "Digital interpolation of discrete images," IEEE Trans. Comput., vol. 25, pp. 196–202, Feb. 1976.
[31] X. Li and M. T. Orchard, "New edge-directed interpolation," IEEE Trans. Image Process., vol. 10, pp. 1521–527, Oct. 2001.
[32] ITU-T SC16/Q6, "Calculation of average PSNR differences between RD-curves," Apr. 2001.
[33] Middleburry Stereo Vision Page [Online]. Available: http://vision.middlebury.edu/stereo
[34] H. Hirschmuller and D. Scharstein, "Evaluation of cost functions for stereo matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2007, pp. 1–8.
[35] D. Scharstein and C. Pal, "Learning conditional random fields for stereo," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2007, pp. 1–8.
[36] D. Scharstein and R. Szeliski, "High-accuracy stereo depth maps using structured light," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2003, pp. 195–202.
[37] D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Comput. Vis. , vol. 47, pp. 7–42, Apr. 2002.
[38] D.-W. Yang, L.-C. Chu, C.-W. Chen, J. Wang, and M.-D. Shieh, "Depth-reliability-based stereo matching algorithm and its VLSI architecture design," IEEE Trans. Circuits Syst. Video Technol., vol. 25, pp. 1038–1050, Jun. 2015.
[39] A. Aysu, M. Sayinta, and C. Cigla, "Low cost FPGA design and implementation of a stereo matching system for 3D-TV applications," in Proc. IEEE Conf. Very Large Scale Integer., Oct. 2013, pp. 204–209.
[40] P. L. Chu, "Stereo matching algorithm with fast disparity propagation under homogeneous texture detection and its VLSI implementation," M.S. thesis, National Cheng Kung University, Tainan, Taiwan, Jul. 2012.
[41] C. C. Tien, "A fast coarse-to-fine stereo matching algorithm and its VLSI implementation," M.S. thesis, National Cheng Kung University, Tainan, Taiwan, Jul. 2013.
[42] H. W. Ho, "A fast local stereo matching algorithm with simple edge detection and its VLSI implementation," M.S. thesis, National Cheng Kung University, Tainan, Taiwan, Jul. 2014.