為了可以直接在三維立體顯示器上享有更舒適的立體視覺效果，本論文提出雙視角轉多視角立體影像生成系統；在此系統中，提出了深度圖融合前處理演算法以調整深度資訊，除了能夠減少其他視角影像生成時所產生的洞及後續影像邊界補洞之負擔，亦能提升整體立體視訊之影像品質。
為了提升深度圖的準確性，採用了參考可靠視差的機制，改善過去只選取最小成本視差的方法，同時利用雙方向檢測填補方法，改進過去左右檢測後只進行單一方向的填補。為了使三維立體電視的效果能夠更完善，在紋理變化巨大之處，提出了新的修補方法，並增加優化影像品質的中值濾波器、邊緣擴展及高斯模糊等方法。
本論文亦提出可在Alter FPGA的開發板上實現的硬體架構。使用的影片解析度為1920 × 1080，總共用了28k個邏輯元件、39k個暫存器及5.9 Mb的記憶體，最高操作頻率為132 MHz。

In this thesis, a two-view to multi-view conversion system, which lets users watch videos with naked-eyes 3D visual effects directly on the 3D displays, is proposed. To enhance the performance, a depth fusion preprocessing algorithm is proposed in this thesis.
A modified winner-takes-all with reliable disparity decision and a left-right check with two-way propagation approaches are adopted to improve the accuracy of depth map. Moreover, texture-based mismatch refinement, modified median filter, edge extension and Gaussian filter are utilized to have a better performance of the final results on 3D displays.
The corresponding hardware architecture of the proposed system is implemented in Altera FPGA. The design of the proposed system requires 28 k ALUTs, 39 k registers and 5.9 Mb RAM. The processing speed of the proposed architecture achieves 60 frames per second at maximum operating frequency 132 MHz with the resolution of 1920 × 1080.

[1] C. Fehn, K. Hopf, and B. Quante, “Key technologies for an advanced 3D TV system, ” Proc. SPIE, Three-dimentional TV, Video, and Display III, vol. 5599, pp. 66–80, Dec. 2004.
[2] C. Fehn, R. De La Barré, and S. Pastoor, “Interactive 3-DTV-concepts and key technologies, ” Proc. IEEE, vol. 94, pp. 524–538, Mar. 2006.
[3] D. Kim, D. Min, and K. Sohn, “A stereoscopic video generation method using stereoscopic display characterization and motion analysis,” IEEE Trans. Broadcast., vol. 54, pp. 188–197, June 2008.
[4] Y. Feng, J. Ren, and J. Jiang, “Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications,” IEEE Trans. Broadcast., vol. 57, pp. 500–509, June 2011.
[5] M. T. Pourazad, P. Nasiopoulos, and R. K. Ward, “An H.264-based scheme for 2D to 3D video conversion,” IEEE Trans. Consumer Electron., vol. 55, pp. 742–748, May 2009.
[6] M. C. Yang, J. Y. Liu, Y. C. Yang, and K. H. Chen, “A quality-improved stereo matching by using incrementally-averaging orthogonally-scanned edge detection,” in Proc. Int. Conf. 3D Syst. Appl. (3DSA), Hsinchu, Taiwan, June 2012, pp. 489–491.
[7] K. Zhang, J. Lu, and G. Lafruit, “Cross-based local stereo matching using orthogonal integral images,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, pp. 1073–1079, July 2009.
[8] C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV,” Proc. SPIE, Stereosc. Displays Virt. Reality Syst. XI, vol. 5291, pp. 93–104, Jan. 2004.
[9] I. P. Howard and B. J. Rogers. (1995, Nov.30), Binocular vision and stereopsis. (1st ed.) [online]. 2. Available DOI: 10.1093/acprof:oso/9780195084764.001.0001.
[10] B. Froba and A.Ernst, “Face detection with the modified census transform,” in Proc. IEEE Int.Conf. Automat. Face Gesture Recognition (FG), Seoul, Korea, June 2004, pp. 91–96.
[11] N. Y. C. Chang, T. H. Tsai, B. H. Hsu, Y. C. Chen, and T. S. Chang, “Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, pp. 792–805, June 2010.
[12] K. J. Yoon and I. S. Kweon, “Adaptive support-weight approach for correspondence search,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, pp. 650–656, Apr. 2006.
[13] C. Zhu and S. Li, “Multiple reference views for hole reduction in DIBR view synthesis,” in Proc. IEEE Int. Symp. Broadband Multimedia Syst., Broadcast. (BMSB), Beijing, China, June 2014, pp. 1–5.
[14] C. Zhu and S. Li, “A new perspective on hole generation and filling in DIBR based view synthesis,” in Proc. Int. Conf. Intell. Inform. Hiding and Multimedia Signal Process. (IIH-MSP), Beijing, China, Oct. 2014, pp. 607–610.
[15] C. Zhu and S. Li, “Depth image based view synthesis: new insights and perspectives on hole generation and filling,” IEEE Trans. Broadcast., vol. 62, pp. 82–93, Mar. 2016.
[16] M. Solh and G. AlRegib, “Hierarchical hole-filling (HHF): depth image based rendering without depth map filtering for 3D-TV,” in Proc. IEEE Int. Workshop Multimedia Signal Process. (MMSP), Saint-Malo, France, Oct. 2010, pp. 87–92.
[17] M. S. Ko, D. W. Kim, D. L. Jones, and J. Yoo, “A new common-hole filling algorithm for arbitrary view synthesis,” in Proc. Int. Conf. 3D Syst. Appl. (3DSA), Hsinchu, Taiwan, June 2012, pp. 242–245.
[18] I. Daribo and H. Saito, “A novel inpainting-based layered depth video for 3DTV,” IEEE Trans. Broadcast., vol. 57, pp. 533–541, June 2011.
[19] P. Ndjiki-Nya, M. Koppel, D. Doshkov, H. Lakshman, P. Merkle, K. Muller, and T. Wiegand, “Depth image-based rendering with advanced texture synthesis for 3-D video,” IEEE Trans. Multimedia, vol. 13, pp. 453–465, June 2011.
[20] A. Criminisi, P. Perez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Trans. Image Process., vol. 13, pp. 1200–1212, Sept. 2004.
[21] H. Q. Shan, W. D. Chien, H. M. Wang, and J. F. Yang, “A homography-based inpainting algorithm for effective depth image based rendering,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Paris, France, Oct. 2014, pp. 5412–5416.
[22] S. Reel, G. Cheung, P. Wong, and L. S. Dooley, “Joint texture-depth pixel inpainting of disocclusion holes in virtual view synthesis,” in Proc. Asia-Pacific Signal Inform. Process. Assoc. Annu. Summit Conf. (APSIPA ASC), Kaohsiung, Taiwan, Oct. 2013, pp. 1–7.
[23] P. C. Lee and E. Su, “Nongeometric distortion smoothing approach for depth map preprocessing,” IEEE Trans. Multimedia, vol. 13, pp. 246–254, Apr. 2011.
[24] L. Zhang and W. J. Tam, “Stereoscopic image generation based on depth images for 3DTV,” IEEE Trans. Broadcast., vol. 50, pp. 191–199, May 2005.
[25] C. Cheng, J. Liu, H. Yuan, X. H. Yang, and W. Liu, “A DIBR method based on inverse mapping and depth aided image inpainting,” in Proc. IEEE China Summit Int. Conf. Signal Inform. Process. (ChinaSIP), Beijing, China, July 2013, pp. 518–522.
[26] I. Ahn and C. Kim, “Depth-based disocclusion filling for virtual view synthesis,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Melbourne, Australia, July 2012, pp. 109–114.
[27] K. Luo, D. X. Li, Y. M. Feng, and M. Zhang, “Depth-aided inpainting for disocclusion restoration of multiview images using depth-image-based rendering,” J. Zhejiang University - Science A, vol. 10, pp. 1738–1749, Dec. 2009.
[28] L. Ma, L. Do, and P. With, “Depth-guided inpainting algorithm for free-viewpoint video,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Florida, USA., July 2012, pp. 1721–1724.
[29] O. L. Meur, J. Gautier, and C. Guillemot, “Examplar-based inpainting based on local geometry,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Brussels, Belgium, Sept. 2011, pp. 3401–3404.
[30] X. Sun, X. Mei, S. Jiao, M. Zhou, and H. Wang, “Stereo matching with reliable disparity propagation,” in Proc. 3D Imaging, Modeling, Process., Visual. Transmission (3DIMPVT), Hangzhou, China, May 2011, pp. 132–139.
[31] H. Wang, M. Wu, Y. Zhang, and L. Zhang, “Effective stereo matching using reliable points based graph cut,” in Proc. Visual Commun. and Image Process. (VCIP), Kuching, Malaysia, Nov. 2013, pp. 1–6.
[32] Y. Zhan, Y. Gu, K. Huang, C. Zhang, and K. Hu, “Accurate image-guided stereo matching with efficient matching cost and disparity refinement,” IEEE Trans. Circuits Syst. Video Technol., vol. 26, pp. 1632–1645, Sep. 2016.
[33] Middleburry Stereo Vision Page [Online]. Available: http://vision.middlebury.edu/stereo.
[34] H. Hirschmuller and D. Scharstein, “Evaluation of cost functions for stereo matching,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog.(CVPR), Minneapolis, USA, June 2007, pp. 1063–6919.
[35] D. Scharstein and R. Szeliski, “High-accuracy stereo depth maps using structured light,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2003, pp. 195–202.
[36] D. Scharstein and R. Szeliski, “High-accuracy stereo depth maps using structured light,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), Madison, WI, USA, June 2003, pp. 195–202.
[37] W. Wang, J. Yan, N. Xu, Y. Wang, and F.-H. Hsu, “Real-time high-quality stereo vision system in FPGA,” IEEE Trans. Circuits Syst. Video Technol., vol. 25, pp. 1696–1708, Jan. 2015.
[38] D. W. Yang, L. C. Chu, C. W. Chen, J. Wang, and M. D. Shieh, “Depth-reliability-based stereo matching algorithm and its VLSI architecture design,” IEEE Trans. Circuits Syst. Video Technol., vol. 25, pp. 1038–1050, June 2015.
[39] A. Aysu, M. Sayinta, and C. Cigla, “Low cost FPGA design and implementation of a stereo matching system for 3D-TV applications,” in Proc. IEEE Conf. Very Large Scale Integer. (VLSI-SoC), Istanbul, Turkey, Oct. 2013, pp. 204–209.
[40] P. L. Chu, "Stereo matching algorithm with fast disparity propagation under homogeneous texture detection and its VLSI implementation," M.S. thesis, National Cheng Kung University, Tainan, Taiwan, July 2012.
[41] C. C. Tien, “A fast coarse-to-fine stereo matching algorithm and its VLSI implementation,” M.S. thesis, National Cheng Kung University, Tainan, Taiwan, July 2013.
[42] H. W. Ho, “A fast local stereo matching algorithm with simple edge detection and its VLSI implementation,” M.S. thesis, National Cheng Kung University, Tainan, Taiwan, July 2014.
[43] K. L. Lo, “A real-time stereo matching algorithm with iterative aggregation and its VLSI implementation,” M.S. thesis, National Cheng Kung University, Tainan, Taiwan, July 2015.
[44] C. L. Chou, “Stereo matching with double refinements for centralized texture depth packing and their VLSI implementations,” M.S. thesis, National Cheng Kung University, Tainan, Taiwan, July 2016.
[45] K. J. Hsu, “GPU implementation for centralized texture depth depacking and depth image-based rendering,” M.S. thesis, National Cheng Kung University, Tainan, Taiwan, July 2017.
[46] H. K. Tseng, “A patch-matched hole filling algorithm for multiview synthesis and its VLSI implementation,” M.S. thesis, National Cheng Kung University, Tainan, Taiwan, July 2015.