在深度影像繪圖中，特別是對於現在的多視角3D電視系統，補洞是決定結果品質的重要步驟之一。為了達到照片一般的效果，我們在這篇論文中提出了一種新型的補洞演算法。對比傳統利用洞周圍的資訊來補洞之方法，我們提出利用一張額外的參考影像來輔助修補的演算法。首先，我們會使用流光法和圖割優化理論來分割提取參考影像中的不同平面。接著通過加速穩健特徵（SURF）來找出各個平面與有洞的虛擬影像之間的對應關係，並利用這些關係來對這些平面進行轉換，然後將其填入虛擬影像的洞中。之後再進行後處理來減低不同曝光條件造成的差異。最後，我們會通過實驗來測試此方法，驗證提出的方法能得到比舊有補洞的方法更好的結果。

In the depth-image-based rendering (DIBR), hole-filling is one of the most important procedures for the quality of the DIBR results, especially in modern multi-view 3D-TV systems. To achieve photorealistic performance, in this thesis, we propose a novel inpainting algorithm for hole-filling. Rather than fill the holes by using neighboring pixels, the proposed method makes use of a reference image which contains the occluded information of the input image. The reference image is first segmented according to planes in the image by optical flow and graph cuts optimization. And a set of homographies between the new virtual image and all planar segments is estimated by SURF features. Then the segments are transformed by these homographies and filled into the holes of the virtual image. A post-processing procedure is conducted to reduce the exposure errors. The experimental results validate the proposed method which outperforms previous hole-filling methods.

[1] Cmglee, “Wikipedia.” [Online]. Available: http://en.wikipedia.org/wiki/File:Parallax_barrier_vs_lenticular_screen.svg.
[2] P. Kauff, N. Atzpadin, C. Fehn, M. Müller, O. Schreer, A. Smolic, and R. Tanger, “Depth Map Creation and Image-based Rendering for Advanced 3DTV Services Providing Interoperability and Scalability,” Signal Processing: Image Communication, vol. 22, no. 2, pp. 217–234, Feb. 2007.
[3] J. Ohm, K. Grüneberg, E. Hendriks, E. I. M, D. Kalivas, M. Karl, D. Papadimatos, and A. Redert, “A Realtime Hardware System for Stereoscopic Video Conferencing with Viewpoint Adaptation,” Signal Processing: Image Communication, 1998.
[4] C. Fehn, “Depth-image-based rendering (DIBR), Compression, and Transmission for a New approach on 3D-TV,” in Proceedings of SPIE 5291, Stereoscopic Displays and Virtual Reality Systems XI, 93, 2004, vol. 5291, pp. 93–104.
[5] Z. Tauber, Z. Li, and M. Drew, “Review and Preview: Disocclusion by Inpainting for Image-based Rendering,” Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE, vol. 37, no. 4, pp. 527–540, 2007.
[6] L. Zhang, W. Tam, and D. Wang, “Stereoscopic Image Generation Based on DepthImages,” in Proceedings of 2004 International Conference on Image Processing, 2004, vol. 5, pp. 2993–2996.
[7] L. Zhang and W. J. Tam, “Stereoscopic Image Generation Based on Depth Images for 3D TV,” IEEE Transactions on Broadcasting, vol. 51, no. 2, pp. 191–199, Jun. 2005.
[8] K. Jung, Y. Park, J. Kim, and H. Lee, “Depth Image Based Rendering for 3D Data Service Over T-DMB,” 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 2–5, 2008.
[9] S.-B. Lee and Y.-S. Ho, “Discontinuity-adaptive Depth Map Filtering for 3D View Generation,” in Proceedings of the 2nd International ICST Conference on Immersive Telecommunications, 2009.
[10] Y. Morvan and P. H. N. de With, “Quality Improving Techniques in DIBR for Free-viewpoint Video,” in Proceedings of 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, 2009, pp. 1–4.
[11] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image Inpainting,” in Proceedings of the 27th annual conference on Computer Graphics and Interactive Techniques - SIGGRAPH ’00, 2000, pp. 417–424.
[12] A. Criminisi, P. Perez, and K. Toyama, “Object Removal by Exemplar-based Inpainting,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, vol. 2, pp. II–721–II–728.
[13] L. Azzari, F. Battisti, and A. Gotchev, “Comparative analysis of occlusion-filling techniques in depth image-based rendering for 3D videos,” in Proceedings of the 3rd workshop on Mobile Video Delivery - MoViD ’10, 2010, p. 57.
[14] M. Richard and M. Chang, “Fast Digital Image Inpainting,” in Proceedings of the International Conference on Visualization, Imaging and Image Processing, 2001, pp. 106-107.
[15] K. Oh, Y. Sehoon, and Y. Ho, “Hole-filling Method Using Depth based In-painting for View Synthesis in Free Viewpoint Television (FTV) and 3D Video,” Picture Coding Symposium, 2009.
[16] L. Wang, H. Jin, R. Yang, and M. Gong, “Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images,” 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, Jun. 2008.
[17] A. Hervieu, N. Papadakis, A. Bugeau, P. Gargallo, and V. Caselles, “Stereoscopic Image Inpainting: Distinct Depth Maps and Images Inpainting,” in Proceedings of 20th International Conference on Pattern Recognition, 2010, pp. 4101–4104.
[18] M. Solh and G. AlRegib, “Hierarchical Hole-Filling For Depth-Based View Synthesis in FTV and 3D Video,” IEEE Journal of Selected Topics in Signal Processing, vol. 6, no. 5, pp. 495–504, Sep. 2012.
[19] A. Hartley, R.~I. and Zisserman, Multiple View Geometry in Computer Vision, Second. Cambridge University Press, ISBN: 0521540518, 2004.
[20] M. Fischler and R. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Communications of the ACM, vol. 24, no. 6, 1981.
[21] C. Harris and M. Stephens, “A Combined Corner and Edge Detector,” Procedings of the Alvey Vision Conference 1988, pp. 23.1–23.6, 1988.
[22] D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, Nov. 2004.
[23] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-Up Robust Features (SURF),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359, Jun. 2008.
[24] M. Muja and D. Lowe, “Fast Approximate Nearest Neighbors With Automatic Algorithm Configuration,” Proceedings of International Conference on Computer Vision Theory and Applications, 2009.
[25] D. Greig, B. Porteous, and A. Seheult, “Exact Maximum A Posteriori Estimation for Binary Images,” Journal of the Royal Statistical Society. Series B (Methodological), 1989.
[26] V. Kolmogorov and R. Zabih, “What Energy Functions Can Be Minimized Via Graph Cuts?,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147–59, Feb. 2004.
[27] Y. Boykov and M. Jolly, “Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images,” in Proceedings of Eighth IEEE International Conference on Computer Vision. ICCV 2001, 2001, vol. 1, no. July, pp. 105–112.
[28] C. Rother, V. Kolmogorov, and A. Blake, “Grabcut: Interactive Foreground Extraction using Iterated Graph Cuts,” ACM Transactions on Graphics (TOG), 2004.
[29] V. Lempitsky, P. Kohli, C. Rother, and T. Sharp, “Image Segmentation with A Bounding Box Prior,” in Proceedings of 2009 IEEE 12th International Conference on Computer Vision, 2009, pp. 277–284.
[30] Y. Boykov and V. Kolmogorov, “An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision.,” IEEE transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124–37, Sep. 2004.
[31] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High Accuracy Optical Flow Estimation Based on A Theory for Warping,” Proceedings of the 4th International Conference on Image Analysis and Recognition, vol. 4, no. May, pp. 25–36, 2004.
[32] C. Fehn, “A 3D-TV Approach Using Depth-image-based Rendering (DIBR),” Proceedings of Visualization, Imaging and Image Processing, 2003.
[33] L. Mcmillan, “An Image-based Approach To Three-dimensional Computer Graphics,” University of North Carolina, Chapel Hill, USA, 1997.
[34] P. Bhat, K. C. Zheng, N. Snavely, A. Agarwala, M. Agrawala, M. F. Cohen, and B. Curless, “Piecewise Image Registration in the Presence of Multiple Large Motions,” in Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR’06), 2006, vol. 2, pp. 2491–2497.
[35] J. Wills, S. Agarwal, and S. Belongie, “What Went Where,” in Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. I–37–I–44.
[36] R. C. Gonzalez, Digital Image Processing. Pearson Education, 2009.
[37] M. Granados, K. Kim, and J. Tompkin, “Background Inpainting for Videos with Dynamic Objects and A Free-moving Camera,” in Proceedings of the 12th European Conference on Computer Vision, 2012, vol. 7572, pp. 682–695.
[38] L.-M. Po, S. Zhang, X. Xu, and Y. Zhu, “A New Multidirectional Extrapolation Hole-filling Method for Depth-image-based Rendering,” in 2011 18th IEEE International Conference on Image Processing, 2011, pp. 2589–2592.
[39] A. Telea, “An ImageInpainting Technique Based on the Fast Marching Method,” Journal of Graphics Tools, vol. 9, no. 1, pp. 25–36, 2004.
[40] S. Kang, T. Chan, and S. Soatto, “Landmark Based Inpainting From Multiple Views,” UCLA Math CAM, 2002.
[41] M. Powell, A Thin Plate Spline Method for Mapping Curves into Curves in Two Dimensions. University of Cambridge, 1995.
[42] M. Brown and D. G. Lowe, “Automatic Panoramic Image Stitching using Invariant Features,” International Journal of Computer Vision, vol. 74, no. 1, pp. 59–73, Dec. 2006.