簡易檢索 / 詳目顯示

研究生: 吳政岳
Wu, Cheng-Yueh
論文名稱: 基於GPU加速可支援多樣3D顯示器之多功能3D多媒體播放器
Multifunctional 3D Media Player Based on GPU Acceleration for Versatile 3D Displays
指導教授: 楊家輝
Yang, Jar-Ferr
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 電腦與通信工程研究所
Institute of Computer & Communication Engineering
論文出版年: 2015
畢業學年度: 103
語文別: 英文
論文頁數: 105
中文關鍵詞: 深度繪圖多視角立體影像三維多媒體播放器
外文關鍵詞: DIBR, multi-view stereo image, 3D media player
相關次數: 點閱:87下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 近年來三維立體電影在全球形成一股多媒體風潮,伴隨著許多三維影像分享服務的興起,但是普遍並不支援裸眼立體影像的播放。所以我們的目標是建立一個三維多媒體播放器,可以支援多種立體效果和格式。三維多媒體播放器是基於Media Foundation,並且使用GPU去加速處理影像。本系統不只提供許多功能,還針對各種立體顯示器去合成多視角三維影像。與傳統的立體顯示系統相比,我們提供使用者更好的視覺體驗,整體的效能可在高解析度影像標準下達到即時的播放。此外藉由與自行開發MFT結合的ActiveX控制項將系統嵌入網頁中,本三維網路播放器可以從網站取得影像串流,並播放三維影像。

    In recent years, the 3D movies in the global entertainment market have become an important trend in multimedia application. The 3D video sharing service is growing up with YouTube. However, they do not support the naked-eye 3D displays. The goal of this thesis is to build a 3D media player to support several 3D formats in most 3D displays. This 3D media player is based on the Media Foundation and uses GPU to accelerate the 3D rendering process. Our system not only provides many functions of a media player but also synthesizes multi-view 3D images to satisfy a variety of stereo displays. Compared to the traditional stereo systems, our 3D media player can provide the users a revolutionary viewing experiences. The performance can reach the real time requirement for full-high-definition displays. Furthermore, we design an ActiveX controller, which is combined with the proposed MFT, to embed our system into 3D websites. Thus, the proposed 3D web player can load video streaming from web servers and play 3D videos.

    摘 要 I Abstract II 誌 謝 III Contents IV List of Tables VII List of Figures VIII Chapter 1 Introduction 1 1.1 Background 1 1.1.1 Binocular Vision 2 1.1.2 Categorization of 3D Displays 5 1.1.3 Information of Depth Map 10 1.1.4 Transmission Packing Formats of 3D Video 11 1.1.5 Overview of Media Player 13 1.2 Motivations 14 1.3 The Organization of Thesis 15 Chapter 2 Fundamentals 16 2.1 Depth Image-based Rendering (DIBR) 16 2.2 Template-based Adaptive Multi-view Rendering (TAMR) 21 2.3 High Efficient GPU-based Multi-view Rendering (HEMR) 24 2.4 General Purpose GPU 27 2.4.1 NVIDIA’s Compute Unified Device Architecture (CUDA) 28 2.5 Media Foundation 29 2.5.1 Component Object Model 30 2.5.2 Dynamic-link Library 31 2.5.3 Media Foundation System Architecture 32 2.5.4 Media Foundation Pipeline 33 Chapter 3 The Proposed 3D Media Player 35 3.1 Overview of the Proposed 3D Media Player 35 3.2 3D Conversion Media Foundation Transform 38 3.2.1 A Basic Media Player 38 3.2.2 Overview of Custom Media Foundation Transform (MFT) 40 3.2.3 A Basic Media Foundation Transform (MFT) 41 3.2.4 3D Conversion Media Foundation Transform (MFT) 46 3.3 Multifunctional 3D Visual System 47 3.3.1 Overview of Multifunctional 3D Visual System 47 3.3.2 Two-view Rendering Algorithm 48 3.3.3 Partial 3D Effect 50 3.4 Integrated GPU-based Multi-view Rendering System 51 3.4.1 Overview of the Integrated GPU-based Multi-view Rendering System 52 3.4.2 Template Information of 3D Display 53 3.4.3 Depacking Process 55 3.4.4 Depth Preprocessing 63 3.4.5 Comfortable Adjustment 72 3.4.6 Integrated GPU-based Multi-view Rendering Algorithm 73 3.5 Implementation of 3D Media Player 78 3.5.1 User Interface 78 3.5.2 ActiveX Controls 82 Chapter 4 Experimental Results 85 Chapter 5 Conclusions and Future Work 99 References 101

    [1] M. Lambooij, W. A. IJsselsteijn, M. Fortuin, and I. Heynderickx, “Visual discomfort and visual fatigue of stereoscopic displays: a review,” Journal of Imaging Science and Technology, vol. 53, no. 3, pp. 1–14, 2009.
    [2] L. Lipton, StereoGraphics Developers’ Handbook, San Rafael, CA, StereoGraphics Corporation,1997.
    [3] Y.S. Izmantoko, A.B. Suksmono, and T.L. Mengko, “Implementation of anaglyph method for stereo microscope image display,” International Conference on Electrical Engineering and Informatics, pp. 1–6, Jul. 2011.
    [4] S. Kejian, and W. Fei, “The development of stereoscopic display technology,” 2010 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), vol.4, pp. 276–280, Aug. 2010.
    [5] H. J. Lee, H. Nam, and J. D. Lee, “A high resolution autostereoscopic display employing a time division parallax barrier,” in SID Symposium Digest of Technical Papers, vol. 37, pp. 81–84, Jun. 2006.
    [6] K. Muller, “3-D video representation using depth maps,” Proceedings of the IEEE, vol. 99, no. 4, pp. 643–656, Apr. 2011.
    [7] A. Smolic, K. Mueller and P. Merkle, “3D video and free viewpoint video - technologies, applications and mpeg standards,” IEEE International Conference on Multimedia and Expo, pp. 2161–2164, Jul. 2006.
    [8] C. Fehn, “Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV,” Proc. SPIE 5291, Stereoscopic Displays and Virtual Reality Systems XI, May 2004.
    [9] W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, “Smoothing depth maps for improved stereoscopic image quality,” Proc. of SPIE, vol. 5599, pp.162–172, Oct. 2004.
    [10] L. Zhang and W. J. Tam, “Stereoscopic image generation based on depth images for 3DTV,” IEEE Trans. Broadcasting, vol. 51, no. 2, pp. 191–199, Jun. 2005.
    [11] L.H. Wang and X.J. Huang, “An asymmetric edge adaptive filter for depth generation and hole filling in 3DTV,” IEEE Transactions on Broadcasting, vol. 56, no. 3, pp. 425–431, Sep. 2010.
    [12] Pei-Jun Lee and Effendi, “Nongeometric distortion smoothing approach for depth map preprocessing,” IEEE Transactions on Multimedia, vol. 13, no. 2, pp. 246–254, Apr. 2011.
    [13] 劉楷哲,吳其霖, “基於3D顯示器格式之即時3D內容合成技術,” 工業技術研究院 電子與光電研究所,影像與識別, vol. 16 no. 2, 2010.
    [14] Y.M. Feng, DX Li, K Luo and M Zhang, “Asymmetric bidirectional view synthesis for free viewpoint and three-dimensional video,” IEEE Transactions on Consumer Electronics, vol.55, no.4, pp.2349–2355, Nov. 2009.
    [15] W.K. Giloi, J.L. Encarnação, and W. Straßer, “The Giloi’s school of computer graphics,” Computer Graphics, vol. 35, no. 4, pp. 12–16, Nov. 2001.
    [16] S. Il Lee, Y. J. Jung, H. Sohn, F. Speranza, and Y. M. Ro, “Effect of stimulus width on the perceived visual discomfort in viewing stereoscopic 3DTV,” IEEE Transaction on Broadcasting., vol. 59, no. 4, pp. 580–590, Dec. 2013.
    [17] L. Zhang, W. J. Tam, and D. Wang, “Stereoscopic image generation based on depth images,” IEEE International Conference on Image Processing, vol. 5, pp. 2993–2996, Oct. 2004.
    [18] L. Zhang, and W. J. Tam, “Stereoscopic image generation based on depth images for 3D TV,” IEEE Transaction on Broadcasting, vol. 51, no. 2, pp. 191–199, Jun. 2005.
    [19] L. Azzari, F. Battisti, and A. Gotchev, “Comparative analysis of occlusion-filling techniques in depth image-based rendering for 3D videos,” 3rd Workshop Mobile Video Delivery, pp.57–62, 2010.
    [20] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” ACM SIGGRAPH, Intern. Conf. on Computer Graphics and Interactive Techniques, pp. 417–424, Jul. 2000.
    [21] C. Vázquez, W. J. Tam, and F. Speranza, “Stereoscopic imaging: Filling disoccluded areas in depth image-based rendering,” in Proc.Soc. Photo-Opt. Instrum. Eng. (SPIE) Conf. Series, vol. 6392, Oct. 2006.
    [22] Lai-Man Po, and Shihang Zhang, “A new multidirectional extrapolation hole-filling method for depth-image-based rendering,” IEEE International Conference on Image Processing (ICIP), pp. 2589–2592, Sep. 2011.
    [23] Hung-Wei Tsai, “Template-based adaptive multi-view rendering for versatile 3D displays with GPU realization,” Master Thesis, National Cheng Kung University, July. 2013.
    [24] Yu-Hang Chen, “A high efficient GPU-based multi-view rendering for ultra high definition 3D displays,” Master Thesis, National Cheng Kung University, June. 2014.
    [25] NVIDIA DEVELOPER ZONE: CUDA C Programming Guide http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#axzz34Vc9Eacv
    [26] S. Izadi, D. Kim, O. Hilliges, and D. Molyneaux, “KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera,” Proceeding of the 24th annual ACM symposium on User interface software and technology, pp. 559–568, Oct. 2011.
    [27] S. Zinger, D. Ruijters, L. Do, and P. H. N. de With, “View interpolation for medical images on autostereoscopic displays,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 1, pp. 128–137, Jan. 2012.
    [28] M. Ren, S. Zhang, Y. Lei, and M. Zhang, “CUDA-based real-time face recognition system,” IEEE International Conference on Digital Information and Communication Technology and its Application (DICTAP), pp. 237–241, May. 2014.
    [29] Jeong-Woo Lee, Bumho Kim and Ki-Song Yoon, “CUDA-based JPEG2000 encoding scheme,” IEEE International Conference on Advanced Communication Technology (ICACT), pp. 671–674, Feb. 2014.
    [30] Dale Rogerson, Inside COM, Microsoft Press, 1997.
    [31] Don Box, Essential COM, Addison-Wesley Professional, 1998.
    [32] Anton Polinger, and Anton Polinger, “Developing Microsoft Media Foundation Applications,” Microsoft Press, 2011.
    [33] Yin-Peng Huang, “Designs of 3D media player based on Microsoft Media Foundation,” Master Thesis, National Cheng Kung University, Jul. 2013
    [34] His-chu Tseng, “Efficient 3D video packing and comfort disparity modification for traditional broadcasting systems,” Master Thesis, National Cheng Kung University, Jul. 2013.
    [35] J.-F. Yang, K.-Y. Liao, H.-M. Wang and C.-Y. Chen, “Centralized Texture-Depth Packing (CTDP) SEI Message,” Joint Collaborative Team on 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document no. JCT3V-K0027, Geneva, Feb. 2015.
    [36] Lanczos C., “An iteration method for the solution of the eigenvalue problem of linear differential and integral operators,” Journal of Research of the National Bureau of Standards, vol. 45, pp. 255–282, 1950.
    [37] Deza E., and Deza M.M., Dictionary of Distances, Elsevier Science, 2006
    [38] Sung-Hyuk Cha, “Comprehensive survey on distance/similarity measures between probability density functions,” International Journal of Mathematical Models and Methods in Applied Sciences, vol. 1, no. 4, 2007.
    [39] Jeff Prosise, Programming Windows with MFC, Microsoft Press, 1999.
    [40] Charles Petzold, Programming Windows, Microsoft Press, 1998.
    [41] Microsoft Developer Network http://msdn.microsoft.com/en-US/
    [42] Adam Denning, ActiveX Controls Inside Out, Microsoft Press, 1997.
    [43] X. Lu, Y. Gao, L. Lei, X. Wang, and S. Guo, “ActiveX control development of virtual instrument based on COM technology,” 2010 Second International Workshop on Education Technology and Computer Science, vol. 1, pp. 513–516, Mar. 2010.

    無法下載圖示 校內:2020-08-05公開
    校外:不公開
    電子論文尚未授權公開,紙本請查館藏目錄
    QR CODE