簡易檢索 / 詳目顯示

研究生: 林家漢
Lin, Chia-Han
論文名稱: 格網上基於分散式共享記憶體技術的區塊式資料共享機制之研究
A DSM-Based Block Level Data Sharing Mechanism on Grid
指導教授: 謝錫堃
Shieh, Ce-Kuen
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 電腦與通信工程研究所
Institute of Computer & Communication Engineering
論文出版年: 2007
畢業學年度: 95
語文別: 英文
論文頁數: 52
中文關鍵詞: 資料格網區塊式共享分散式共享記憶體
外文關鍵詞: block level sharing, DSM, Data Grid
相關次數: 點閱:62下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 在科學的研究上,會有許多科學儀器會產生出大量的資料,全世界的科學家可以透過格網的服務來取得、共享這些資料,他們透過資料格網的服務將這些在遠處的資料產生一份複本在本地端,通常都只用一個大檔案的一小部分,然而,整個檔案的複製需要讓這些科學家等待一段長時間的複製時間,並且在本地端要準備一塊非常大的磁碟空間來儲存這個檔案,再者當檔案有多份的複本的時候,如何在眾多複本中維護資料的一致性也是一種挑戰。在這篇論文中提出了「水龍頭」的區塊式共享機制,它可以減少在複製檔案時資料傳輸的時間並且避免浪費本地端磁碟的空間,它只會搬移使用者所需要的資料區塊而不是把整個檔案從遠端整個搬移到本地端,此外對於使用者而言,它利用了分散式共享記憶體的子系統來提供使用者所需要的資料和達到資料一致性。在實驗的結果中可以看到區塊式共享機制可以有效的節省傳輸資料的時間。

    A large amount of data is produced by scientific instrument and shared with world-wide scientific communities in the grid environment. Scientists could replicate a shared file through data grid service. They usually use only a small part of a large file.
    However, the file level replication burdens scientists on waiting for long transfer time and
    preparing a large size space on local disk. Furthermore, keeping the synchronization of
    the numerous of the replica is a challenge. In this thesis, a DSM-based block level sharing mechanism for Grid named SPIGOT is proposed. It can reduce the data transfer time and avoid wasting users’ local disk space. It makes users just get the necessary part of data instead of the whole file. Besides, it is transparent to users because the DSM
    subsystem of SPIGOT provides the necessary part of data and maintains the synchronization of the replicas. Experimental results show that SPIGOT successfully reduces the turnaround time of the data-intensive application which uses a small part of a large shared file.

    Chapter 1 Introduction ............................................. 1 1.1. Data Grid ........................................... 2 1.2. Motivation .......................................... 4 1.3. Goal ................................................ 5 Chapter 2 Related work and Background .................... 8 2.1. Related Work......................................... 8 2.2. DSM ................................................ 10 2.3. Teamster-G ......................................... 12 2.4. FUSE ............................................... 13 Chapter 3 System Overview.................................15 3.1. Architecture ....................................... 16 3.2. Data Location ...................................... 18 3.3. Scenario............................................ 20 Chapter 4 Implementation................................. 24 4.1. Modules Illustration ................................24 4.2. I/O Redirection..................................... 27 4.3. Block Level Sharing................................. 30 4.4. Dynamic Mapping..................................... 31 4.5. Mass Owner Transfer ................................ 33 Chapter 5 Performance Evaluation ........................ 37 5.1. Experiments of the read............................. 37 5.2. Overhead of the SPIGOT ............................. 42 5.3. Experiments of the write ........................... 44 5.4. Experiment of MassOwnerTransfer .................... 45 Chapter 6 Conclusion and Future Work .................... 48 Reference:............................................... 49

    [1]. A. Shoshani, A. Sim, J. Gu, “Storage Resource Managers: Middleware Components for Grid Storage”, Nineteenth IEEE Symposium on Mass Storage Systems, 2002 (MSS ’02)
    [2]. A. Shoshani, A. Sim, and J. Gu, “Storage Resource Managers: Essential Components for the Grid”, in Grid Resource Management: State of the Art and Future Trends, Edited by Jarek Nabrzyski, Jennifer M. Schopf, Jan weglarz, Kluwer Academic Publishers, 2003.
    [3]. A. Sim, J. Gu, A. Shoshani, V. Natarajan, “DataMover: Robust Terabyte-Scale Multi-file Replication over Wide-Area Networks”, 16th International Conference on Scientific and Statistical Database Management (SSDBM’04), 2004.
    [4]. A. Abbas. “Grid Computing: A Practical Guide to Technology and Applications”, Charles River Media 2004
    [5]. A Chervenak, R. Schuler, C. Kesselman, S. Koranda, B. Moe, “Wide Area Data Replication for Scientific Collaborations”, Proceedings of 6th IEEE/ACM International Workshop on Grid Computing (Grid2005), November 2005.
    [6]. B. Allcock, J. Bester, J. Bresnahan, AL. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, S. Tueke, “Data Management and Transfer in High-Performance Computational Grid Environments”, Parallel Computing 2002, 28: 749-771.
    [7]. C. Baru, R. Moore, A. Rajasekar, M. Wan, “The SDSC Storage Resource Broker”, Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research
    [8]. A. Chervenak, E. Deelman, I. Foster, L. Guy, W. Hoschek, A. Iamnitchi, C. Lesselman, P. Kunszt, M. Ripeanu, B. Schwartzkopf, H. Stockinger, K. Stockinger, B. Tierney, “Giggle: a framework for constructing scalable replica location service” ,Proceedings of the 2002 ACM/IEEE conference on Supercomputing.
    [9]. D. Dullmann, W. Hoschek, J. Jaen-Martinez, B. Segal, H. Stockinger, K. Stockinger, “ Models for Replica Synchronisation and Consistency in a Data Grid”, 10th IEEE International Symposium on High Performance Distributed Computing (HPDC-10 ’01) p.0067.
    [10]. D Thain , J Basney , SC Son , M. Livny, “The Kangaroo Approach to Data Movement on the Grid”, 10th IEEE International Symposium on High Performance Distributed Computing (HPDC-10 ’01) p.0325.
    [11]. D. Abramson, R. Buyya, and J. Giddy, “Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Broker”, Future Generation Computer Systems (FGCS) Journal, Volume 18, Issue 8, p.1061-1074, 2002.
    [12]. I. Foster, C. Kesselman, “Globus: A metacomputing infrastructure toolkit” International Journal of Supercomputer Applications 11(2): p.115, 28, 1997.
    [13]. J. Frey, T Tannenbaum, M. Livny, I. Foster, S. Tuecke, “Condor-G: A Computation Management Agent for Multi-Institutional Grids”, The Proceedings of 10th IEEE International Symposium on High Performance Distributed Computing, p.55-63, 2001
    [14]. A. Grimshaw, W. Wulf, “The Legion vision of worldwide virtual computer”, Communications of the ACM 1997; 40(1). 25. Avaki Corporation.
    [15]. L. Gong, “Project JXTA: A technology overview”, Sun Whitepaper, Auguest 2001.
    [16]. G. Antoniu, J.F. Deverge, S. Monnet, “How to bring together fault tolerance and data consistency to enable Grid data sharing”, in the Proceeding of the Workshop on Adaptive Grid Middleware (AGRIDM 2004).
    [17]. G. Antoniu, L. Bougé, and M. Jan, “JuxMem: An adaptive supportive platform for data sharing on the grid”, Scalable Computing: Practice and Experience, 6(3):45-55, Sept. 2005.
    [18]. G. Antoniu, M. Bertier, E. Caron, Frédéric Desprez, Luc Bougé, Mathieu Jan, Sébastien Monnet, and Pierre Sens. “GDS: an Architecture Proposal for a Grid Data-Sharing Service”, Research Report RR-5593, INRIA, IRISA, Rennes, France, June 2005.
    [19]. H. Casanova, G. Obertelli, F. Berman, R. Wolski, “The AppLeS Parameters Sweep Template: User-Level Middleware for the Grid”, Proceedings of Super Computing 00, 2000.
    [20]. I. Foster, C. Kesselman, S. Tuecke. “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, International Journal of High Performance Computing Applications, Vol. 15, No. 3, 200-222 (2001)
    [21]. J.B. Chang, T.Y. Liang, C.K. Shieh, “Teamster: A Transparent Distributed Shared memory for Clustered Symmetric Multiprocessors”, accepted for publication in the special issue of The Journal of Supercomputing, September 6, 2003.
    [22]. K. Li. “IVY: A shared virtual memory system for parallel computing”, In Proceeding of the 1988 International Conference on Parallel Processing (ICPP’88), pp. 94-101, 1988
    [23]. N.T. Karonis, B. Toonen, I. Foster, “MPICH-G2: A Grid-Enabled Implementation of the Message Passing Interface”, Journal of Parallel and Distributed Computing, Volume 63, Issue 5 , May 2003, Pages 551-563 Special Issue on Computational Grids
    [24]. P. Busetta, M. Carman, L. Serafini, F. Zini, “Query Optimisation – Use Case Working Document”, working document of the European Organization for Nuclear Research 9, April, 2001.
    [25]. P.C Chen, J.B. Chang, T.Y. Liang, C.K. Shieh, Y.C. Zhuang, “A Multi-Layer Resource Reconfiguration Framework for Grid Computing”, proceeding of the 4th international workshop on middleware for grid computing, MGC’06.
    [26]. R. Buyya and S. Venugopal, “The Gridbus Toolkit for Service Oriented Grid and Utility Computing: An Overview and Status Report”, Proceedings of the First IEEE International Workshop on Grid Economics and Business Model, p.19-36, 2004
    [27]. T.Y. Liang, C.Y. Wu, J.B. Chang, C.K. Shieh, “Teamster-G: a grid -enabled software DSM system” , Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid2005).
    [28]. V. Gera, “FUSE Kernel Operations Function Specifications”, Market Development Engineering Sun Microsystems, Inc, September 22, 2006.
    [29]. W. Hoschek, J. Jaen-Martinez, A. Samar, H. Stockinger, K. Stockinger. “Data Management in an International Data Grid Project”, Proceedings of the First IEEE/ACM International Workshop on Grid Computing, Bangalore, India, December, 2000.
    [30]. HPSS. High Performance Storage System, http://www.sdsc.edu/HPSS, San Diego Supercomputer Center, La Jolla, CA, 1997.
    [31]. W. Allcock, J. Bresnahan, R, Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, I, Foster, “The Clobus Striped GridFTP Framework and Server”, Proceeding of Super Computing 2005 (SC05), November 2005.

    無法下載圖示 校內:2057-08-30公開
    校外:2106-08-30公開
    電子論文尚未授權公開,紙本請查館藏目錄
    QR CODE