雲端服務必須有良好的網路環境支援，所以網路I/O虛擬化對雲端服務來說是很重要的，為了在虛擬化環境中提供健全的網路品質，本論文歸納幾個要素: (1)網路I/O虛擬化架構，(2)服務品質保證和 (3)VMM排程器。我們利用兩種不同的架構來虛擬化網路卸載引擎。第一，我們利用VMM模擬出虛擬的網路卸載引擎，並將虛擬網路卸載引擎分別指定給不同的虛擬機器。但是模擬網路引擎會消耗多餘的處理器資源，為了避免這個問題，我們利用直接存取架構來增進網路效能。直接存取架構中的網路卸載引擎提供多個控制介面來讓虛擬機器存取，每個虛擬機器可以存取自己的控制介面。經由直接存取架構，即可消去模擬硬體所消耗的處理器資源。除了增進虛擬化環境中的網路效能外，VMM還必須保證網路服務質量。我們提出了一套分配機制，利用欠額輪替演算法來分配不同虛擬機器的網路流量。傳統的VMM的排程專注在公平地分配處理器資源給各個虛擬機器，然而這種排程方法可能會造成低落的I/O效能，因此我們分析處理器排程器和I/O的關係，並且找到合適的排程器演算法來得到合理的I/O效能並且處理器資源亦合理的被分配。為了要有效地驗證網路系統，我們使虛擬平台能與實際的網路連接，透過與真實網路連線的驗證方法，讓整個開發過程更有效率。

Network I/O virtualization plays an important role in the cloud computing. This thesis identifies three critical factors to provide a robust network service in a virtualization environment: I/O virtualization architectures, Quality of Service (QoS), and the VMM scheduler. In order to achieve high network throughput, we virtualize TOE in different ways. First, we use device emulation in which the VMM emulates virtual TOE for each virtual machine. In order to eliminate the hypervisor overheads of device emulation, we use direct I/O access to improve network performance. The direct I/O access architecture provides the abstraction that each virtual machine is connected directly to its own control interface. Through direct I/O access, the VMM interception overhead can be eliminated and the network throughput is 5% to 20% higher than the device emulation.
In addition to improve the network performance, we propose the QoS dispatcher which employs deficit weighted round robin (DWRR) algorithm to guarantee the Quality of Service (QoS) within all virtual machines. Traditionally, the VMM scheduler focuses on fairly sharing of the processor resources; however, this scheduling policy can result in poor I/O performance. We explore the relationship between the guest scheduling policy and I/O performance to find the suitable policy which can share the CPU resource fairly while obtain the acceptable I/O performance.

[1] N. Bierbaum, Sandia National Laboratories, Livermore, CA, “MPI and Embedded TCP/IP Gigabit Ethernet Cluster Comnputing,” in the IEEE Conference on Local Computer Network (LCN), Nov. 6-8, 2002, pp. 733 - 734.
[2] B. Zhang, et al., “A Survey on I/O Virtualization and Optimization,” in the 2010 Fifth Annual, ChinaGrid Conference (ChinaGrid), July. 16-18, 2010, pp. 117 - 123.
[3] P. Barham, et al., “Xen and the Art of Virtualization,” in the Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles (SOSP), New York, USA, Dec. 2003, pp. 164-177.
[4] J. Sugerman, G. Venkitachalam, and B.-H. Lim, “Virtualizing I/O Devices on VMware Workstation’s Hosted Virtual Machine Monitor,” in the Proceedings of the General Track: 2002 USENIX Annual Technical Conference (ATC), 2001, pp. 1-14.
[5] C.-T. Liu, “CASL Hypervisor and its Virtualization Platform,” 2012 master thesis of National Cheng Kung University.
[6] I. Ahmad, et al., “An Analysis of Disk Performance in VMware ESX Server Virtual Machines,” in the IEEE International Workshop on Workload Characterization (WWC), Oct. 27, 2003, pp. 65-76.
[7] K. Fraser, et al., “Safe Hardware Access with the Xen Virtual Machine Monitor,” in the 1st Workshop on Operating System and Architectural Support for the on demand IT InfraStructure (OASIS) , 2004.
[8] A. Menon, A. L. Cox, and W. Zwaenepoel, “Optimizing Network Virtualization in Xen,” in the Proceedings of the annual conference on USENIX '06 Annual Technical Conference, 2006, pp. 2.
[9] J. Wang, K.-L. Wright, and K. Gopalan, “XenLoop: A Transparent High Performance Inter-Vm Network Loopback,” in the Proceedings of the 17th international symposium on High performance distributed computing (HDPC), New York, USA, 2008, pp. 109-118.
[10] J. S. Renato, et al., “Bridging the Gap Between Software and Hardware Techniques for I/O Virtualization,” in the Proceedings of the USENIX’08 Annual Technical Conference (ATC), 2008, pp. 29-42.
[11] H. Raj, and K. Schwan, “High Performance and Scalable I/O Virtualization via Self-Virtualized Devices,” in the Proceedings of the 16th international symposium on High performance distributed computing (HDPC), New York, USA, 2007, pp. 179-188.
[12] Ram, Kaushik Kumar, et al., “Achieving 10 Gb/s Using Safe and Transparent Network Interface Virtualization,” in the Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments (VEE), 2009, pp. 61-70.
[13] Willmann, Paul, et al., “Concurrent Direct Network Access for Virtual Machine Monitors,” in the IEEE 13th International Symposium on High Performance Computer Architecture (HPCA), Scottsdale, AZ, Feb. 10-14, 2007, pp. 306-317.
[14] Dong, Yaozu, et al., “High Performance Network Virtualization with SR-IOV,” in the IEEE 13th International Symposium on High Performance Computer Architecture (HPCA), Bangalore, Jan. 9-14, 2010, pp. 1-10.
[15] ARM Architecture Reference Manual, 2012.
[16] AMD, “IO Virtualization Technology (IOMMU) Specification.” AMD Pub 34434 (2007).
[17] D.-J. Kang, et al., “TCP/IP Offload Engine Module Supporting Binary Compatibility for Standard Socket Interfaces,” in the Grid and Cooperative Computing (GCC), Springer Berlin Heidelberg, vol. 3795, pp. 357-369, 2005.
[18] C.‐C. Wang and C.‐H. Chen, “A System‐Level Network Virtual Platform for IPsec Processor Development,” IEICE Transactions on Information and Systems, Vol.E96-D, No.5, pp. 1095-1104, May 2013.
[19] G. Martin, B. Bailey, and A. Piziali, “ESL Design and Verification: a Prescription for Electronic System Level Methodology,” Morgan Kaufmann/Elsevier, 2007.
[20] M. Shreedhar and G. Varghese, “Efficient Fair Queueing using Deficit Round Robin,” in the ACM SIGCOMM Computer Communication Review, Vol. 25, No. 4, Oct. 1995, pp. 231-242.
[21] Wireshark, open source packet analyzer, http://www.wireshark.org/
[22] H. Jang, S.-H. Chung, and D.-H. Yoo, “Implementation of an Efficient RDMA Mechanism Tightly Coupled with a TCP/IP Offload Engine,” in the International Symposium on Industrial Embedded Systems (SIES), June. 11-13, 2008, pp. 82-88.
[23] D.-J. Kang, et al., “Design and Implementation of Kernel S/W for TCP/IP Offload Engine (TOE),” in the 7th International Conference on Advanced Communication Technology (ICACT), 2005, pp. 706-709.
[24] Z.-Z Wu and H.-C. Chen, “Design and Implementation of TCP/IP Offload Engine System over Gigabit Ethernet,” in the Proceedings of 15th International Conference on Computer Communications and Networks (ICCCN), Oct. 9-11, 2006, pp. 245-250.
[25] Y. Ji and Q.-S. Hu. “40Gbps Multi-connection TCP/IP Offload Engine,” in the International Conference on Wireless Communications and Signal Processing (WCSP), Nanjing, Nov. 9-11, 2011, pp.1-5.
[26] H. Jang, S.-H. Chung, and S.-C. Oh, “Implementation of a Hybrid TCP/IP Offload Engine Prototype,” in the Advances in Computer Systems Architecture, Springer Berlin Heidelberg, vol. 3740, 2005, pp. 464-477.
[27] ARM Generic Interrupt Controller Architecture Specification, 2011.
[28] ARM Dual-Timer Module (SP840) Technical Reference Manual, 2004.
[29] PrimeCell UART (PL011) Technical Reference Manual, 2005.
[30] R. Mijat, A. Nightingale, “Virtualization is Coming to a Platform Near You,” ARM White Paper, 2011.
[31] D. Ongaro, A. L. Cox, and S. Rixner, “Scheduling I/O in Virtual Machine Monitors.” in the Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments (VEE), 2008, pp.1-10.
[32] S. Makineni and R. Iyer, “Architectural Characterization of TCP/IP Packet Processing on the Pentium® M Microprocessor,” in the Proceedings of the 10th International Symposium on High Performance Computer Architecture, 2004.
[33] www.linuxfoundation.org/collaborate/workgroups/networking/toe
[34] N. Fugier and et al., “Studying Network Protocol Offload with Emulation: Approach and Preliminary Results,” in the Proceedings of the 12th Annual IEEE Symposium on High Performance Interconnects, 2004.
[35] T. Liu, H. Zhu, C. Zhou, G. Chang, “A New Method of TOE Architecture Based on the ML403 Development Board in Linux,” in the First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS), Vol. 2, June. 20-24, 2006, pp. 160-166.
[36] Ang, B. Seong, “An Evaluation of an Attempt at Offloading TCP/IP Protocol Processing onto an i960RN-based iNIC,” Computer Systems and Technology Laboratory HP Laboratories, 2001.
[37] K. Hashimoto and V. G. Moshnyaga, “A New Approach for TCP/IP Offload Engine Implementation in Embedded Systems,” in the Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers (ASILOMAR), Pacific Grove, CA, Nov. 7-10, 2010, pp. 1249-1253.
[38] H. Ghadia, “Benefits of Full TCP/IP Offload in the NFS Services,” NFS Industry Conference, 2003.
[39] Z. Chang and et al., “Adjustable Credit Scheduling for High Performance Network Virtualization,” in the IEEE International Conference on Cluster Computing (CLUSTER), Beijing, Sept. 24-28, 2012, pp. 337-345.