現今殭屍網路威脅越來越大，於防範殭屍網路上，除了算法開發外如何將算法實現為能供人使用系統也很重要。在我們先前的研究中，我們開發了一個基於Hadoop平台的P2P殭屍網路檢測演算法 ─ BotCluster ，用於通過使用非監督式機器學習於長期的NetFlow上偵測P2P殭屍網路。由於使用BotCluster前需架設Hadoop集群環境，需要一定的技術門檻而不適合無相關背景知識的使用者使用。
在本文中，我們使用虛擬化技術 ─ Docker Container來封裝整個BotCluster運行環境，以降低使用者對於環境架設的負擔。並為了能夠將容器化BotCluster建置於多節點叢集，使用了Kubernetes來佈署和管理容器集群。另外也提出了兩種容器化Hadoop的架構，根據使用者不同的需求可以有不同的架構選擇。最後，設計了RESTful API來做為系統與使用者之間的串接，達到簡易操作與系統自動化的目的。

Nowadays, the threat of botnets is growing. To defend against botnets, in addition to algorithm development, how to implement algorithms into systems that can be used by people is also very important. In our previous research, we developed BotCluster, which is a P2P botnet detection algorithm based on long-term NetFlow and unsupervised machine learning. Since BotCluster is implemented on a Hadoop cluster environment, it requires a certain technical skill and is not suitable for users without the relevant background knowledge. In this paper, we use the virtualization technology ─ Docker Container to encapsulate the entire BotCluster operating environment to reduce the users’ burden while establishing the runtime environment of BotCluster. And in order to build a containerized BotCluster in a multi-node cluster, Kubernetes is used to deploy and manage container clusters. Besides, two containerized Hadoop architectures were also discussed and proposed. Different architectures can be selected according to the different needs of users. Finally, we design the RESTful APIs for users to perform simple system operations in an efficient way.

[1] AWS Hit With a Record 2.3 Tbps DDoS Attack, https://www.cbronline.com/news/record-ddos-attack-aws
[2] Botnet attacks on companies worldwide went up by 50% in 2019, https://timesofindia.indiatimes.com/gadgets-news/botnet-attacks-on-companies-worldwide-went-up-by-50-in-2019-report/articleshow/73299269.cms
[3] Docker Container https://www.docker.com/
[4] Kubernetes https://kubernetes.io/
[5] Apache Hadoop Map Reduce https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html
[6] Amit M Potdar, Narayan D G, Shivaraj Kengond, Mohammed Moin Mulla, "Performance Evaluation of Docker Container and Virtual Machine," Procedia Computer Science, Volume 171, pp. 1419-1428, 2020
[7] Anuj Kumar Yadav, M. L. Garg, Ritika, "Docker Containers Versus Virtual Machine-Based Virtualization", Emerging Technologies in Data Mining and Information Security, Volume 814, pp. 141-150, 2018
[8] Amr A. Mohallel, Julian M. Bass, Ali Dehghantaha, "Experimenting with docker: Linux container and base OS attack surfaces", International Conference on Information Society (i-Society), 2016
[9] C.-Y. Wang, C.-L. Ou, Y.-E. Zhang, F.-M. Cho, J.-B. Chang, and C.-K. Shieh, "BotCluster: A Session-based P2P Botnet Clustering System on NetFlow," Computer Networks, Volume 145, pp. 175-189, 2018
[10] Garrett Lahmann, Thom McCann, Wes Lloyd, "Container Memory Allocation Discrepancies: An Investigation on Memory Utilization Gaps for Container-Based Application Deployments", IEEE International Conference on Cloud Engineering (IC2E), 2018
[11] Haejin Chung, Yunmook Nah, "Performance Comparison of Distributed Processing of Large Volume of Data on Top of Xen and Docker-Based Virtual Clusters," DASFAA 2017: Database Systems for Advanced Applications, pp. 103-113, 2017
[12] Javier Rey, Matias Cogorno, Sergio Nesmachnow, Luiz Angelo Steffenel, "Efficient Prototyping of Fault Tolerant Map-Reduce Applications with Docker-Hadoop," IEEE International Conference on Cloud Engineering, 2015
[13] Jay Shah, Dushyant Dubaria, " Building Modern Clouds: Using Docker, Kubernetes & Google Cloud Platform", IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), 2019
[14] Leila Abdollahi Vayghan, Mohamed Aymen Saied, Maria Toeroe, Ferhat Khendek, "Deploying Microservice Based Applications with Kubernetes: Experiments and Lessons Learned", IEEE 11th International Conference on Cloud Computing (CLOUD), 2018
[15] Nitin Naik, "Building a virtual system of systems using docker swarm in multiple clouds", IEEE International Symposium on Systems Engineering (ISSE), 2016
[16] P. China Venkanna Varma, K. V. Kalyan ChakravarthyV. Valli KumariS. Viswanadha Raju, "Analysis of Network IO Performance in Hadoop Cluster Environments Based on Docker Containers," Proceedings of Fifth International Conference on Soft Computing for Problem Solving, pp. 227-237, 2016
[17] Soonsung Hwang, Jaehyoung Lee, Dongyeon Kim, Jongpil Jeong, "Design and Performance Analysis of Docker-Based Smart Manufacturing Platform Based on Deep Learning Model",International Conference on Computational Science and Its Applications, pp. 94-104, 2019
[18] Tasneem Salah, M. Jamal Zemerly, Chan Yeob Yeun, Mahmoud Al-Qutayri, Yousof Al-Hammadi, "Performance comparison between container-based and VM-based services", 20th Conference on Innovations in Clouds, Internet and Networks (ICIN), 2017
[19] Víctor Medel Rafael, Tolosana-Calasanz, José Ángel Bañares, Unai Arronategui, Omer F.Rana, "Characterising resource management performance in Kubernetes", Computers & Electrical Engineering, Volume 68, pp. 286-297, 2018
[20] Vladimir Korkhov, Sergey Kobyshev, Alexander Degtyarev, Alexander Bogdanov, "Light-Weight Cloud-Based Virtual Computing Infrastructure for Distributed Applications and Hadoop Clusters," ICCSA 2017: Computational Science and Its Applications, pp. 399-411, 2017
[21] Víctor Medel, Omer Rana, José Ángel Bañares, Unai Arronategui, "Adaptive Application Scheduling under Interference in Kubernetes", IEEE/ACM 9th International Conference on Utility and Cloud Computing (UCC), 2016
[22] Walter Blair, Aspen Olmsted, Paul Anderson, "Docker vs. KVM: Apache spark application performance and ease of use", 12th International Conference for Internet Technology and Secured Transactions (ICITST), 2018
[23] Xili Wan, Xinjie Guan, Tianjing Wang, Guangwei Bai, Baek-Yong, Choi, "Application deployment using Microservice and Docker containers: Framework and optimization", Journal of Network and Computer Applications, Volume 119, pp. 97-109, 2018
[24] Xiao-Lan Xie, Peng Wang, Qi Wang, "The performance analysis of Docker and rkt based on Kubernetes", 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), 2017
[25] Xueyuan Wang, Brian Lee, Yuansong Qiao, "Experimental evaluation of memory configurations of Hadoop in Docker environments," 27th Irish Signals and Systems Conference (ISSC), 2016
[26] Xinjie Guan, Xili Wan, Baek-Young Choi, Sejun Song, Jiafeng Zhu, "Application Oriented Dynamic Resource Allocation for Data Centers Using Docker Containers", IEEE Communications Letters, Volume 21, pp. 504-507, 2016
[27] Yi-wei Chen, Shih-Hao Hung, Chia-Heng Tu, Chih Wei Yeh, "Virtual Hadoop: MapReduce over Docker Containers with an Auto-Scaling Mechanism for Heterogeneous Environments", RACS '16: Proceedings of the International Conference on Research in Adaptive and Convergent Systems, pp. 201-206, 2016
[28] Y A Auliya1, Y Nurdinsyah, D A R Wulandari, "Performance Comparison of Docker and LXD with ApacheBench" Journal of Physics: Conference Series, Volume 1211, The 2nd International Conference of Combinatorics, Graph Theory, and Network Topology, 2018
[29] Zhiheng Zhong, Rajkumar Buyya, "A Cost-Efficient Container Orchestration Strategy in Kubernetes-Based Cloud Computing Infrastructures with Heterogeneous Resources", ACM Transactions on Internet Technology, Volume 20, No. 2, 2020
[30] Zheng Li, Maria Kihl, Qinghua Lu, Jens A. Andersson, Zheng Li, Maria Kihl, Qinghua Lu, Jens A. Andersson, "Performance Overhead Comparison between Hypervisor and Container Based Virtualization", IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), 2017