在這項研究中，我們介紹了兩種機器學習模型，“原始 Transformer”和“時間
序列 Transformer”，旨在創建一個強大且高效的入侵檢測系統（IDS）。 我們的目
標是增強 BERT 基礎模型的性能，該模型最初是在英語文本數據上進行預訓練的，用
於入侵檢測的特定任務。 我們計劃通過修改各種參數、激活函數和其他因素來實現
這一目標。對於“原始 Transformer”方法，我們對 CSE-CIC-IDS-2018 數據集進行
預處理，以刪除重複和不完整的數據。 然後將清理後的數據輸入到 BERT 基礎模型
中進行訓練。 “時間序列轉換器”方法涉及根據標籤和時間戳對數據進行分類，然
後每十個實例對記錄進行分組，以有效捕獲時間模式。 處理後的數據用於訓練 BERT
模型。我們預計我們的實驗將突出我們修改的有效性，強調“時間序列變換器”在特
定方面相對於“原始變換器”方法的優越性。 此外，由於我們在雲環境中的微調和
優化工作，我們預計 IDS 的穩健性和效率將得到全面提高。本研究旨在提高 IDS 檢
測入侵攻擊的準確性，並增強網絡環境中的其他性能指標。 在這項研究中，我們深
入記錄了所有實驗的各個方面。結果表明，多分類的最佳準確率為 82.41%。
我們在 CIC-IDS2018 數據集上的數據包多分類準確率提高了 7%。 我們的創新
包括使用以前未使用的大型模型和新穎的時間序列方法。 與以前的論文不同，我們
在訓練集和測試集之間保持了明確的分離，並利用了整個數據集。

In this research, we introduce two machine learning models, the "original Transformer" and the "Time Series Transformer," aimed at creating a robust and efficient Intrusion Detection System (IDS). Our objective is to enhance the performance of the BERT base model, originally pre-trained on English text data, for the specific task of intrusion detection. We plan to achieve this by modifying various parameters, activation functions, and other factors. For the "original Transformer" approach, we preprocess the CSE-CIC-IDS-2018 dataset using to remove repetitive and incomplete data. The cleaned data is then fed into the BERT base model for training. The "Time Series Transformer" approach involves classifying data based on labels and timestamps, followed by grouping records every ten instances to capture temporal patterns effectively. This processed data is used to train the BERT model. We anticipate our experiments will highlight the effectiveness of our modifications, emphasizing the superiority of the "Time Series Transformer" over the "raw Transformer" method in specific aspects. Additionally, we expect overall improvements in IDS robustness and efficiency due to our fine-tuning and optimization efforts in the cloud environment.This study aims to improve the accuracy of IDS detection of intrusion attacks and enhance other performance metrics in a network environment. In this study, we documented in depth all aspects of all our experiments. The results show that the best accuracy for multi-classification is 82.41%.
We innovation includes the use of previously unused large models and a novel time series approach. Unlike previous papers, we maintained a clear separation between training and test sets and utilized the entire dataset.

[1] tensorflow, https://www.tensorflow.org/learn
[2] Keras, https://keras.io/
[3] CUDA, https://developer.nvidia.com/cuda-zone
[4] Mininet, http://mininet.org/
[5] CSE-CIC-IDS2018, https://www.unb.ca/cic/datasets/ids-2018.html
[6] V. Kanimozhi∗, T. Prem Jacob, “Artificial Intelligence outflanks all other machine learning classifiers in Network Intrusion Detection System on the realistic cyber dataset CSE-CIC-IDS2018 using cloud computing”, Department of CSE, Sathyabama Institute of Science and Technology, Chennai, India,ICT Express Volume 7, Issue 3, September 2021, Pages 366-370, doi : 10.1016/j.icte.2020.12.004
[7] S. Haider et al., "A Deep CNN Ensemble Framework for Efficient DDoS Attack Detection in Software Defined Networks," in IEEE Access, vol. 8, pp. 53972-53983, 2020, doi: 10.1109/ACCESS.2020.2976908.
[8] V. Deepa, K. M. Sudar and P. Deepalakshmi, "Detection of DDoS Attack on SDN Control plane using Hybrid Machine Learning Techniques," 2018 International Conference on Smart Systems and Inventive Technology (ICSSIT), 2018, pp. 299-303, doi: 10.1109/ICSSIT.2018.8748836.
[9] Haomin Wang * and Wei Li, "DDosTC: A Transformer-Based Network Attack Detection Hybrid Mechanism in SDN," 2021 AI-Enabled Security Technologies, 2021, doi: 10.3390/s21155047
[10] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin, “Attention Is All You Need”, Conference on Neural Information Processing Systems (NIPS 2017), 2017, doi: 10.48550/arXiv.1706.03762
[11] Ilya Sutskever, Oriol Vinyals, Quoc V. Le, “Sequence to Sequence Learning with Neural Networks", 2014, doi: 10.48550/arXiv.1409.3215
[12] Wojciech Zaremba, Ilya Sutskever, Oriol Vinyals, “Recurrent Neural Network Regularization”, Under review as a conference paper at ICLR 2015, doi: 10.48550/arXiv.1409.2329
[13] Robert Hecht-Nielsen, “Theory of the backpropagation neural network”, International 1989 Joint Conference on Neural Networks, 2002, doi: 10.1109/IJCNN.1989.118638
[14] Mohammed Alauthman, Hany Elgala, and Mohamed Eltoweissy, “A Convolutional Encoder Network for Intrusion Detection in Controller Area Networks,” 2021 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), 2021, pp. 1-6, doi: 10.1109/SmartGridComm51366.2021.9553371.
[15] I. Sharafaldin∗, A. H. Lashkari, A. A. Ghorbani, “Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization”, University of New Brunswick, Fredericton, Canada; York University, Toronto, Canada, Proceedings of the 4th International Conference on Information Systems Security and Privacy, January 2018, doi : 10.5220/0006639801080116
[16] S. Haider∗, A. Akhunzada, I. Mustafa, T. B. Patel, A. Fernandez, K. K. R. Choo, J. Iqbal, “A Deep CNN Ensemble Framework for Efficient DDoS Attack Detection in Software Defined Networks”, IEEE Access Volume 8, 27 February 2020, Pages 53972-53983, doi : 10.1109/ACCESS.2020.2976908
[17] CICFlowMeter, https://github.com/ahlashkari/CICFlowMeter
[18] Fawaz, Hassan Ismail, et al. "Deep learning for time series classification: a review." Data Mining and Knowledge Discovery 33.4 (2019): 917-963. https://doi.org/10.1007/s10618-019-00619-1. Received: 12 September 2018 / Accepted: 25 February 2019 / Published online: 2 March 2019. The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2019.
[19] Loshchilov, I., & Hutter, F. (2019). Decoupled Weight Decay Regularization. In Proceedings of the International Conference on Learning Representations (ICLR). arXiv preprint arXiv:1711.05101 [cs.LG]. Retrieved from **https://arxiv.org/abs/1711.05101**
[20] Sinha, K., Jia, R., Hupkes, D., Pineau, J., Williams, A., & Kiela, D. (2021). Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) arXiv preprint arXiv:2104.06644 [cs.CL]. Retrieved from **https://arxiv.org/abs/2104.06644**
[21] Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies arXiv preprint arXiv:1810.04805 [cs.CL]. Retrieved from **https://arxiv.org/abs/1810.04805**
[22] Pytorch, https://pytorch.org/
[23] Sklearn, https://scikit-learn.org/stable/
[24] Numpy, https://numpy.org/
[25] Pandas, https://pandas.pydata.org/