Web API 是現代網頁開發中的重要組成部分，其使得服務整合和自動化成為可能。確保其效能和功能至關重要，但由於效能漏洞難以測定的特性，效能測試相關研究相較於功能測試的研究來得較少。本文提出了一種基於回應時間指引的基因演算法模糊測試方法，在黑箱環境中發掘 web API 的效能延遲。與傳統的隨機輸入生成不同，我們的方法使用基因演算法，並透過該方法的交換和突變來改進輸入，並以回應時間為基礎作為適性分數來指引。我們提出了兩種種子生成方法：使用微軟的 Pairwise Independent Combinatorial Testing（PICT）進行成對組合測試以及隨機配對組合。我們將我們的方法與經典的隨機模糊測試進行了比較。在五個實際的 web API 上的實驗顯示，我們的方法顯著優於經典隨機模糊測試，能夠識別出回應時間延遲 1.5 到 26.3 倍的輸入。此外，以 PICT 生成的種子在 5 個 API 中有 2 個顯示出該方法的效能比隨機配對組合更好。我們的研究結果強調了基於基因演算法的模糊測試在發掘 web API 效能延遲方面的潛力，並倡導在該領域進行進一步的研究。

Web APIs are integral to modern web development, enabling service integration and automation. Ensuring their performance and functionality is critical, yet performance testing is less explored due to the difficulty in detecting performance bugs. This paper presents a response time-guided genetic algorithm (GA) fuzzing approach to uncover web API performance latency in a black-box setting. Unlike traditional random input generation, our method uses GA to refine inputs through crossover and mutation, guided by response time-based fitness. We propose two seed generation methods: pairwise combinatorial testing using Mircosoft’s Pairwise Independent Combinatorial Testing (PICT) and randomly-paired combinations. We compared our method with classic random fuzzing. Experiments on five real-world web APIs show that our approach significantly outperforms classic random fuzzing, identifying inputs with response times 1.5 to 26.3 times longer. Additionally, PICT-generated seeds demonstrated superior performance compared to randomly-paired combinations in 2 out of 5 APIs. Our findings highlight the potential of GA-based fuzzing to reveal web API performance latency, advocating for further research in this area.

[1] P. Lianglu, S. Cohney, T. Murray and V.-T. Pham, "EDEFuzz: A Web API Fuzzer for Excessive Data Exposures," Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, pp. 1-12, 2024.
[2] V. Atlidakis, P. Godefroid and M. Polishchuk., "Restler: Stateful rest api fuzzing.," IEEE/ACM 41st International Conference on Software Engineering (ICSE), pp. 748-758, 2019.
[3] C.-H. Tsai, S.-C. Tsai and S.-K. Huang, "REST API fuzzing by coverage level guided blackbox testing," 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), pp. 291-300, 2021.
[4] A. Martin-Lopez, S. Segura and A. Ruiz-Cortés, "RESTest: automated black-box testing of RESTful web APIs," Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2021), pp. 682-685, 2021.
[5] D. Felício, J. Simão and N. Datia, "Rapitest: Continuous black-box testing of restful web apis," Procedia Computer Science, vol. 219, pp. 537-545, 2023.
[6] J. Wang, X. Bai, L. Li, Z. Ji and H. Ma, "A model-based framework for cloud API testing," 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC), vol. 2, pp. 60-65, 2017.
[7] Z. Hatfield-Dodds and D. Dygalo, "Deriving semantics-aware fuzzers from web API schemas," Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings, pp. 345-346, 2022.
[8] G. Jin, L. Song, X. Shi, S. Joel and S. Lu, "Understanding and detecting real-world performance bugs," Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '12). Association for Computing Machinery, pp. 77-88, 2012.
[9] D. Ray and S. Seshan, "CC-fuzz: genetic algorithm-based fuzzing for stress testing congestion control algorithms," Proceedings of the 21st ACM Workshop on Hot Topics in Networks, pp. 31-37, 2022.
[10] C. Killian, K. Nagaraj, S. Pervez, R. Braud, J. W. Anderson and R. Jhala, "Finding latent performance bugs in systems implementations," Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering (FSE '10), pp. 17-26, 2010.
[11] M. A. Domingues, "Performance testing of open-source HTTP web," Proceedings of the 12th Doctoral Symposium in Informatics Engineering, pp. 8-18, 2017.
[12] K. Kronis and U. Marina, "Performance comparison of java ee and asp. net core technologies for web api development," Applied Computer Systems, pp. 37-44, 2018.
[13] C. Lemieux, R. Padhye, K. Sen and D. Song, "PerfFuzz: automatically generating pathological inputs," Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 254-265, 2018.
[14] X. Zhu, S. Wen, S. Camtepe and Y. Xiang, "Fuzzing: A Survey for Roadmap," ACM Computing Surveys (CSUR), vol. 54, pp. 1-36, 2022.
[15] G. Zhao, S. Hassan, Y. Zou, D. Truong and T. Corbin, "Predicting Performance Anomalies in Software Systems at Run-time," ACM Transactions on Software Engineering and Methodology (TOSEM), pp. 1-33, 2021.
[16] F. Duchene, S. Rawat, J.-L. Richier and R. Groz, "KameleonFuzz: evolutionary fuzzing for black-box XSS detection," Proceedings of the 4th ACM conference on Data and application security and privacy, pp. 37-48, 2014.
[17] "American Fuzzy Lop (AFL)," [Online]. Available: https://github.com/google/AFL. [Accessed 30 5 2024].
[18] R. L. Seagle Jr, "A Framework for File Format Fuzzing with Genetic Algorithms," University of Tennessee, 2012.
[19] E. Jääskelä, "Genetic algorithm in code coverage guided fuzz testing," University of Oulu, 2016.
[20] D. R. Kuhn, R. N. Kacker and Y. Lei, Introduction to combinatorial testing, CRC press, 2013.
[21] "PICT - Pairwise Independent Combinatorial Testing.," [Online]. Available: https://github.com/Microsoft/pict. [Accessed 30 5 2024].
[22] M. Tian, T. Voigt, T. Naumowicz, H. Ritter and J. Schiller, "Performance considerations for mobile web services," Computer communications, pp. 1097-1105, 2004.
[23] J. Xu, Y. Wang, P. Chen and P. Wang, "Lightweight and Adaptive Service API Performance Monitoring in Highly Dynamic Cloud Environment," 2017 IEEE International Conference on Services Computing (SCC), pp. 35-43, 2017.
[24] M. Hendayun, A. Ginanjar and Y. Ihsan, "Analysis of application performance testing using load testing and stress testing methods in API service," J. Sisfotek Glob, pp. 28-34, 2023.
[25] J. Stählin, S. Lang, F. Kajzar and C. Zirpins, "Consumer-Driven API Testing with Performance Contracts," In Advances in Service-Oriented and Cloud Computing: Workshops of ESOCC, pp. 135-143, 2016.
[26] X. Zhou and B. Wu, "Web Application Vulnerability Fuzzing Based On Improved Genetic Algorithm," 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), vol. 1, pp. 977-981, 2020.
[27] O. Chang, J. Metzman, M. Moroz, M. Barbella and A. Arya, "OSS-Fuzz: Continuous Fuzzing for Open Source Software," [Online]. Available: https://github.com/google/oss-fuzz. [Accessed 30 5 2024].
[28] Y. Deng, C. S. Xia, H. Peng, C. Yang and L. Zhang, "Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models," Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis, pp. 423-435, 2023.
[29] S. Karamcheti, G. Mann and D. Rosenberg, "Adaptive grey-box fuzz-testing with thompson sampling," Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security, pp. 37-47, 2018.
[30] M. Xu, S. Kashyap, H. Zhao and T. Kim, "Krace: Data Race Fuzzing for Kernel File Systems," IEEE Symposium on Security and Privacy (SP), pp. 1643-1660, 2020.
[31] Y. Zheng, A. Davanian, H. Yin, C. Song, H. Zhu and L. Sun, "FIRM-AFL: High-Throughput greybox fuzzing of IoT "rmware via augmented process emulation," In 28th USENIX Security Symposium, pp. 1099-1114, 2019.
[32] R. Zhong, Y. Chen, H. Hu, H. Zhang, W. Lee and D. Wu., "SQUIRREL: Testing database management systems with language validity and coverage feedback," In The ACM Conference on Computer and Communications Security, pp. 955-970, 2020.
[33] J. Chen, W. Diao, Q. Zhao, C. Zuo, Z. Lin, X. F. Wang and W. C. Lau, "IoTFUZZER: Discovering memory corruptions in IoT through app-based fuzzing," In The Network and Distributed System Security Symposium (NDSS’18), pp. 1-15, 2018.
[34] P. C. Jorgensen, Software testing: a craftsman's approach, Auerbach Publications, 2013.
[35] D. J. Russo, B. R. Van, A. Kazerouni, I. Osband and Z. Wen, "A Tutorial on Thompson Sampling," Foundations and Trends® in Machine Learning, pp. 1-96, 2018.
[36] "got - Human-friendly and powerful HTTP request library for Node.js," [Online]. Available: https://github.com/sindresorhus/got. [Accessed 30 5 2024].
[37] "http-timer - Performance timings for HTTP requests," [Online]. Available: https://github.com/szmarczak/http-timer. [Accessed 30 5 2024].
[38] M. Mitchell, An introduction to genetic algorithms, MIT press, 1998.
[39] M. N. Razali and J. Geraghty, "Genetic algorithm performance with different selection strategies in solving TSP," Proceedings of the world congress on engineering, vol. 1, pp. 1-6, 2011.
[40] G. Syswerda, "Uniform crossover in genetic algorithms," ICGA, vol. 3, pp. 2-9, 1989.
[41] O. Chapelle and L. Lihong, "An empirical evaluation of thompson sampling," Advances in neural information processing systems, Vols. 2249-2257, 2011.