網路釣魚攻擊常透過偽造網址、仿冒頁面與社交工程誘使使用者輸入敏感資料，對偵測模型而言，關鍵挑戰不僅是辨識釣魚頁面，也包括在大量良性流量中控制誤報。若僅於釣魚比例較高的測試資料上評估，可能高估模型於實際部署時的告警品質。本研究以 PhreshPhish 資料集（v1.0.1）為基礎，聚焦於 URL 字串與清洗後 HTML 可視文字作為輸入，建立 E1-E8 共八組線性模型對照實驗，比較單模態、早期融合與晚期融合模型於標準與低基準率測試集之表現。
資料集共包含 666,315 筆樣本，本研究採時間序列方式建立訓練集、驗證集與標準測試集，並以 PhreshPhish 資料集之低基準率測試集清單自標準測試集對齊指定樣本。特徵表示採 TF-IDF：URL 使用字元級 3-5 元語法，HTML 使用清洗後可視文字之詞級 1-2 元語法；分類器則比較 Logistic Regression 與經 sigmoid 校準之 LinearSVC。早期融合串接 URL 與 HTML 稀疏向量後訓練單一分類器；晚期融合則整合兩個單模態預測機率，並以驗證集在召回率達 0.90 時之精確率（Precision@Recall=0.90, P@R=0.90）搜尋融合權重。評估指標除精確率（Precision）、召回率（Recall）、F1-score、誤報率（False Positive Rate, FPR）與接收者操作特徵曲線下面積（ROC-AUC）外，並以平均精確率（Average Precision, AP）與 P@R=0.90 作為低基準率情境下之核心指標。
結果顯示，在本研究特徵設定下，URL 單模態模型於低基準率測試中明顯優於 HTML 之可視文字單模態模型。多模態模型中，早期融合效果最明確，E6（早期融合搭配校準後 LinearSVC）於 0.05% 至 5.00% 各低基準率測試集皆取得最高平均 AP 與平均 P@R=0.90，並於標準測試集取得 AP=0.9929、P@R0.90=0.9902 與 F1-score=0.9553。E8（晚期融合搭配校準後 LinearSVC）雖未在低基準率主要指標上超越 E6，但在標準測試集固定閾值 τ=0.5 下取得最低 FPR（0.0093）與最高 Precision（0.9879），可作為誤報成本較高情境下之替代模型。
研究結果說明，僅依標準測試集判斷釣魚偵測模型效能，可能低估低基準率情境下的誤報成本。在本研究設定下，E6 於排序型指標與高召回條件下取得最佳平均表現，較適合作為初期高風險 URL 排序模型；E8 則可作為固定閾值下誤報控制較保守的候選配置。由於 0.05% 基準率下即使最佳模型仍會產生大量誤報，實際部署仍需搭配閾值調整、分級告警、人工審查或第二層模型篩選。本研究建立之 E1–E8 對照流程與低基準率評估方式，可供後續比較不同模態、校準方法與部署策略時沿用。

Phishing websites remain a persistent cybersecurity threat because attackers combine deceptive URLs, impersonated web content, and social engineering to steal credentials or financial information. A key deployment challenge is that phishing events are rare in real traffic. Models that perform well on nearly balanced test sets may therefore be overly optimistic when applied to practical, low base-rate environments.
This thesis uses the PhreshPhish dataset (v1.0.1, 666,315 samples) and builds a reproducible E1-E8 comparison matrix across URL-only, HTML-only, early-fusion, and late-fusion settings. Data are chronologically split into training, validation, and standard test sets, with the final 25% reserved as the standard test set. URL features use character-level TF-IDF (3-5 grams), and HTML features use word-level TF-IDF (1-2 grams) from cleaned visible text. Logistic Regression and sigmoid-calibrated LinearSVC are used as linear classifiers. Low base-rate evaluation follows base rates from 0.05% to 5.00%.
Results show that URL-only models are consistently stronger than HTML-only models under severe class imbalance. Among multimodal settings, early fusion provides the clearest improvement, while late fusion is more useful when fixed-threshold false-positive control is prioritized. E6 (early fusion with calibrated LinearSVC) is the most stable configuration in this experimental setting: it achieves the highest average precision (AP) and P@R=0.90 across all low base-rate settings, and on the standard test set it reaches AP=0.9929, P@R0.90=0.9902, and F1=0.9553. E8 (late fusion with calibrated LinearSVC branches) does not exceed E6 on ranking-oriented metrics but provides the lowest false-positive rate (FPR=0.0093) at the fixed threshold τ=0.5.
Overall, the thesis shows that low base-rate evaluation is necessary for realistic phishing-model assessment. In this setting, feature-level URL/HTML fusion provides the strongest overall evaluation results, while score-level late fusion remains a practical alternative when false-positive control is the primary deployment objective.

[1] Ahmed Abbasi, Zhu Zhang, David Zimbra, Hsinchun Chen, and Jay F Nunamaker Jr. Detecting fake websites: The contribution of statistical learning theory. MIS Quarterly, 34(3):435–461, 2010. doi:10.2307/25750686.
[2] Ali Aljofey, Saifullahi Aminu Bello, Jian Lu, and Chen Xu. Comprehensive phishing detection: A multi-channel approach with variants TCN fusion leveraging URL and HTML features. Journal of Network and Computer Applications, 238:104170, 2025. URL: https://www.sciencedirect.com/science/article/pii/S1084804525000670, doi:10.1016/j.jnca.2025.104170.
[3] Ali Aljofey, Qingshan Jiang, Abdur Rasool, Hui Chen, Wenyin Liu, Qiang Qu, and Yang Wang. An effective detection approach for phishing websites using URL and HTML features. Scientific Reports, 12(1):8842, 2022. URL: https://www.nature.com/articles/s41598-022-10841-5, doi:10.1038/s41598-022-10841-5.
[4] AHA Almakhamreh et al. Crossphire: Benefiting multimodality for robust phishing web page identification. Applied Sciences, 16(2):751, 2026. doi:10.3390/app16020751.
[5] Anti-Phishing Working Group. Phishing activity trends report, 4th quarter 2025. Technical report, Anti-Phishing Working Group, 2026. Accessed 2026-05-15. URL: https://docs.apwg.org/reports/apwg_trends_report_q4_2025.pdf.
[6] Daniel Arp, Erwin Quiring, Feargus Pendlebury, Alexander Warnecke, Fabio Pierazzi, Christian Wressnegger, Lorenzo Cavallaro, and Konrad Rieck. Dos and don’ts of machine learning in computer security. In 31st USENIX Security Symposium (USENIX Security 22), pages 3971–3988, 2022. doi:10.5445/IR/1000139897.
[7] Stefan Axelsson. The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security (TISSEC), 3(3):186–205, 2000. doi:10.1145/357830.357849.
[8] Alejandro Correa Bahnsen, Eduardo Contreras Bohorquez, Sergio Villegas, Javier Vargas, and Fabio A González. Classifying phishing URLs using recurrent neural networks. In 2017 APWG Symposium on Electronic Crime Research (eCrime), pages 1–8. IEEE, 2017. doi:10.1109/ecrime.2017.7945048.
[9] Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2):423–443, 2018. doi:10.1109/tpami.2018.2798607.
[10] Aaron Blum, Brad Wardman, Thamar Solorio, and Gary Warner. Lexical feature based phishing URL detection using online learning. In Proceedings of the 3rd ACM Workshop on Artificial Intelligence and Security, pages 54–60, 2010. doi:10.1145/1866423.1866434.
[11] Ahmet Selman Bozkir, Firat Coskun Dalgic, and Murat Aydos. Grambeddings: A new neural network for URL based identification of phishing web pages through n-gram embeddings. Computers & Security, 124:102964, 2023. doi:10.1016/j.cose.2022.102964.
[12] Davide Canali, Marco Cova, Giovanni Vigna, and Christopher Kruegel. Prophiler: a fast filter for the large-scale detection of malicious web pages. In Proceedings of the 20th International Conference on World Wide Web, pages 197–206, 2011. doi:10.1145/1963405.1963436.
[13] William B Cavnar, John M Trenkle, et al. N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd Annual Symposium on Document Analysis and Information Retrieval, volume 161, pages 161–175. Citeseer, 1994.
[14] Amitabh Chakravorty, Matthew Price, Nelly Elsayed, and Zag ElSayed. Context-aware phishing email detection using machine learning and NLP. arXiv preprint arXiv:2603.27326, 2026. doi:10.48550/arXiv.2603.27326.
[15] Neil Chou, Robert Ledesma, Yuka Teraguchi, Dan Boneh, and John C Mitchell. Client-side defense against web-based identity theft. Computer Science Department, Stanford University, 2004.
[16] Cloudflare. What is a phishing attack?, 2026. Accessed: 2026-05-12. URL: https://www.cloudflare.com/learning/access-management/phishing-attack/.
[17] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20(3):273–297, 1995. doi:10.1007/bf00994018.
[18] David R Cox. The regression analysis of binary sequences. Journal of the Royal Statistical Society Series B: Statistical Methodology, 20(2):215–232, 1958. doi:10.1111/j.2517-6161.1958.tb00292.x.
[19] Thomas Dalton, Hemanth Gowda, Girish Rao, Sachin Pargi, Alireza Hadj Khodabakhshi, Joseph Rombs, Stephan Jou, and Manish Marwah. PhreshPhish: A real-world, high-quality, large-scale phishing website dataset and benchmark. arXiv preprint arXiv:2507.10854v2, 2025. Version 2. doi:10.48550/arXiv.2507.10854.
[20] Thomas Dalton, Hemanth Gowda, Girish Rao, Sachin Pargi, Alireza Hadj Khodabakhshi, Joseph Rombs, Stephan Jou, and Manish Marwah. PhreshPhish dataset, 2026. Hugging Face Dataset Card. Accessed: 2026-03-26. URL: https://huggingface.co/datasets/phreshphish/phreshphish.
[21] Avisha Das, Shahryar Baki, Ayman El Aassal, Rakesh Verma, and Arthur Dunbar. SoK: a comprehensive reexamination of phishing research from the security perspective. IEEE Communications Surveys & Tutorials, 22(1):671–708, 2020. doi:10.1109/COMST.2019.2957750.
[22] Jesse Davis and Mark Goadrich. The relationship between precision-recall and ROC curves. In Proceedings of the 23rd International Conference on Machine Learning, pages 233–240, 2006. doi:10.1145/1143844.1143874.
[23] European Union Agency for Cybersecurity. ENISA threat landscape 2025. Technical report, ENISA, 2025. Accessed 2026-05-15. URL: https://www.enisa.europa.eu/sites/default/files/2025-11/ENISA%20Threat%20Landscape%202025.pdf.
[24] Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. The Journal of Machine Learning Research, 9:1871–1874, 2008. URL: https://jmlr.csail.mit.edu/papers/v9/fan08a.html.
[25] Federal Bureau of Investigation Internet Crime Complaint Center. 2025 internet crime report. Technical report, Federal Bureau of Investigation, 2026. Accessed 2026-05-15. URL: https://www.ic3.gov/AnnualReport/Reports/2025_IC3Report.pdf.
[26] Anthony Y Fu, Liu Wenyin, and Xiaotie Deng. Toward detecting phished web pages by visual similarity. In Proceedings of the 15th International Conference on World Wide Web, pages 1055–1056, 2006.
[27] Sujata Garera, Niels Provos, Monica Chew, and Aviel D Rubin. A framework for detection and measurement of phishing attacks. In Proceedings of the 2007 ACM Workshop on Recurring Malcode, pages 1–8, 2007. doi:10.1145/1314389.1314391.
[28] A Ghafoor, Munam Ali Shah, Mohammed Al-Naeem, and Carsten Maple. Decoding phishing evasion: Analyzing attacker strategies to circumvent detection systems. IEEE Access, 2025. doi:10.1109/access.2025.3565619.
[29] Abdelhakim Hannousse and Salima Yahiouche. Webpage phishing detection. Mendeley Data, 3, 2020. doi:10.17632/c2gw7fy2j4.3.
[30] Abdelhakim Hannousse and Salima Yahiouche. Towards benchmark datasets for machine learning based website phishing detection: An experimental study. Engineering Applications of Artificial Intelligence, 104:104347, 2021. URL: https://www.sciencedirect.com/science/article/pii/S0952197621001950, doi:10.1016/j.engappai.2021.104347.
[31] Haibo He and Edwardo A. Garcia. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9):1263–1284, 2009. doi:10.1109/tkde.2008.239.
[32] Jahirul Islam, Chanchal Patra, Pritam Kumar Mani, Soumik Biswas, Debasis Giri, and Tanmoy Maitra. Phishing URL detection via machine learning: A comprehensive survey. In International Conference on Network Security and Blockchain Technology, pages 141–159. Springer, 2025. doi:10.1007/978-981-96-6348-4_12.
[33] Ashvini Jadhav and Pankaj R Chandre. Survey and comparative analysis of phishing detection techniques: current trends, challenges, and future directions. IAES International Journal of Artificial Intelligence (IJ-AI), 14(2):853, 2025. doi:10.11591/ijai.v14.i2.pp853-866.
[34] Haifeng Jiang, Yu Chen, Yingjie Zhu, Xin Xu, Yuejia Song, and Qiwei Chen. D-phishnet: A dual-branch network for URL and HTML feature fusion in phishing webpage detection. Computer Networks, 271:111648, 2025. doi:10.1016/j.comnet.2025.111648.
[35] Thorsten Joachims. Text categorization with support vector machines: Learning with many relevant features. In European Conference on Machine Learning (ECML), pages 137–142. Springer, 1998. doi:10.1007/bfb0026683.
[36] Josef Kittler, Mohamad Hatef, Robert P.W. Duin, and Jiri Matas. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(3):226–239, 1998. doi:10.1109/34.667881.
[37] Michael Kolton and Shyh-Yuan Maa. Competing in the cognitive domain: Lessons from Taiwan’s anti-fraud initiative. Security Nexus, 27:1–15, February 2026. URL: https://www.jstor.org/stable/resrep78542.
[38] Ludmila I Kuncheva. Combining pattern classifiers: methods and algorithms. John Wiley & Sons, 2014. doi:10.1002/9781118914564.
[39] Anh Le, Athina Markopoulou, and Michalis Faloutsos. Phishdef: URL names say it all. In 2011 Proceedings IEEE INFOCOM, pages 191–195. IEEE, 2011. doi:10.1109/infcom.2011.5934995.
[40] Hung Le, Quang Pham, Doyen Sahoo, and Steven CH Hoi. URLNet: Learning a URL representation with deep learning for malicious URL detection. arXiv preprint arXiv:1802.03162, 2018. Version 2. doi:10.48550/arXiv.1802.03162.
[41] Yuexin Li, Chengyu Huang, Shumin Deng, Mei Lin Lock, Tri Cao, Nay Oo, Hoon Wei Lim, and Bryan Hooi. KnowPhish: Large language models meet multimodal knowledge graphs for enhancing reference-based phishing detection. In 33rd USENIX Security Symposium (USENIX Security 24), pages 793–810, 2024. doi:10.48550/arXiv.2403.02253.
[42] Hsuan-Tien Lin, Chih-Jen Lin, and Ruby C. Weng. A note on Platt’s probabilistic outputs for support vector machines. Machine Learning, 68(3):267–276, 2007. doi:10.1007/s10994-007-5018-6.
[43] Ruofan Liu, Yun Lin, Xianglin Yang, Siang Hwee Ng, Dinil Mon Divakaran, and Jin Song Dong. Inferring phishing intention via webpage appearance and dynamics: A deep vision based approach. In 31st USENIX Security Symposium (USENIX Security 22), pages 1633–1650, 2022. URL: https://www.usenix.org/conference/usenixsecurity22/presentation/liu-ruofan.
[44] Justin Ma, Lawrence K Saul, Stefan Savage, and Geoffrey M Voelker. Beyond blacklists: learning to detect malicious web sites from suspicious URLs. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1245–1254, 2009. doi:10.1145/1557019.1557153.
[45] Pratyush Maneriker, Justin Pearson, Feras Al-Obeidat, et al. URLTran: Improving phishing URL detection using transformers. In MILCOM 2021 - 2021 IEEE Military Communications Conference (MILCOM), pages 197–204, 2021. doi:10.1109/milcom52596.2021.9653028.
[46] Samuel Marchal, Jérôme François, Radu State, and Thomas Engel. Phishstorm: Detecting phishing with streaming analytics. IEEE Transactions on Network and Service Management, 11(4):458–471, 2014. doi:10.1109/tnsm.2014.2377295.
[47] D Kevin McGrath and Minaxi Gupta. Behind phishing: an examination of phisher modi operandi. In Proceedings of the 1st USENIX Workshop on Large-Scale Exploits and Emergent Threats (LEET), pages 1–8, 2008. URL: https://www.usenix.org/conference/leet-08/presentation/behind-phishing-examination-phisher-modi-operandi.
[48] Rami M Mohammad, Fadi Thabtah, and Lee McCluskey. Phishing websites features. School of Computing and Engineering, University of Huddersfield, 138, 2015. doi:10.24432/C51W2X.
[49] Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng. Multimodal deep learning. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 689–696, 2011. URL: https://icml.cc/Conferences/2011/papers/399_icmlpaper.pdf.
[50] Chidimma Opara, Bo Wei, and Yingke Chen. HTMLPhish: Enabling phishing webpage detection by applying deep learning techniques on HTML analysis. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2020. doi:10.1109/IJCNN48605.2020.9207707.
[51] Feargus Pendlebury, Fabio Pierazzi, Roberto Jordaney, Johannes Kinder, and Lorenzo Cavallaro. Tesseract: Eliminating experimental bias in malware classification across space and time. In 28th USENIX Security Symposium (USENIX Security 19), pages 729–746, 2019. doi:10.48550/arXiv.1807.07838.
[52] John Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers, pages 61–74. MIT Press, 1999.
[53] Pawan Prakash, Manish Kumar, Ramana Rao Kompella, and Minaxi Gupta. PhishNet: predictive blacklisting to detect phishing attacks. In 2010 Proceedings IEEE INFOCOM, pages 1–5. IEEE, 2010. doi:10.1109/infcom.2010.5462216.
[54] Asadullah Safi and Satwinder Singh. A systematic literature review on phishing website detection techniques. Journal of King Saud University - Computer and Information Sciences, 35(2):590–611, 2023. URL: https://www.sciencedirect.com/science/article/pii/S1319157823000034, doi:10.1016/j.jksuci.2023.01.004.
[55] Ozgur Koray Sahingoz, Ebubekir Buber, Onder Demir, and Banu Diri. Machine learning based phishing detection from URLs. Expert Systems with Applications, 117:345–357, 2019. doi:10.1016/j.eswa.2018.09.029.
[56] Gerard Salton and Christopher Buckley. Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5):513–523, 1988. doi:10.1016/0306-4573(88)90021-0.
[57] Steve Sheng, Brad Wardman, Gary Warner, Lorrie Faith Cranor, Jason Hong, and Cong Zhang. An empirical analysis of phishing blacklists. In Proceedings of the 6th Annual Collaboration, Electronic Messaging, Anti-Abuse and Spam Conference (CEAS), volume 6, pages 1–10, 2009. URL: https://kilthub.cmu.edu/articles/journal_contribution/An_Empirical_Analysis_of_Phishing_Blacklists/6469805.
[58] Cees G M Snoek, Marcel Worring, and Arnold W M Smeulders. Early versus late fusion in semantic video analysis. In Proceedings of the 13th Annual ACM International Conference on Multimedia, pages 399–402, 2005. doi:10.1145/1101149.1101236.
[59] Karen Sparck Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28(1):11–21, 1972. doi:10.1108/eb026526.
[60] Farid Tajaddodianfar, Jack W. Stokes, and Arun Gururajan. Texception: a character/word-level deep learning model for phishing URL detection. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2857–2861. IEEE, 2020. doi:10.1109/icassp40776.2020.9053670.
[61] Maruf Ahmed Tamal, Md Kabirul Islam, Touhid Bhuiyan, and Abdus Sattar. Dataset of suspicious phishing URL detection. Frontiers in Computer Science, 6:1308634, 2024. doi:10.3389/fcomp.2024.1308634.
[62] Ahsan Ullah, Hm Alvee Hasan, Mohammad Anwar Hossain, Noshin Un Noor, Kazi Hassan Robin, and Mst Nishita Aktar. Balanced deep neural network (BDNN) for scalable, real-time phishing URL detection using hybrid machine learning and deep learning ensembles. In 2025 IEEE International Conference on Signal Processing, Information, Communication and Systems (SPICSCON), pages 219–224. IEEE, 2025. doi:10.1109/spicscon69221.2025.11504230.
[63] Alexey Vulfin, Alexey Sulavko, Vladimir Vasiliev, Alexander Minko, Anastasia Kirillova, and Alexander Samotuga. A multimodal phishing website detection system using explainable artificial intelligence technologies. Machine Learning and Knowledge Extraction, 8(1):11, 2026. doi:10.3390/make8010011.
[64] Weiyao Wang, Du Tran, and Matt Feiszli. What makes training multi-modal classification networks hard? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12695–12705, 2020. doi:10.1109/cvpr42600.2020.01271.
[65] Jacek Łukasz Wilk-Jakubowski, Aleksandra Sikora, and Dawid Maciejski. The effectiveness of machine learning in detecting phishing websites. Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska, 15(3):105–109, 2025. doi:10.35784/iapgos.8143.
[66] Guang Xiang, Jason Hong, Carolyn P Rose, and Lorrie Cranor. Cantina+: A feature-rich machine learning framework for detecting phishing web sites. ACM Transactions on Information and System Security (TISSEC), 14(2):1–28, 2011. doi:10.1145/2019599.2019606.
[67] Jun-Ho Yoon, Seok-Jun Buu, and Hae-Jung Kim. Phishing webpage detection via multimodal integration of HTML DOM graphs and URL features based on graph convolutional and transformer networks. Electronics, 13(16):3344, 2024. URL: https://www.mdpi.com/2079-9292/13/16/3344, doi:10.3390/electronics13163344.
[68] Yue Zhang, Jason I Hong, and Lorrie F Cranor. Cantina: a content-based approach to detecting phishing web sites. In Proceedings of the 16th International Conference on World Wide Web, pages 639–648, 2007. doi:10.1145/1242572.1242659.
[69] 內政部警政署. 內政部警政署 165 全民防騙網, 2026. Accessed: 2026-04-20. URL: https://165dashboard.tw/.