Comparative Analysis of Decision Tree, Random Forest, and Naive Bayes for Phishing Website Detection Using K-Fold Cross Validation and Multi-Metric Evaluation
DOI:
https://doi.org/10.25181/rt.v4i2.5082Keywords:
Decision Tree, K-Fold Cross Validation, Naive Bayes, Phishing Detection, Random ForestAbstract
Phishing websites remain difficult to detect because attackers can create new domains faster than blacklist systems can update. This study compares Decision Tree, Random Forest, and Naive Bayes for phishing website classification using 11,055 records with 30 technical features. The models were evaluated with Stratified 10-Fold Cross-Validation and five metrics: accuracy, precision, recall, F1-score, and ROC-AUC. Random Forest produced the best and most stable performance, with 97.23% accuracy, 97.70% precision, 96.02% recall, 96.85% F1-score, and 99.58% ROC-AUC. Decision Tree also performed strongly, while Naive Bayes showed very high recall but many false positives. Feature importance analysis identified SSLfinal_State and URL_of_Anchor as the most influential predictors, supporting lightweight technical-feature screening for phishing detection.Downloads
References
[1] D. Popescul and L. D. Radu, “AI in phishing detection: a bibliometric review,” Front. Artif. Intell., vol. 8, Oct. 2025, doi: 10.3389/frai.2025.1496580.
[2] R. Zieni, L. Massari, and M. C. Calzarossa, “Phishing or Not Phishing? A Survey on the Detection of Phishing Websites,” IEEE Access, vol. 11, pp. 18499–18519, 2023, doi: 10.1109/ACCESS.2023.3247135.
[3] E. Marechal and B. Donnet, “Targeted Attack through Network Fingerprinting,” Journal of Cyber Security and Mobility, Apr. 2021, doi: 10.13052/jcsm2245-1439.1022.
[4] J. L. Wilk-Jakubowski, L. Pawlik, G. Wilk-Jakubowski, and A. Sikora, “Machine Learning and Neural Networks for Phishing Detection: A Systematic Review (2017–2024),” Electronics (Basel)., vol. 14, no. 18, p. 3744, Sep. 2025, doi: 10.3390/electronics14183744.
[5] A. Hannousse and S. Yahiouche, “Towards benchmark datasets for machine learning based website phishing detection: An experimental study,” Eng. Appl. Artif. Intell., vol. 104, p. 104347, Sep. 2021, doi: 10.1016/j.engappai.2021.104347.
[6] A. Butnaru, A. Mylonas, and N. Pitropakis, “Towards Lightweight URL-Based Phishing Detection,” Future Internet, vol. 13, no. 6, p. 154, Jun. 2021, doi: 10.3390/fi13060154.
[7] A. Ejaz, A. N. Mian, and S. Manzoor, “Life-long phishing attack detection using continual learning,” Sci. Rep., vol. 13, no. 1, p. 11488, Jul. 2023, doi: 10.1038/s41598-023-37552-9.
[8] M. Dandotiya, N. K. Goyal, A. Khunteta, and B. Tiwari, “Real time identification of phishing attacks through machine learning enhanced browser extensions,” Sci. Rep., vol. 16, no. 1, p. 6612, Jan. 2026, doi: 10.1038/s41598-026-35655-7.
[9] F. Salahdine, Z. El Mrabet, and N. Kaabouch, “Phishing Attacks Detection A Machine Learning-Based Approach,” in 2021 IEEE 12th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), IEEE, Dec. 2021, pp. 0250–0255. doi: 10.1109/UEMCON53757.2021.9666627.
[10] A. Almomani et al., “Phishing Website Detection With Semantic Features Based on Machine Learning Classifiers,” Int. J. Semant. Web Inf. Syst., vol. 18, no. 1, pp. 1–24, Feb. 2022, doi: 10.4018/IJSWIS.297032.
[11] M. A. Tamal, M. K. Islam, T. Bhuiyan, A. Sattar, and N. U. Prince, “Unveiling suspicious phishing attacks: enhancing detection with an optimal feature vectorization algorithm and supervised machine learning,” Front. Comput. Sci., vol. 6, Jul. 2024, doi: 10.3389/fcomp.2024.1428013.
[12] D. Kalla and S. Kuraku, “Phishing Website URL’s Detection Using NLP and Machine Learning Techniques,” Journal on Artificial Intelligence, vol. 5, no. 0, pp. 145–162, 2023, doi: 10.32604/jai.2023.043366.
[13] K. Omari, “Comparative Study of Machine Learning Algorithms for Phishing Website Detection,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 9, 2023, doi: 10.14569/IJACSA.2023.0140945.
[14] A. U. Rehman, I. Imtiaz, S. Javaid, and M. Muslih, “Real-Time Phishing URL Detection Using Machine Learning,” in The 7th International Global Conference Series on ICT Integration in Technical Education & Smart Society, Basel Switzerland: MDPI, Sep. 2025, p. 108. doi: 10.3390/engproc2025107108.
[15] M.-A. A. A. H. Qasim and N. A. Flayh, “Enhancing Phishing Website Detection via Feature Selection in URL-Based Analysis,” Informatica, vol. 47, no. 9, Dec. 2023, doi: 10.31449/inf.v47i9.5177.
[16] Ismi Rosia Dwianti et al., “Klasifikasi Email Phishing Menggunakan Metode TF-IDF dan Algoritma Random Forest,” ROUTERS: Jurnal Sistem dan Teknologi Informasi, pp. 125–135, Jul. 2025, doi: 10.25181/rt.v3i2.4310.
[17] D. Mustafa Abdullah and A. Mohsin Abdulazeez, “Machine Learning Applications based on SVM Classification A Review,” Qubahan Academic Journal, vol. 1, no. 2, pp. 81–90, Apr. 2021, doi: 10.48161/qaj.v1n2a50.
[18] E. Gandotra, D. Gupta, M. Dhalaria, N. Gupta, and S. Redkar, “PhishDetect: A ranking-based classifier integration approach for improving phishing website detection,” Sci. Rep., Jun. 2026, doi: 10.1038/s41598-026-57807-5.
[19] A. Jain et al., “PhishNet 1.0: optuna-optimized stacking ensemble with Boruta-based feature selection for phishing URL detection,” Sci. Rep., vol. 16, no. 1, p. 1480, Dec. 2025, doi: 10.1038/s41598-025-31447-7.
[20] R. S. Rao, C. Kondaiah, A. R. Pais, and B. Lee, “A hybrid super learner ensemble for phishing detection on mobile devices,” Sci. Rep., vol. 15, no. 1, p. 16839, May 2025, doi: 10.1038/s41598-025-02009-8.
[21] H. Ghalechyan, E. Israyelyan, A. Arakelyan, G. Hovhannisyan, and A. Davtyan, “Phishing URL detection with neural networks: an empirical study,” Sci. Rep., vol. 14, no. 1, p. 25134, Oct. 2024, doi: 10.1038/s41598-024-74725-6.
[22] J. Zhou et al., “An integrated CSPPC and BiLSTM framework for malicious URL detection,” Sci. Rep., vol. 15, no. 1, p. 6659, Feb. 2025, doi: 10.1038/s41598-025-91148-z.
[23] R. Gobinath and S. Manikandan, “Deep learning-based phishing classification framework for accurate detection using optimized URL intelligence,” Sci. Rep., vol. 16, no. 1, p. 15794, Apr. 2026, doi: 10.1038/s41598-026-46481-2.
[24] Q. E. ul Haq, M. H. Faheem, and I. Ahmad, “Detecting Phishing URLs Based on a Deep Learning Approach to Prevent Cyber-Attacks,” Applied Sciences, vol. 14, no. 22, p. 10086, Nov. 2024, doi: 10.3390/app142210086.
[25] F. Türk and M. Kılıçaslan, “Malicious URL Detection with Advanced Machine Learning and Optimization-Supported Deep Learning Models,” Applied Sciences, vol. 15, no. 18, p. 10090, Sep. 2025, doi: 10.3390/app151810090.
[26] T. Kehkashan et al., “Explainable phishing website detection for secure and sustainable cyber infrastructure,” Sci. Rep., vol. 15, no. 1, p. 41751, Nov. 2025, doi: 10.1038/s41598-025-27984-w.
[27] A. Vulfin, A. Sulavko, V. Vasiliev, A. Minko, A. Kirillova, and A. Samotuga, “A Multimodal Phishing Website Detection System Using Explainable Artificial Intelligence Technologies,” Mach. Learn. Knowl. Extr., vol. 8, no. 1, p. 11, Jan. 2026, doi: 10.3390/make8010011.
[28] G. Goldenits, P. König, S. Raubitzek, and A. Ekelhart, “Small Language Models for Phishing Website Detection: Cost, Performance, and Privacy Trade-Offs,” Journal of Cybersecurity and Privacy, vol. 6, no. 2, p. 48, Mar. 2026, doi: 10.3390/jcp6020048.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Nasir Usman, Muhammad Faisal, Alvina Felicia Watratan , Emil Agusalim Habi Talib, Nurahmad
This work is licensed under a Creative Commons Attribution 4.0 International License.
