Open Access Open Access  Restricted Access Subscription or Fee Access

A Comparative Study of Current Dataset Used to Evaluate Intrusion Detection System

(*) Corresponding author

Authors' affiliations



Today, since the world increasingly depends on computers and automation, securing applications and systems has become critical. The possibility of threat and violence is expanding rapidly due to the growing complexity of modern networks and services. This study focuses on current datasets, the gaps and limitations of these datasets, and their effects on creating Intrusion Detection Systems (IDS). The study outlines the existing datasets that show an apparent absence of real-world threats, attack representation, and outdated threats, limiting the efficiency of machine learning IDS techniques. This work presents the most Machine Learning techniques for implementing IDS and a comprehensive review of the available benchmark datasets. We believe this review will enable future researchers in this field to understand better different kinds of IDS datasets, advantages, and limitations.
Copyright © 2022 Praise Worthy Prize - All rights reserved.


Anomaly; Intrusion Detection System; Cybersecurity; Machine Learning; Datasets

Full Text:



Pawlicki, M., Choraś, M., &Kozik, R. (2020). Defending network intrusion detection systems against adversarial evasion attacks. Future Generation Computer Systems, 110, 148-154.

Ferrag, M. A., Maglaras, L., Moschoyiannis, S., &Janicke, H. (2020). Deep learning for cybersecurity intrusion detection: Approaches, datasets, and comparative study. Journal of Information Security and Applications, 50, 102419.

Min, E., Long, J., Liu, Q., Cui, J., & Chen, W. (2018). TR-IDS: Anomaly-based intrusion detection through text-convolutional neural network and random forest. Security and Communication Networks, 2018.

Li, Z., Rios, A. L. G., Xu, G., &Trajković, L. (2019, May). Machine learning techniques for classifying network anomalies and intrusions. In 2019 IEEE international symposium on circuits and systems (ISCAS) (pp. 1-5). IEEE.

Tavallaee, M., Bagheri, E., Lu, W., &Ghorbani, A. A. (2009, July). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE symposium on computational intelligence for security and defense applications (pp. 1-6). IEEE.

McHugh, J. (2000). Testing intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by the lincoln laboratory. ACM Transactions on Information and System Security (TISSEC), 3(4), 262-294.

Thakkar, A., &Lohiya, R. (2020). A Review of the Advancement in Intrusion Detection Datasets. Procedia Computer Science, 167, 636-645.

Kovács, G., Sebestyen, G., &Hangan, A. (2019). Evaluation metrics for anomaly detection algorithms in time-series. Acta Universitatis Sapientiae, Informatica, 11(2), 113-130.

Hindy, H., Brosset, D., Bayne, E., Seeam, A. K., Tachtatzis, C., Atkinson, R., &Bellekens, X. (2020). A taxonomy of network threats and the effect of current datasets on intrusion detection systems. IEEE Access, 8, 104650-104675.

Scarfone, K., & Mell, P. (2007). Guide to intrusion detection and prevention systems (idps). NIST special publication, 800(2007), 94.

Ashoor, A. S., & Gore, S. (2011). Importance of intrusion detection system (IDS). International Journal of Scientific and Engineering Research, 2(1), 1-4.

Liu, I. H., Lo, C. H., Liu, T. C., Li, J. S., Liu, C. G., & Li, C. F. (2020). IDS Malicious Flow Classification. Journal of Robotics, Networking and Artificial Life, 7(2), 103-106.

Anderson, J. P. (1980). Computer security threat monitoring and surveillance. Technical Report, James P. Anderson Company.

Bhattacharyya, D. K., &Kalita, J. K. (2013). Network anomaly detection: A machine learning perspective. CRC Press.

Ali, W. A., Manasa, K. N., Aljunid, M., Bendechache, M., &Sandhya, P. (2020). A Review of Current Machine Learning Approaches for Anomaly Detection in Network Traffic. Journal of Telecommunications and the Digital Economy, 8(4), 64-95.

Chakir, E. M., Moughit, M., &Khamlichi, Y. I. (2018). An Effective Intrusion Detection Model Based On SVM With Feature Selection And Parameters Optimization. Journal of Theoretical & Applied Information Technology, 96(12).

M. M. Min and Y. Y. Aung, An Analysis of K-means Algorithm Based Network Intrusion Detection System. Advances in Science Technology and Engineering Systems Journal, vol 3, pp 496-501.

Thakare, Y. S., &Bagal, S. B. (2015). Performance evaluation of K-means clustering algorithm with various distance metrics. International Journal of Computer Applications, 110(11), 12-16.

Kherbache, M., Espes, D., &Amroun, K. (2021, July). An Enhanced approach of the K-means clustering for Anomaly-based intrusion detection systems. In 2021 International Conference on Computing, Computational Modelling and Applications (ICCMA) (pp. 78-83). IEEE.

Kamal Idrissi, H., Kartit, Z., Kartit, A., El Marraki, M., CKMSA: an Anomaly Detection Process Based on K-Means and Simulated Annealing Algorithms, (2016) International Review on Computers and Software (IRECOS), 11 (1), pp. 42-48.

E. Besharati, M. Naderan, and E. Namjoo, LR-HIDS: logistic regression host-based intrusion detection system for cloud environments, Journal of Ambient Intelligence and Humanized Computing volume, vol. 10, p. 3669-3692, 2019.

Shah, R. A., Qian, Y., Kumar, D., Ali, M., &Alvi, M. B. (2017). Network intrusion detection through discriminative feature selection by using sparse logistic regression. Future Internet, 9(4), 81.

Meftah, S., Rachidi, T., &Assem, N. (2019). Network based intrusion detection using the UNSW-NB15 dataset. International Journal of Computing and Digital Systems, 8(5), 478-487.

B. S. Sharmila and R. Nagapadma, Intrusion Detection System using Naive Bayes algorithm. In IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE), 2019.

Mehmood, A., Mukherjee, M., Ahmed, S. H., Song, H., & Malik, K. M. (2018). NBC-MAIDS: Naïve Bayesian classification technique in multi-agent system-enriched IDS for securing IoT against DDoS attacks. The Journal of Supercomputing, 74(10), 5156-5170.

Wu, K., Chen, Z., & Li, W. (2018). A novel intrusion detection model for a massive network using convolutional neural networks. IEEE Access, 6, 50850-50859

Shatnawi, M., Bani Yassein, M., Aljawarneh, S., Alodibat, S., Meqdadi, O., Hmeidi, I., Al Zoubi, O., An Improvement of Neural Network Algorithm for Anomaly Intrusion Detection System, (2020) International Journal on Communications Antenna and Propagation (IRECAP), 10 (2), pp. 84-93.

Nguyen, M. T., & Kim, K. (2020). Genetic convolutional neural network for intrusion detection systems. Future Generation Computer Systems, 113, 418-427.

Paffenroth, R., Kay, K., &Servi, L. (2018). Robust PCA for anomaly detection in cyber networks. arXiv preprint arXiv:1801.01571.

A. Khraisat, I. Gondal, P. Vamplewand J. Kamruzzaman, Survey of intrusion detection systems: techniques, datasets, and challenges, Cybersecurity, vol. 20, 2019.

Brugger, T. (2007). KDD Cup'99 dataset (Network Intrusion) considered harmful. KDnuggets newsletter, 7(18), 15.

M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, A detailed analysis of the KDD CUP 99 data set, in IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009, pp. 1-6.

Moustafa, N., & Slay, J. (2015, November). UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In 2015 military communications and information systems conference (MilCIS) (pp. 1-6). IEEE.

S. M. Kasongo and Y. Sun, Performance Analysis of Intrusion Detection Systems Using a Feature Selection Method on the UNSW-NB15 Dataset, Journal of Big Data, p. 105, 2020.

Shiravi, A., Shiravi, H., Tavallaee, M., &Ghorbani, A. A. (2012). Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Computers & Security, 31(3), 357-374.

Sharafaldin, I., Lashkari, A. H., Hakak, S., &Ghorbani, A. A. (2019, October). Developing realistic distributed denial of service (DDoS) attack dataset and taxonomy. In 2019 International Carnahan Conference on Security Technology (ICCST) (pp. 1-8). IEEE.

Hussain, F., Abbas, S. G., Husnain, M., Fayyaz, U. U., Shahzad, F., & Shah, G. A. (2020, November). IoT DoS and DDoS attack detection using ResNet. In 2020 IEEE 23rd International Multitopic Conference (INMIC) (pp. 1-6). IEEE.

Intrusion Detection Evaluation Dataset (CICIDS2017).

CSE-CIC-IDS2018 on AWS. /cic/datasets/ids-2018.html

Kim, J., Shin, Y., & Choi, E. (2019). An intrusion detection model based on a convolutional neural network. Journal of Multimedia Information System, 6(4), 165-172.

Sharafaldin, I., Gharib, A., Lashkari, A. H., &Ghorbani, A. A. (2018). Towards a reliable intrusion detection benchmark dataset. Software Networking, 2018(1), 177-200.

Koch, R., Golling, M., &Rodosek, G. D. (2014, May). Towards comparability of intrusion detection systems: New data sets. In TERENA Networking Conference (Vol. 7).

Sharafaldin, I., Lashkari, A. H., &Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISS, p, 1, 108-116.

Vijayanand, R., Devaraj, D., &Kannapiran, B. (2018). Intrusion detection system for wireless mesh network using multiple support vector machine classifiers with genetic-algorithm-based feature selection. Computers & Security, 77, 304-314.

Ullah, S., Khan, M. A., Ahmad, J., Jamal, S. S., e Huma, Z., Hassan, M. T., ... & Buchanan, W. J. (2022). HDL-IDS: a hybrid deep learning architecture for intrusion detection in the Internet of Vehicles. Sensors, 22(4), 1340.

Seth, S., Chahal, K. K., & Singh, G. (2021). A novel ensemble framework for an intelligent intrusion detection system. IEEE Access, 9, 138451-138467.


  • There are currently no refbacks.

Please send any question about this web site to
Copyright © 2005-2024 Praise Worthy Prize