Open Access Open Access  Restricted Access Subscription or Fee Access

A Comparative Study of Current Dataset Used to Evaluate Intrusion Detection System


(*) Corresponding author


Authors' affiliations


DOI: https://doi.org/10.15866/irea.v10i5.21030

Abstract


Today, since the world increasingly depends on computers and automation, securing applications and systems has become critical. The possibility of threat and violence is expanding rapidly due to the growing complexity of modern networks and services. This study focuses on current datasets, the gaps and limitations of these datasets, and their effects on creating Intrusion Detection Systems (IDS). The study outlines the existing datasets that show an apparent absence of real-world threats, attack representation, and outdated threats, limiting the efficiency of machine learning IDS techniques. This work presents the most Machine Learning techniques for implementing IDS and a comprehensive review of the available benchmark datasets. We believe this review will enable future researchers in this field to understand better different kinds of IDS datasets, advantages, and limitations.
Copyright © 2022 Praise Worthy Prize - All rights reserved.

Keywords


Anomaly; Intrusion Detection System; Cybersecurity; Machine Learning; Datasets

Full Text:

PDF


References


Pawlicki, M., Choraś, M., &Kozik, R. (2020). Defending network intrusion detection systems against adversarial evasion attacks. Future Generation Computer Systems, 110, 148-154.
https://doi.org/10.1016/j.future.2020.04.013

Ferrag, M. A., Maglaras, L., Moschoyiannis, S., &Janicke, H. (2020). Deep learning for cybersecurity intrusion detection: Approaches, datasets, and comparative study. Journal of Information Security and Applications, 50, 102419.
https://doi.org/10.1016/j.jisa.2019.102419

Min, E., Long, J., Liu, Q., Cui, J., & Chen, W. (2018). TR-IDS: Anomaly-based intrusion detection through text-convolutional neural network and random forest. Security and Communication Networks, 2018.
https://doi.org/10.1155/2018/4943509

Li, Z., Rios, A. L. G., Xu, G., &Trajković, L. (2019, May). Machine learning techniques for classifying network anomalies and intrusions. In 2019 IEEE international symposium on circuits and systems (ISCAS) (pp. 1-5). IEEE.
https://doi.org/10.1109/ISCAS.2019.8702583

Tavallaee, M., Bagheri, E., Lu, W., &Ghorbani, A. A. (2009, July). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE symposium on computational intelligence for security and defense applications (pp. 1-6). IEEE.
https://doi.org/10.1109/CISDA.2009.5356528

McHugh, J. (2000). Testing intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by the lincoln laboratory. ACM Transactions on Information and System Security (TISSEC), 3(4), 262-294.
https://doi.org/10.1145/382912.382923

Thakkar, A., &Lohiya, R. (2020). A Review of the Advancement in Intrusion Detection Datasets. Procedia Computer Science, 167, 636-645.
https://doi.org/10.1016/j.procs.2020.03.330

Kovács, G., Sebestyen, G., &Hangan, A. (2019). Evaluation metrics for anomaly detection algorithms in time-series. Acta Universitatis Sapientiae, Informatica, 11(2), 113-130.
https://doi.org/10.2478/ausi-2019-0008

Hindy, H., Brosset, D., Bayne, E., Seeam, A. K., Tachtatzis, C., Atkinson, R., &Bellekens, X. (2020). A taxonomy of network threats and the effect of current datasets on intrusion detection systems. IEEE Access, 8, 104650-104675.
https://doi.org/10.1109/ACCESS.2020.3000179

Scarfone, K., & Mell, P. (2007). Guide to intrusion detection and prevention systems (idps). NIST special publication, 800(2007), 94.
https://doi.org/10.6028/NIST.SP.800-94

Ashoor, A. S., & Gore, S. (2011). Importance of intrusion detection system (IDS). International Journal of Scientific and Engineering Research, 2(1), 1-4.

Liu, I. H., Lo, C. H., Liu, T. C., Li, J. S., Liu, C. G., & Li, C. F. (2020). IDS Malicious Flow Classification. Journal of Robotics, Networking and Artificial Life, 7(2), 103-106.
https://doi.org/10.2991/jrnal.k.200528.006

Anderson, J. P. (1980). Computer security threat monitoring and surveillance. Technical Report, James P. Anderson Company.

Bhattacharyya, D. K., &Kalita, J. K. (2013). Network anomaly detection: A machine learning perspective. CRC Press.
https://doi.org/10.1201/b15088

Ali, W. A., Manasa, K. N., Aljunid, M., Bendechache, M., &Sandhya, P. (2020). A Review of Current Machine Learning Approaches for Anomaly Detection in Network Traffic. Journal of Telecommunications and the Digital Economy, 8(4), 64-95.
https://doi.org/10.18080/jtde.v8n4.307

Chakir, E. M., Moughit, M., &Khamlichi, Y. I. (2018). An Effective Intrusion Detection Model Based On SVM With Feature Selection And Parameters Optimization. Journal of Theoretical & Applied Information Technology, 96(12).

M. M. Min and Y. Y. Aung, An Analysis of K-means Algorithm Based Network Intrusion Detection System. Advances in Science Technology and Engineering Systems Journal, vol 3, pp 496-501.
https://doi.org/10.25046/aj030160

Thakare, Y. S., &Bagal, S. B. (2015). Performance evaluation of K-means clustering algorithm with various distance metrics. International Journal of Computer Applications, 110(11), 12-16.
https://doi.org/10.5120/19360-0929

Kherbache, M., Espes, D., &Amroun, K. (2021, July). An Enhanced approach of the K-means clustering for Anomaly-based intrusion detection systems. In 2021 International Conference on Computing, Computational Modelling and Applications (ICCMA) (pp. 78-83). IEEE.
https://doi.org/10.1109/ICCMA53594.2021.00021

Kamal Idrissi, H., Kartit, Z., Kartit, A., El Marraki, M., CKMSA: an Anomaly Detection Process Based on K-Means and Simulated Annealing Algorithms, (2016) International Review on Computers and Software (IRECOS), 11 (1), pp. 42-48.
https://doi.org/10.15866/irecos.v11i1.8272

E. Besharati, M. Naderan, and E. Namjoo, LR-HIDS: logistic regression host-based intrusion detection system for cloud environments, Journal of Ambient Intelligence and Humanized Computing volume, vol. 10, p. 3669-3692, 2019.
https://doi.org/10.1007/s12652-018-1093-8

Shah, R. A., Qian, Y., Kumar, D., Ali, M., &Alvi, M. B. (2017). Network intrusion detection through discriminative feature selection by using sparse logistic regression. Future Internet, 9(4), 81.
https://doi.org/10.3390/fi9040081

Meftah, S., Rachidi, T., &Assem, N. (2019). Network based intrusion detection using the UNSW-NB15 dataset. International Journal of Computing and Digital Systems, 8(5), 478-487.
https://doi.org/10.12785/ijcds/080505

B. S. Sharmila and R. Nagapadma, Intrusion Detection System using Naive Bayes algorithm. In IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE), 2019.
https://doi.org/10.1109/WIECON-ECE48653.2019.9019921

Mehmood, A., Mukherjee, M., Ahmed, S. H., Song, H., & Malik, K. M. (2018). NBC-MAIDS: Naïve Bayesian classification technique in multi-agent system-enriched IDS for securing IoT against DDoS attacks. The Journal of Supercomputing, 74(10), 5156-5170.
https://doi.org/10.1007/s11227-018-2413-7

Wu, K., Chen, Z., & Li, W. (2018). A novel intrusion detection model for a massive network using convolutional neural networks. IEEE Access, 6, 50850-50859
https://doi.org/10.1109/ACCESS.2018.2868993

Shatnawi, M., Bani Yassein, M., Aljawarneh, S., Alodibat, S., Meqdadi, O., Hmeidi, I., Al Zoubi, O., An Improvement of Neural Network Algorithm for Anomaly Intrusion Detection System, (2020) International Journal on Communications Antenna and Propagation (IRECAP), 10 (2), pp. 84-93.
https://doi.org/10.15866/irecap.v10i2.18735

Nguyen, M. T., & Kim, K. (2020). Genetic convolutional neural network for intrusion detection systems. Future Generation Computer Systems, 113, 418-427.
https://doi.org/10.1016/j.future.2020.07.042

Paffenroth, R., Kay, K., &Servi, L. (2018). Robust PCA for anomaly detection in cyber networks. arXiv preprint arXiv:1801.01571.

A. Khraisat, I. Gondal, P. Vamplewand J. Kamruzzaman, Survey of intrusion detection systems: techniques, datasets, and challenges, Cybersecurity, vol. 20, 2019.
https://doi.org/10.1186/s42400-019-0038-7

Brugger, T. (2007). KDD Cup'99 dataset (Network Intrusion) considered harmful. KDnuggets newsletter, 7(18), 15.

M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, A detailed analysis of the KDD CUP 99 data set, in IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009, pp. 1-6.
https://doi.org/10.1109/CISDA.2009.5356528

Moustafa, N., & Slay, J. (2015, November). UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In 2015 military communications and information systems conference (MilCIS) (pp. 1-6). IEEE.
https://doi.org/10.1109/MilCIS.2015.7348942

S. M. Kasongo and Y. Sun, Performance Analysis of Intrusion Detection Systems Using a Feature Selection Method on the UNSW-NB15 Dataset, Journal of Big Data, p. 105, 2020.
https://doi.org/10.1186/s40537-020-00379-6

Shiravi, A., Shiravi, H., Tavallaee, M., &Ghorbani, A. A. (2012). Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Computers & Security, 31(3), 357-374.
https://doi.org/10.1016/j.cose.2011.12.012

Sharafaldin, I., Lashkari, A. H., Hakak, S., &Ghorbani, A. A. (2019, October). Developing realistic distributed denial of service (DDoS) attack dataset and taxonomy. In 2019 International Carnahan Conference on Security Technology (ICCST) (pp. 1-8). IEEE.
https://doi.org/10.1109/CCST.2019.8888419

Hussain, F., Abbas, S. G., Husnain, M., Fayyaz, U. U., Shahzad, F., & Shah, G. A. (2020, November). IoT DoS and DDoS attack detection using ResNet. In 2020 IEEE 23rd International Multitopic Conference (INMIC) (pp. 1-6). IEEE.
https://doi.org/10.1109/INMIC50486.2020.9318216

Intrusion Detection Evaluation Dataset (CICIDS2017).
https://www.unb.ca/cic/datasets/ids2017.html

CSE-CIC-IDS2018 on AWS.
https://www.unb.ca /cic/datasets/ids-2018.html

Kim, J., Shin, Y., & Choi, E. (2019). An intrusion detection model based on a convolutional neural network. Journal of Multimedia Information System, 6(4), 165-172.
https://doi.org/10.33851/JMIS.2019.6.4.165

Sharafaldin, I., Gharib, A., Lashkari, A. H., &Ghorbani, A. A. (2018). Towards a reliable intrusion detection benchmark dataset. Software Networking, 2018(1), 177-200.
https://doi.org/10.13052/jsn2445-9739.2017.009

Koch, R., Golling, M., &Rodosek, G. D. (2014, May). Towards comparability of intrusion detection systems: New data sets. In TERENA Networking Conference (Vol. 7).

Sharafaldin, I., Lashkari, A. H., &Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISS, p, 1, 108-116.
https://doi.org/10.5220/0006639801080116

Vijayanand, R., Devaraj, D., &Kannapiran, B. (2018). Intrusion detection system for wireless mesh network using multiple support vector machine classifiers with genetic-algorithm-based feature selection. Computers & Security, 77, 304-314.
https://doi.org/10.1016/j.cose.2018.04.010

Ullah, S., Khan, M. A., Ahmad, J., Jamal, S. S., e Huma, Z., Hassan, M. T., ... & Buchanan, W. J. (2022). HDL-IDS: a hybrid deep learning architecture for intrusion detection in the Internet of Vehicles. Sensors, 22(4), 1340.
https://doi.org/10.3390/s22041340

Seth, S., Chahal, K. K., & Singh, G. (2021). A novel ensemble framework for an intelligent intrusion detection system. IEEE Access, 9, 138451-138467.
https://doi.org/10.1109/ACCESS.2021.3116219


Refbacks

  • There are currently no refbacks.


Please send any question about this web site to info@praiseworthyprize.com
Copyright © 2005-2024 Praise Worthy Prize