Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adversarial Environment
•Download as PPTX, PDF•
1 like•55 views
Network Intrusion Detection System (NIDS) is a method that is utilized to categorize network traffic as malicious or normal. Anomaly-based method and signature-based method are the traditional approaches used for network intrusion detection. The signature-based approach can only detect familiar attacks whereas the anomaly-based approach shows promising results in detecting new unknown attacks. Machine Learning (ML) based approaches have been studied in the past for anomaly-based NIDS. In recent years, the Deep Learning (DL) algorithms have been widely utilized for intrusion detection due to its capability to obtain optimal feature representation automatically. Even though DL based approaches improves the accuracy of the detection tremendously, they are prone to adversarial attacks. The attackers can trick the model to wrongly classify the adversarial samples into a particular target class. In this paper, the performance analysis of several ML and DL models are carried out for intrusion detection in both adversarial and non-adversarial environment. The models are trained on the NSLKDD dataset which contains a total of 148,517 data points. The robustness of several models against adversarial samples is studied.
Report
Share
Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adversarial Environment
- 1. Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adversarial Environment Sriram S1, Simran K1, Vinayakumar R2,1, Akarsh S1, Soman KP1 1 Center for computational engineering and networking, Amrita school of engineering, Amrita vishwa vidyapeetham, Coimbatore, India. sri27395ram@gmail.com, simiketha19@gmail.com 2 Division of biomedical informatics, Cincinnati children's hospital centre, Cincinnati, OH, United States. Vinayakumar.Ravi@cchmc.org, vinayakumarr77@gmail.com
- 2. • Motivation • Background information • Methodology • Results • Conclusion and Future work • References 2 Outline
- 3. Motivation • In today's world, cyber-attacks and threats on Information and Communication Technologies (ICT) systems are growing rapidly. Various security structures and methods are used to deal with these malicious attacks namely firewall, Intrusion Detection System (IDS), encryption and decryption methods, etc. • IDS plays a big role in defending the network from all kinds of intrusion and malicious acts, both from outside and inside the network. 3
- 4. Motivation (cont'd) • Various Machine Learning (ML) and Deep Learning (DL) based IDS are proposed by the researchers [1,2,3] and they have achieved good results. However, ML and DL models are prone to adversarial attacks. Attackers can fool the detection system by using adversarial samples and make the classifier misclassify those sample data [4]. • Therefore, it is necessary to check the robustness of ML and DL models that are used for critical security systems like IDS against adversarial attacks. In this work, many DL and ML models are trained on the NSLKDD dataset [5] and their robustness in the adversarial environment is studied. 4
- 5. 5 Background information • IDS is a tool that deals with unauthorized access and threats to systems and information by any type of user or software. • Two major categories of IDS are Host IDS and Network IDS. Host IDS is a tool that monitors the system in which it is installed to detect both external and internal intrusion, misuse and responds by recording activities and alerts the authority. • Network IDS is utilized to monitor and analyze network traffic to safeguard a system from network-based attacks. This work concentrates only on Network IDS.
- 7. • Fast Gradient Sign Method (FGSM) [6] creates adversarial samples by calculating a small deviation in the direction of the gradient and it is defined as follows: • In the above equation, p is the perturbation, ∈is a small constant, ▽xL(θ, x, y) is the gradient of loss function L which is used for training the model, x denotes the input and y denotes the class of input x. This perturbation is added to the input data to generate adversarial samples • FGSM is computationally more efficient when compared to JSMA. But it has a lower rate of success. 7 Methodology
- 8. 8 Methodology (cont'd) • Jacobian-based Saliency Map Attack (JSMA) [7] uses the concept of the saliency map to generate adversarial samples. • The saliency map rates each feature of how influential it is for causing the model to predict a target class. • JSMA causes the model to misclassify the resulting adversarial sample to a specific erroneous target class by modifying the high-saliency features.
- 9. 9 Methodology (cont'd) • The DL models that are trained in this work are 5 level Deep Neural Network (DNN), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM). • The ML classifiers that are trained in this work are Support Vector Machine (SVM), Naive Bayes (NB), K-Nearest Neighbour (KNN), Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Adaboost (AB). • The NSLKDD dataset contains a total of 148,517 network records that can be categorized into 5 different classes (normal and 4 attack traffic records).
- 10. 10 Results
- 13. 13 Results (cont'd) • Based on the accuracy metric, the DNN, CNN, and DT are the top three models in a non-adversarial environment. • The top three most affected models by FGSM in terms of accuracy are DNN, LSTM, and DT whereas the top three most affected models by JSMA are CNN, DNN, and DT. • The FGSM attack worked well in the case of LSTM and NB whereas the JSMA method worked better than FGSM in all other cases. • RBF SVM, Linear SVM, KNN, and NB are the models which show more robustness against both adversarial attacks when compared to the rest of the models.
- 14. 14 Conclusion and Future work • This work studies the robustness of many ML and DL models against adversarial attacks. • The adversarial samples that are created using the DNN model generalize well over other DL and ML models as well. In other words, the attack samples, which are created by both FGSM and JSMA for the DNN model as the target, also affected the performance of other ML and DL models. • In future more adversarial attack methods can be employed and the defense techniques against adversarial attacks can be studied.
- 15. [1] Tsai, C. F., Hsu, Y. F., Lin, C. Y., & Lin, W. Y. (2009). Intrusion detection by machine learning: A review. expert systems with applications, 36(10), 11994-12000. [2] Vinayakumar, R., Alazab, M., Soman, K. P., Poornachandran, P., Al- Nemrat, A., & Venkatraman, S. (2019). Deep Learning Approach for Intelligent Intrusion Detection System. IEEE Access, 7, 41525-41550. [3] Vinayakumar, R., Soman, K. P., & Poornachandran, P. (2017, September). Applying convolutional neural network for network intrusion detection. In 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (pp. 1222-1228). IEEE. [4] Rigaki, M. (2017). Adversarial deep learning against intrusion detection classifiers. 15 References
- 16. 16 References (cont'd) [5] Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009, July). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (pp. 1-6). IEEE. [6] Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. [7] Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016, March). The limitations of deep learning in adversarial settings. In 2016 IEEE European Symposium on Security and Privacy (EuroS&P) (pp. 372-387). IEEE.
- 17. 17 Thank you
- 18. 18 Appendix
- 20. 20 Performance of the models in non- adversarial environment
- 21. 21 Performance of the models tested on adversarial samples generated by FGSM
- 22. 22 Performance of the models tested on adversarial samples generated by JSMA