Comparative Evaluation of VAEs, VAE-GANs and AAEs for Anomaly Detection in Network Intrusion Data
Abstract
With cyberattacks growing in frequency and sophistication, effective anomaly detection is critical for securing networks and systems. This study provides a comparative evaluation of deep generative models for detecting anomalies in network intrusion data. The key objective is to determine the most accurate model architecture. Variational autoencoders (VAEs), VAE-GANs, and adversarial autoencoders (AAEs) are tested on the NSL-KDD dataset containing normal traffic and different attack types. Results show that AAEs significantly outperform VAEs and VAE-GANs, achieving AUC scores up to 0.96 and F1 scores of 0.76 on novel attacks. The adversarial regularization of AAEs enables superior generalization capabilities compared to standard VAEs. VAE-GANs exhibit better accuracy than VAEs, demonstrating the benefits of adversarial training. However, VAE-GANs have higher computational requirements. The findings provide strong evidence that AAEs are the most effective deep anomaly detection technique for intrusion detection systems. This study delivers novel insights into optimizing deep learning architectures for cyber defense. The comparative evaluation methodology and results will aid researchers and practitioners in selecting appropriate models for operational network security.
Downloads
References
V. Chandola, A. Banerjee, and V. Kumar, Anomaly detection: A survey. ACM computing surveys (CSUR), Vol 41, No. 3, pp. 1-58, Jul 2009. DOI: https://doi.org/10.1145/1541880.1541882
A.L. Buczak, and E. Guven, A survey of data mining and machine learning methods for cyber security intrusion detection IEEE Communications surveys & tutorials, Vol. 18, No. 2, pp. 1153-1176, Apr 2016. DOI: https://doi.org/10.1109/COMST.2015.2494502
A. Javaid, Q. Niyaz, W. Sun, and M. Alam, A deep learning approach for network intrusion detection system, in Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (Formerly BIONETICS), pp. 21-26, Dec 2016. DOI: https://doi.org/10.4108/eai.3-12-2015.2262516
M. Sakurada, and T. Yairi, Anomaly detection using autoencoders with nonlinear dimensionality reduction, in Proceedings of the MLSDA 2014 2nd workshop on machine learning for sensory data analysis, pp. 4-11, Jul 2014. DOI: https://doi.org/10.1145/2689746.2689747
D.P. Kingma, and M. Welling, Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, Dec 2013.
A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, Adversarial autoencoders. arXiv preprint arXiv:1511.05644, Nov 2015.
A.B. Larsen, S.K. Sønderby, H. Larochelle, and O. Winther, Autoencoding beyond pixels using a learned similarity metric, in International Conference on Machine Learning (PMLR), pp. 1558-1566, Jun 2016.
Y. Tang, Y. Wang, Y. Wang, and B. Gao, Integrating Variational Autoencoder with Generative Adversarial Network for Anomaly Detection, IEEE International Conference on Multimedia and Expo (ICME), 2020.
S.M. Erfani, S. Rajasegarar, S. Karunasekera, and C. Leckie, High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning, Pattern Recognition, Vol. 58, pp. 121-134, 2016. DOI: https://doi.org/10.1016/j.patcog.2016.03.028
J. An, and S. Cho, Variational autoencoder based anomaly detection using reconstruction probability, Special Lecture on IE, Vol. 2, No.1, pp. 1-8, Dec 2015.
C. Yin, Y. Zhu, J. Fei, and X. He, A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks, IEEE Access. Vol. 5, pp. 21954-21961, 2017. DOI: https://doi.org/10.1109/ACCESS.2017.2762418
S.K. Lim, Y. Loo, N.T. Tran, N.M. Cheung, G. Roig, and Y. Elovici, DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN. arXiv preprint arXiv:1904.13215, 2015.
S. Akcay, A. Atapour-Abarghouei, and T.P. Breckon, GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training. Asian Conference on Computer Vision, pp. 622-637, 2019.
H. Zenati, M. Romain, C.S. Foo, B. Lecouat, and V. Chandrasekhar, Efficient GAN-Based Anomaly Detection, Workshop on Mining and Learning from Time Series (ICLR), 2018.
X. Li, Y. Li, R. Wang, L. Zhang, and P. Wang, Adversarial examples detection in deep networks with convolutional filter statistics, in Proceedings of the IEEE International Conference on Computer Vision, pp. 5764-5772, 2019.
M. Tavallaee, E. Bagheri, W. Lu, and A.A. Ghorbani, A detailed analysis of the KDD CUP 99 data set, in 2009 IEEE symposium on computational intelligence for security and defense applications, pp. 1-6, 2009. DOI: https://doi.org/10.1109/CISDA.2009.5356528
J. An, and S. Cho, Variational Autoencoder based Anomaly Detection using Reconstruction Probability. SNU Data Mining Center, 2020.
S. Mukkamala, G. Janoski, and A. Sung, Intrusion detection using neural networks and support vector machines, in Proceedings of the 2002 International Joint Conference on Neural Networks (IJCNN’02), Vol. 2, pp. 1702-1707, May 2002.
Y. Liao, and V.R. Vemuri, Use of k-nearest neighbor classifier for intrusion detection, Computers & Security, Vol. 21, No. 5, pp. 439-448, 2002. DOI: https://doi.org/10.1016/S0167-4048(02)00514-X
J. Zhang, M. Zulkernine, and A. Haque, Random-forests-based network intrusion detection systems, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 38, No. 5, pp. 649-659, 2008. DOI: https://doi.org/10.1109/TSMCC.2008.923876
B. Zong, Q. Song, M.R. Min, W. Cheng., C. Lumezanu, D. Cho, and H. Chen, Deep autoencoding Gaussian mixture model for unsupervised anomaly detection, International Conference on Learning Representations, 2018.
M. Barron, and G. Wornell, Variational autoencoders for generative adversarial networks, arXiv preprint arXiv:1803.05449, 2018.
W. Al-Yaseen, Z.A. Othman, and M.Z.A. Nazri, Multi-level hybrid support vector machine and extreme learning machine based on modified K-means for intrusion detection system, Expert Systems with Applications. Vol. 67, pp. 296-303, 2017. DOI: https://doi.org/10.1016/j.eswa.2016.09.041
R. Chalapathy, A.K. Menon, A, and S. Chawla, Anomaly Detection with Robust Deep Auto-encoders, International Conference SIGKDD, 2019.
J. Kim, and C.D. Scott, Robust Kernel Density Estimation by Scaling and Projection in the Hilbert Space, Advances in Neural Information Processing Systems, 2014.
D. Abati, Porrello, A., Calderara, S., & Cucchiara, R. (2019). Latent space autoregression for novelty detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 481-490, 2019. DOI: https://doi.org/10.1109/CVPR.2019.00057
F. Ghasemi, A. Karbalayghareh, M.R. Aghamohammadi, Intrusion detection using a novel hybrid deep autoencoder based on hyper-parameter optimization and stacking ensemble learning, Applied Intelligence, Vol. 51, No. 1, pp. 498-513, 2021.
I. Golan, and R. El-Yaniv, Deep Anomaly Detection Using Geometric Transformations, Advances in Neural Information Processing Systems 31 (NeurIPS 2018).
Y. Yamanaka, M. Iwamura, and K. Kise, Autoencoding Binary Classifiers for Supervised Anomaly Detection. arXiv preprint arXiv:1809.10816, 2018.
S. Kim, S. Park, Anomaly detection for industrial control systems using autoencoder based deep learning, in Asian Conference on Intelligent Information and Database Systems, Springer, Cham, pp. 441-449, 2019.
A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, Adversarial autoencoders. arXiv preprint arXiv:1511.05644, 2015.
T.A. Tang, L. Mhamdi, D. McLernon, S.A.R. Zaidi, and M. Ghogho, Deep Learning Approach for Network Intrusion Detection in Software Defined Networking, International Conference on Wireless Networks and Mobile Communications (WINCOM), 2016. DOI: https://doi.org/10.1109/WINCOM.2016.7777224
D. Chen, X. Song, J. Ni, Z. Zhao, A VAE and GAN combined network for anomaly detection on industrial control system, in Proceedings of the 2019 3rd International Conference on Big Data Technologies, pp. 54-59, 2019.
M. Sabokrou, M. Khalooei, M. Fathy, and E. Adeli, Adversarially Learned One-Class Classifier for Novelty Detection, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3379- 3388, 2018. DOI: https://doi.org/10.1109/CVPR.2018.00356
P. Perera, R. Nallapati, and B. Xiang, OCGAN: One-class novelty detection using GANs with constrained latent representations. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2898- 2906, 2019. DOI: https://doi.org/10.1109/CVPR.2019.00301
L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S.A. Siddiqui, A. Binder, E. Müller, and M. Kloft, Deep one-class classification, in Proceeding of Machine Learning Research (PMLR), Vol. 80, 2018.
S. Akcay, A. Atapour-Abarghouei, and T.P. Breckon, GANomaly: Semi-supervised anomaly detection via adversarial training, in Computer Vision ACCV 2018, pp. 622-637, 2018. DOI: https://doi.org/10.1007/978-3-030-20893-6_39
S. Ding, X. Xu, R. Nie, Extreme learning machine and its applications, Neural Computing and Applications, Vol. 25, No. 3-4, pp. 549-557, 2014. DOI: https://doi.org/10.1007/s00521-013-1522-8
S.K. Lim, Y. Loo, N.T. Tran, N.M. Cheung, G. Roig, and Y. Elovici, DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN. arXiv preprint arXiv:1904.13215, 2015.
R. Perdisci, D. Ariu, P. Fogla, G. Giacinto, W. Lee, McPAD: A multiple classifier system for accurate payload-based anomaly detection. Computer Networks, Vol. 53, No. 6, pp. 864-881, 2009. DOI: https://doi.org/10.1016/j.comnet.2008.11.011
H. Zenati, M. Romain, C.S. Foo, B. Lecouat, and V. Chandrasekhar, Efficient GAN-Based Anomaly Detection. arXiv preprint arXiv:1802.06222, 2018.
L. Khan, M. Awad, and B. Thuraisingham, A new intrusion detection system using support vector machines and hierarchical clustering The VLDB Journal, Vol. 16, No. 4, pp. 507-521, 2007. DOI: https://doi.org/10.1007/s00778-006-0002-5
J. Snoek, H. Larochelle, and R.P. Adams, Practical bayesian optimization of machine learning algorithms, Advances in neural information processing systems. Vol 16, pp. 2951-2959, 2012.
S. Usman, I. Winarno, and A. Sudarsono, SDN-Based Network Intrusion Detection as DDoS defense system for Virtualization Environment, EMITTER International Journal of Engineering Technology, vol. 9, no. 2, pp. 252–267, 2021. DOI: https://doi.org/10.24003/emitter.v9i2.616
Copyright (c) 2023 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
