Regresar

Machine-learning-based online distributed denial-of-service attack detection using spark streaming

Abstract:

In order to cope with the increasing number of cyber attacks, network operators must monitor the whole network situations in real time. Traditional network monitoring method that usually works on a single machine, however, is no longer suitable for the huge traffic data nowadays due to its poor processing ability. In this paper, we propose a machine-learning based online Internet traffic monitoring system using Spark Streaming, a stream- processing-based big data framework, to detect DDoS attacks in real time. The system consists of three parts, collector, messaging system and stream processor. We use a correlation-based feature selection method and choose 4 most necessary network features in our machine- learning-based DDoS detection algorithm. We verify the result of feature selection method by a comparative experiment and compare the detection accuracy of 3 machine learning methods - Naive Bayes, Logistic Regression and Decision Tree. Finally, we conduct experiments in a cluster with the standalone mode, showing that our system can detect 3 typical DDoS attacks - TCP flooding, UDP flooding and ICMP flooding at the accuracy of more than 99.3%. It also shows the system performs well even for large Internet traffic.