Using Neuro-Fuzzy Techniques to Reduce False Alerts in IDS Pravesh Gaonjur, N.Z. Tarapore, and S.G. Pukale

Abstract—The problems related to security for network systems are relative to the design of network architectures, which is typically based on open standards. Monitoring tools based on pattern recognition or behavioral analysis is typically used to ensure network security. SNORT is one such tool which is based on pattern recognition. SNORT alerts system administrators whenever it receives packets of information that match predetermined signatures contained in the SNORT ruleset, thereby protecting network systems. Unfortunately, due to the nature of this design, SNORT operates at the packet level and thereby has no concept of the specific properties of the network it is trying to protect. This paper provides the analysis of NEFCLASS and JRip which, upon taking SNORT alerts as input and learning from training, attempts to reduce false-positive and negative alerts sent to the system administrator. The major drawback of SNORT is the amount of false alerts generated by the SNORT engine, which must then be analyzed and classified by system administrators. This paper demonstrates that Neuro-Fuzzy Classifiers can be used to lessen this burden and considerably reduce the workload of having to classify alerts by human beings. Keywords—IDS, Security, Networks, False Alerts, Neuro-Fuzzy, JRip.

I. I NTRODUCTION

T

He IDS looked at most closely in this paper, SNORT, is a rules-based network intrusion detection system (NIDS). Martin Roesch, in his paper entitled “SNORT - Lightweight Intrusion Detection for Networks,” says “SNORT fills an important ecological niche in the realm of network security: a cross-platform, lightweight network intrusion detection tool that can be deployed to monitor small TCP/IP networks and detect a wide variety of suspicious network traffic as well as outright attacks”. The SANS Institute also reported SNORT as becoming the standard among intrusion detection experts due to the fact that it is open-source, frequently updated, and free of charge [17]. A. False Alerts Problem in SNORT One of the main problems in existing security sensors is their tendency of producing high rates of false positive logs and alerts. Often, a false alert is generated when in fact the event that triggered the alarm can be considered harmless. This condition is aggravated when the attacker has some

Pravesh Gaonjur is a Research Scholar from Mauritius, he is currently researching on IDS at the Department of Computer Engineering, Vishwakarma Institute of Technology, Pune, email: [email protected] N.Z. Tarapore and S.G. Pukale are Assistant Professors at the Department of Computer Engineering, Vishwakarma Institute of Technology, Pune, email: [email protected],[email protected]

prior knowledge of the techniques employed by the security sensor and thus purposely crafts network data to trigger these false alerts. This will not only allow an attacker to control the security sensors, but also overwhelm the ability of the security sensor to function properly due to the large amount of traffic that matches its rules or other triggering alert mechanisms, and hence wasting processing resources. Although an excellent tool, SNORT has three major drawbacks: • Packet Dropping • False Positive Alerts • False Negative Alerts SNORT may not pick up all packets due to speed issues with a network. Other factors which can affect SNORT in this way are the speed of the promiscuous interface and the stack implementation of the operating system. It is important to note that SNORT is able to be overrun with packet flooding which then makes the detection of intrusions more difficult. False positives occur when SNORT sends alerts when it shouldn’t, in other words a false alarm. This can happen for various reasons. Some of these include: • Placement of SNORT outside of the security perimeter: In this case SNORT receives DNS scans, web proxy scans and other various benign informational network that would cause overload for the system administrator. • Site Policy allowing activity that causes IDS alarms: For instance, using the default setting for SNORT which would increase the data inflow to an unmanageable level. • Lack of site awareness in the IDS: Not being aware of services running on hosts, such as IIS (Internet Information Services) attacks on Apache web servers could lead to false alarms. False Negatives occur because of any attack not matching a signature in the ‘known attack’ database. This can happen because of poor rule design, encrypted or otherwise cleverly [2] disguised traffic, or simply because the attack is new and has never been signature matched. B. Proposed Solution The proposed framework is based on Artificial Intelligence Techniques, which is expected to improve the percentage in the reduction of False Positive alerts. Also the framework should be able to cater to the main problem in the NeuroFuzzy Technique, which could not reduce the number of

False Negatives significantly enough. Objectives: 1) The first and foremost contribution is the design and implementation of an intelligent technique that allows the system (IDS) to reduce false alerts. 2) Secondly, the system should be fine tuned such that the number of False Negatives are also reduced. 3) Finally, an empirical comparison of the results obtained in tests conducted using the previously used technique and the improved technique is demonstrated.

II. A RTIFICIAL I NTELLIGENCE T ECHNIQUES Artificial Intelligence or AI as it is known has been around for quite some time. It is a field of computer science that attempts to mimic or copy human-type thinking and action. Unlike simple processing of information with selection statements and working memory, artificial intelligence attempts to replicate thought processes such as reasoning, intuition, learning from past trial and error, and generalizations [9]. Although difficult, some success in replication of human intelligence has been achieved by what are known as expert systems. Usually these systems reside on very powerful machines operating at extremely high speeds and the programs themselves are incredibly complex. Expert systems are actually in a class of artificial intelligence known as rule-based systems [9]. The more an intrusion detection system (IDS) knows about the network it is trying to protect, the better it will be able to protect the network. This is the fundamental principle behind target-based intrusion detection, where an IDS knows about the hosts on the network. SNORT is the IDS in question and this paper describes some of its features that users might not be taking advantage of that would allow the IDS to adapt to networks and detect anomalies. AI alleviates some of the security professionals’ work load by first learning about a network and gauging reactions from a security professional to reduce false positives, and second, by adapting to changes in the network to identify new attacks. There are several different soft computing techniques and algorithms that can be successfully used to detect intrusions. These techniques include [13]: • Fuzzy Logic • Probabilistic Reasoning • Neural Networks • Genetic Algorithms Combinations of these can also be used. For example, genetic algorithms can be used to build a neural network and probabilistic reasoning can be built on fuzzy logic. Neural networks are the most common AI type for an IDS [9]. Our

main focus will be on Fuzzy logic techniques since we are more concerned with what happens inside the AI logic, a black box implementation will not be helpful at all for us. A. NeuroFuzzy Systems A neuro-fuzzy network can be defined as a fuzzy system trained with some algorithm derived from the neural network theory. The integration of neural networks and fuzzy systems aims at the generation of a more robust, efficient and easily interpretable system where the advantages of each model are kept and their possible disadvantages are removed. Some neural network models such as the MLP [1] have been successfully applied to the training of neuro-fuzzy networks. The NEFCLASS model proposed by Nauck and Kruse [3] is based on a three-layer feedforward neural network [1] and the FuNN (Fuzzy Neural Network) proposed by Kasabov is a five-layer feedforward neural network. Both networks use modified versions of the back-propagation algorithm to adjust the membership functions (activation functions) and connection weights of the processing units. Modern neuro-fuzzy approaches are of this form: A neural network and a fuzzy system are combined into one homogeneous architecture. The system may be interpreted either as a special neural network with fuzzy parameters, or as a fuzzy system implemented in a parallel distributed form. Some of these approaches are reinforcement learning types that are especially suited for control tasks and others are multi-purpose models, which use supervised learning, and can be used for data analysis, like the NEFCLASS approach. As we are only interested here in hybrid neuro-fuzzy systems we restrict ourselves, in the further descriptions to information needed as a basis for this approach. NAUCK/KRUSE gives a definition that shall be used here to specify what a neurofuzzy system means in this paper: 1) A neuro-fuzzy system is a fuzzy system trained by a (heuristical) learning algorithm (usually) derived from neural networks. 2) A neuro-fuzzy system can be represented by a feedforward neural network architecture. However, this is not a prerequisite to training, it is merely a convenience to visualise the structure and the flow of data. 3) A neuro-fuzzy system can always be interpreted in terms of fuzzy if-then rules. 4) A neuro-fuzzy system’s training procedure takes the semantics of the underlying fuzzy model into account to preserve the linguistic interpretability of the model. 5) A neuro-fuzzy systems performs (special cases of) function approximation. It has nothing to do with fuzzy logic in the narrow sense. i.e. generalized logical rules. Figure 1 shows this neural network structure which is often used to demonstrate the parallel structure and the data flow through the model, both for learning (backward path) and classification (forward path). Furthermore it is easier to compare NEFCLASS to other fuzzy classification approaches if this

representation is chosen. But it also should be remembered again that this is only one possible visualisation. This system is not a neural network. It is a hybrid neuro-fuzzy system which is an integrated system.

Fig. 1.

Neural Network Structure.

B. RIPPER RIPPER was developed by William Cohen [5] based on repeated application of Furnkranz and Widmer’s IREP algorithm followed by two new global optimization procedures. Like other rule-based learners, RIPPER grows rules in a greedy fashion guided by an information gain heuristic. It is comparable in accuracy to similar algorithms such as C4.5 rules, but is significantly more efficient. This efficiency combined with RIPPER’s implementation of setvalued features allows learning in much larger feature spaces than would be possible with C4.5 rules. RIPPER has already been applied to a number of standard problems in classification with quite promising results [5]. It is important to emphasize that RIPPER is a rule-based machine learning system that has made its mark in a field dominated by purely statistical algorithms such as Nave Bayes, WidrowHoff, or K-Nearest Neighbor. The high dimensionality of most representations of data has in the past lead researchers away from rule or tree based learning systems. This makes RIPPER interesting since most conclusions about the effectiveness of various representations have been drawn in a context that may not apply to a rule-based learner.

C. Application to IDS To reduce the false positive alarms of an IDS, we need an approach which is able to deal with uncertainty in network traffic to predict unforeseen and noisy data accurately. Furthermore, the information provided for alerts through audit data and logs do not hold sufficient facts on the characteristics of the connections made on the network. Fuzzy rule based systems have the ability to explain the fuzzy patterns of alerts attributes. However, these alerts attributes used to train the fuzzy rules for IDS is usually high in dimensionality. For example, alerts generated from DARPA 1999 dataset contains many attributes to be analyzed. Each attribute has a various number of possible values ranging from small number of possible value (e.g. the number of protocols) to the huge number of possible values (e.g. the IP address). Therefore, it is not an easy task to explicitly determine the membership functions for the fuzzy rules. For this type of background knowledge, a Neural Network (NN) approach is acceptable as a powerful learning method to learn from scratch. For these reasons, the NN can be a useful learning approach to refine the fuzzy sets and membership function to be appropriate with the dataset. Due to the reasons mentioned above, the neuro-fuzzy hybrid approach was investigated to reduce false positive alerts. This paper proposes a solution for the problem of false negatives, false positives, and network noise through the use of NeuroFuzzy Classifier. From SNORT documentation, it has been clearly identified that there is simply not enough information present inside the SNORT engine to make any knowledgeable assessment of a true attack. Obviously packet analysis is necessary to detect attacks, but an additional level of information and decision processing is required. The most logical improvement would be the addition of an AI technique that will automatically learn from the history of past attacks. Although SNORT by itself is a successful, highly rated intrusion detection device, it offered no systematic attack analysis by itself. To rectify this situation, a completely separate, parallel knowledge base would work in tandem with SNORT, which would still function as the primary attack detector. Using SNORT detection engine, a NeuroFuzzy Classifier would work in parallel, which would sift through the SNORT alerts intelligently such that it can act as a security assistant for the system administrator. III. E XPERIMENTAL S ETUP Experimental Setup consisted of the following elements: 1) SNORT IDS 2) DARPA Data Set 1999 3) TcpReplay 4) NeuroFuzzy Classifier Firstly SNORT [17], a lightweight intrusion detection system tool that can be deployed on TCP/IP network will be used to detect attacks and generate alerts. The default

configuration of SNORT and rule sets is used intentionally to show how much to reduce the number of false positive alarms. In this paper, SNORT version 2.7.0 was used along with its corresponding rules. Secondly Tcpreplay tool [19] is configured to resend the TCP dump raw data of the DARPA 1999 dataset to generate the alerts and log them into a file. The DARPA 1999 dataset from MIT-Lincoln Lab is a collection of four types of network traffic data, which are inside tcpdump and outside tcpdump, audit data (bsm), and file systems data. The dataset consists of 5 weeks of traffic. The first three weeks of traffic is attack-free except for the second week that includes labeled known attacks. The fourth and fifth weeks are the testing dataset that contains new attacks [18]. Thirdly, Tcpdump binary files of the outside traffic of the DARPA 1999 dataset were used for the experiment. The first three weeks of the dataset are used for training purpose, while the last two weeks of data are used for evaluation. The following diagram depicts the architecture of the proposed experimental setup.

B. Understanding the Alerts Understanding of alerts is a very slow and tedious task. If pre-processing is not done properly, it is nearly an impossible task to train a network and get acceptable classification percentages. The alert of the training phase has to be properly labeled as true or false alert for us to be able to train the NeuroFuzzy Network accordingly. If there is a mistake in the training input, all the analysis that will come later will be biased. Once our training data is complete, we can freeze this phase and start with training our NeuroFuzzy Network. 1) Alert Correlation: Correlation in Intrusion Detection concerns finding a relationship between alerts generated by a single (or multiple) data sources and coupling this information with additional knowledge. •

•

Explicit Correlation Where it is possible to express some connection between known events. This form of knowledge has to be manually entered in the system. Implicit Corrrelation Is used when data analysis brings out some mappings and relations between events. Implicit correlation can be based on learning techniques and statistics.

2) Aggregation: Aggregation, following correlation, is the process of grouping events together according to certain criteria to compute aggregated security level. The goal of aggregation is to discover high-level incidents. Both correlation and aggregation has been used to some extent to Pre-Process SNORT Alerts, the following section gives more detail. Fig. 2.

Experimental Setup Architecture.

A. Role of NEFCLASS NEFCLASS is not an automatic classifier creator where data is fed in and a solution pops out, but it must be seen as a tool that supports users in finding readable fuzzy classifiers. Forward Path: Classification Backward Path: Learning Main goal of NEFCLASS 1) Readable Classifier 2) Acceptable Accuracy Fuzzy Rule-based Systems have the ability to explain the fuzzy patterns of alert attributes. But its main problem is: • •

Alert attributes are high in dimensionality Each attribute has various number of possible values

Solution: A Neural Network to learn this type of background knowledge of alerts. A Neural Network can be used as a learning approach to refine the fuzzy sets and membership function to be appropriate with the dataset.

IV. I MPLEMENTATION The Implementation part consists of taking preprocessed alerts as input to a classifier, in this case NEFCLASS and JRip. These classifiers were trained and tested by modifying the parameters that will ensure a higher classification rate. A. Parameters of NEFCLASS The following table shows the possible parameter modifications that can be made to NEFCLASS to improve training and classification rate. TABLE I NEFCLASS PARAMETER S ETTINGS

Training data file Number of fuzzy sets Type of fuzzy sets Aggregation function Size of the rule base Rule learning procedure Fuzzy set constraints Rule weights Learning rate Validation Stop control

The Parameter Settings Darpa.dat or KDD.dat Any valid number Triangular/Trapezoidal/Bell-Shaped/List Maximum/Weighted Sum Automatic/Manual Best per Class/Best Relative/Overlap/Symmetrical/Intersect Not used/[0-1]/Arbitrary [0-1] No validation/Cross Validation[n]/Single Test[%] Max Epoch/Min Epoch/Optimum/Admissible Error

TABLE III P REPROCESSED KDD D ATASET

B. Testing

We installed SNORT Version 2.7.0 with its default rulesets KDD Dataset: 3 Fuzzy Sets; 100 Epochs; 0.01; Triangular; No Validation and replayed DARPA raw packets to it to generate SNORT Correct Misclassified alerts. These alerts were pre-processed in an appropriate Training 94.09 5.91 87.10 12.90 format to be fed in our Classifier. The parameters were Testing tweaked such that we get a better classification rate which in turn means better detection ratio for attacks. is better than in Table II. The reason behind this is that KDD Data Set contains 41 features out of which we have used 13 features with highest information gain value. C. Fuzzy Rules Generated After training the System, an example of fuzzy rules that will be generated are as follows:Original JRip rules: =========== (0 = 0) and (3 = 3) => Class=0 (3705.0/0.0) (0 = 1) => Class=0 (2176.0/20.0) (0 = 0) and (1 = 2) and (1 = 1) => Class=0 (366.0/0.0) (0 = 2) and (1 = 0) and (6 = 0) => Class=0 (70.0/8.0) (1 = 2) => Class=0 (240.0/64.0) (6 = 1) and (0 = 2) => Class=0 (36.0/6.0) (3 = 3) => Class=0 (9.0/0.0) (1 = 1) => Class=0 (8.0/1.0) => Class=1 (15182.0/8.0) Number of Rules : 9 Class=0 Normal and Class=1 Attacks

The numbers in the bracket stand for coverage / errors in the training data, which follows the standard convention of tree/rule induction. eg. (0 = 0) → Class=0 (3705.0/0.0) means that the rule “(0 = 0) → Class=0” covers instances with total weights of 3705.0, out of which there are instances with weights of 0.0 misclassified. Normally weight 1 means one instance. These rules are then used to test the system and it can be observed that if we can increase classification rate, the number of false alerts are drastically reduced. From the results obtained through various testing, it is noted that SNORT alerts is almost always 95% of false positives. V. R ESULTS The parameters in Table I were used to get different set of results of training and testing on preprocessed SNORT alerts as follows: TABLE II P REPROCESSED SNORT D ATASET SNORT Dataset: 7 Fuzzy Sets; 1000 Epochs; 0.01; 10 Cross Validation Correct Misclassified Week4 89.03 10.97 Week5 83.93 16.07 Average 84.63 15.37

Table II has been generated using 7 variable Fuzzy sets, 1000 epochs, a learning rate of 0.01 and 10 Cross Validations. The overall classification ratio is 84.63% and false positive rate was 0.10% only. With KDD Dataset as input which is originally a part of the DARPA Data Set, we can see that the classification rate

A. Evaluation of Results The above result shows that NEFCLASS and JRip can be used to reduce False alerts in IDS. Using a NEFCLASS detection rate was 84.63%, JRip was 88% and False alerts were reduced as follows: False Alerts Reduction Rate - Jrip Classifier False Positive False Negative Detection Rate

8.48 % 3.52 % 88 %

False Alerts Reduction Rate - NEFCLASS Classifier False Positive False Negative Detection Rate Misclassification Rate

0.10 % 6.67 % 84.63 % 8.60 %

VI. C ONCLUSION In this paper, we demonstrated that a Neuro-Fuzzy Approach can be used to solve the problem of false alerts in IDS. The proposed scheme tries to keep the number of false alerts generated by an IDS to an acceptable level. We have chosen NEFCLASS and JRip as classifiers and DARPA 1999 Data Set and KDD 1999 as our dataset for training and testing purposes. VII. D IRECTIONS FOR F UTURE W ORK •

•

More work can be done to improve on the detection rate, such as using more features. This can be achieved by parsing low level packets captured by SNORT instead of using only the alerts. AI logic can directly be embedded in SNORT as a plugin. R EFERENCES

[1] Alshammari Riyad, Sonamthiang Sumalee, Teimouri Mohsen, Riordan Denis, “Using Neuro-Fuzzy Approach to Reduce False Positive Alerts”, Communication Networks and Services Research, 2007. CNSR ’07. Fifth Annual Conference IEEE Press, pg 345 - 349 [2] Bakar N., Bealton B., and Samsudin A., “False Positives Reduction via Intrusion Alert Quality Framework”, Joint IEEE Malaysia International Conference on Communications and IEEE International Conference on Networks, pp. 547-552, November 2005.

[3] Nauck D., Nauck U., and Kruse R., “NEFCLASS for JAVA New Learning Algorithms”, Proceedings of Fuzzy Information Processing Society(NAFIPS) 18th International Conference of the North American, pp. 472-476. July 1999. [4] Nauck D., and Kruse R., “NEFCLASS: A Neuro-Fuzzy Approach for the Classification of Data”, ACM Symposium on Applied Computing, Nashville, pp. 461-465, Feb 1995. [5] William. W. Cohen, “Fast Effective Rule Induction”, Proceedings of the Twelth International Conference(ML95), 1995. [6] Kayacik H. G., Zincir-Heywood A. N., Heywood M. I., “Selecting Features for Intrusion Detection: A Feature Relevance Analysis on KDD 99 Intrusion Detection Datasets”, Proceedings of the Third Annual Conference on Privacy, Security and Trust, October 2005, St. Andrews, Canada. [7] Dorothy Denning, “An Intrusion-Detection Model”, IEEE Transactions on Software Engineering, no. 2, page 222, February 1987 [8] Biswanath L. Mukherjee, Todd Heberlein, and Karl N. Levitt, “Network Intrusion Detection”, IEEE Network, vol. 8 no. 3, pp. 26-41, May/June 1994. [9] Frank, J., “Artificial Intelligence and Intrusion Detection: Current and Future Directions”, Proceedings of the 17th National Computer Security Conference, October 1994. [10] Srinivas Mukkamla, Andrew H. Sung, “Identifying Significant Features for Network Forensic Analysis Using Artificial Intelligent Techniques”, International Journal of Digital Evidence, Winter 2003 Vol I Issue 4 [11] Kasabov, N. “Foundations of Neural Networks, Fuzzy Systems and Knowledge Engineering.” MIT Press, Cambridge, Massachusetts. 1996. [12] Prechelt, L., “Proben1-A Set of neural Netwok Benchmark Problems and Benchmarks Rules.” Universitt Karlssruche, Germany,1994. [13] Nauck, D., “Design and Implementation of Neuro-Fuzzy Data Analysis Tool in Java.” Technische Universitt Brauschweig, Brauschweig, 1999. [14] Mahoney, M. and Chan, P., “An analysis of the 1999 DARPA Lincoln Laboratory evaluation data for network anomaly detection,” In Recent Advances in Intrusion Detection (RAID2003)- Lecture Notes in Computer Science, Vol. 2820, pp. 220-237. Springer-Verlag, 2003. [15] http://www.cnn.com/2000/TECH/computing/09/06/fear.trinity.idg/ [16] Innella, Paul; Mcmillan Oba. “An Introduction to Intrusion Detection Systems” 2001 http://www.securityfocus.com/infocus/1520 [17] SNORT, Intrusion Detection System, www.snort.org [18] DARPA Data Set, 1999, www.ll.mit.edu/IDS/eval/1999/ [19] TcpReplay, Packet Replay Tool, www.tcpReplay.com [20] JRip (Weka’s implementation of the RIPPER rule learner, www.auknomi.com/categorical learners.html

Pravesh Gaonjur Pravesh Gaonjur obtained his BSc. Software Engineering degree from the University of Technology, Mauritius, 2005. He then worked in the software industry in Mauritius for two years before coming to India. He is currently a Research Scholar at Vishwakarma Institute of Technology, Pune. His main research interest is in the area of Network Security. He has a conference paper on how to mitigate Insider Threats in BPO published in SAM’06 at Las Vegas, Nevada USA. He currently researches on IDS/IPS and techniques of how to make them more efficient using Artificial Intelligence.

N.Z Tarapore N Z Tarapore obtained his M.S. degree from the University of Florida, Gainesville, USA in 1995. He then worked in the software industry in the US for six years before returning to India. He then joined the world of academics where he has been teaching since 2002. His main research interest is in the area of distributed systems.

S.G Pukale S G Pukale obtained a Master of Engineering degree in Computer Science and has been in the field of academics for the last 16 years. His area of specialization is Networking Recognition.