Essential Discriminators for P2P Teletraffic Characterization Tao Ban, Shanqing Guo, Masashi Eto, Daisuke Inoue, and Koji Nakao

?

National Institute of Information and Communications Technology 4-2-1 Nukui-Kitamachi, Tokyo, 184-8795, Japan [email protected] http://www.nict.go.jp

Abstract. Characterization of P2P traffic is an essential step to develop workload models towards capacity planning and cyber threat countermeasure over P2P networks. In our previous work, we have presented an efficient scheme for monitoring and characterizing content-distribution P2P protocols which supports performance tuning between monitoring cost and system response time. In this paper, to further enhance the lightweightness and prediction accuracy of the system, we make use of nonlinear feature selection methods to reduce the number of discriminators used for the classification. Experimental results show that the nonlinear SVM Recursive Feature Elimination method could successfully identify the most significant discriminators. Using only 3 significant features, we could obtain an classification system equally accurate as the one built on 18 defined features, whereas using a linear feature selection method, the number of necessary features is 13.

Key words: Application behavior analysis, feature selection, network traffic classification, network monitoring, P2P protocol

1

Introduction

Today, there is a pressing need for reliable classification of teletraffics according to application layer protocols for better provision and improving the network Quality of Service (QoS), enforcing network operations such as resource reallocation and route planning, and boosting the accuracy and efficiency of network intrusion detection systems. Among the large number of protocols that are widely used in the Internet, peer-to-peer file sharing (P2P) remains as the most important and challenging protocol in question. Recent Internet studies [1] show that (1) the proportion of P2P traffics is still prevalent over the Internet; (2) many P2P applications are bandwidth-intensive, leading to excessive network congestion and possible dissatisfied customers and customer churn; (3) P2P content distribution applications often cause legal concerns because of copyright law infringement; and (4) most P2P clients are vulnerable to cyber attacks and when ?

This research is partially supported by the Key Science-Technology Project of Shandong Province, P. R. China (Grant No. 2010GGX10117).

2

Essential Discriminators for P2P Teletraffic Characterization

compromised will lead to serious information leakage and other catastrophic problems. Towards tractable mitigation of the bandwidth intensity of P2P protocols and proactive detection of malware proliferation over P2P networks, in our previous study, we have presented a cost-effective scheme to characterize Content Distribution P2P (CDP2P) applications [2–5]. To enable the lightweightness and adaptability of the proposed scheme, we made use of meta features extracted from the header fields of the packets for differentiating P2P traffics from normal traffics. And to grasp the overall one-to-many connection pattern of P2P communications, analysis was done at the host level where entropy based features were introduced to measure the topology of the communication among a large number of channels. Based on the 18 features introduced in [3], a Support Vector Machine (SVM) classifier could gave a prediction acurracy of above 98% even on previously unknown protocols. To further reduce the costs on the collection, storage, and analysis of the data, Feature Selection (FS) was adopted to identify the essential features that help differentiate P2P hosts from normal hosts [5]. Experimental results showed that a linear Fisher Discriminant based feature selection method could yield a subset of 13 features so that the prediction accuracy is roughly preserved and the computational cost considerably reduced. On the other hand, it is well-known that if the discriminators are nonlinearly correlated with the class labels, a linear feature selection algorithm may overestimate the number of necessary features. The data collected in the previous study has shown obvious nonlinearity in the distribution indicating that nonlinear feature selection may lead to better inspection to the problem. In this paper, we apply the nonlinear SVM Recursive Feature Elimination (SVM-RFE) method to explore the nonlinearity in the data and identify the essential features for characterizing P2P traffics. Experimental results show that SVM-RFE could significantly reduce redundant features such that equally accurate classification could be achieved with only 3 out of 18 features. This result suggests that nonlinear FS could help to significantly alleviate the collection cost of the features and speed up the learning process. Especially, scattering the samples in the 3dimensional feature space defined by the selected features could give us better insight into the problem as well as improve the model interpretability. The rest of this paper is organized as follows. Section 2 presents previous work on P2P teletraffic monitoring and characterization. Section 3 introduces the methodology adopted for nonlinear feature selection. Section 4 reports experimental results based on the proposed scheme. Section 5 draws the conclusion.

2

Related Work on P2P Network Characterization

Network Traffic Monitoring and Analysis (NTMA) has been a handy tool to counterattack cyberthreats and prevent abusive or illegal resource usage. Following [2–4], this section presents a brief overview of the previous work on NTMA.

Essential Discriminators for P2P Teletraffic Characterization

3

Fig. 1. Overall framework of the traffic monitoring and analysis system.

2.1

Monitoring and Teletraffic Collection

The system introduced in [2–4] tries to strike a balance between traditional monitoring schemes such as transport layer analysis [6], flow-level analysis [7], and application level tracing. As shown in Fig. 1, the system is made up of three layers. The network layer offers accessibility to the outside network and high-performance storage service for the upper layers. The server layer offers a virtualization environment to the guest OSes, captures the individual traces for each of them, and sends the traces to storage servers on the network layer. The virtual machine layer is characterized by guest OSes where specific P2P clients are installed with Internet connection enabled. Thanks to the Virtual Machine (VM) technology, this hybrid approach is featured by the following advantages. First, multiple OSes could co-exists on the same computer, enabling us to build a comparably large-scale network environment without the burden to handle too many physical machines. Second, it makes more efficient usage of the computer resources. With the available resources at hand, we can obtain a much larger-scale P2P network. Then, in case the guest OSes are compromised by malware, VM can sandbox the guest OSes, rendering the hypervisor safe from harm. The last but not least important, thanks to the fast system recovery and reboot capability of the VM technology, it is much easier to redo the experiment or adapt the system to analyze other P2P protocols.

4

Essential Discriminators for P2P Teletraffic Characterization

Such a single-protocol exclusive network has the following merits. First, application specific characteristics can be abstracted from the collected traces. Second, the traces are automatically labelled with good accuracy and little labor, ready for further analysis. Finally, because the traces are collected at networklevel, the system built for one P2P network can be easily reusable for any other (P2P) protocols. 2.2

Data Analysis

Network traces collected by the above monitoring system could offer essential information on the behavior of the hosts. In [3, 4], a classification task is defined to differentiate the hosts which are doing P2P file sharing (positive class) from other hosts (negative class). The analysis makes use of statistical-meta-features/discriminators extracted from the monitored interfaces and detects anomalies associated with P2P hosts. The first group of discriminators include traffic statistics on metrics such as the traffic volume, packet size, and number of preserved connections, which are good indicators of P2P applications. Traditionally, these discriminators are often derived on network flows – traffic channels between communicating peers defined by the 5-tuple, i.e., {source address, destination address, source port, destination port, protocol}. In [3, 4], similar features are adopted, however, at host-level instead of flow-level: all the communications bounded to a target host (associated with a specific IP address) are treated as a single stream and features are defined upon these host-level streams. More specifically, {number of packets, payload volume, time range, payload speed, number of protocols, number of source IPs, number of destination IPs, number of source ports, number of destination ports, number of TCP flags} are defined on all communications bounded to a host within a fixed time window. In addition to the classical features which well represent the bandwidthintensive nature of P2P streams, entropy based features that presents the topological characteristics of P2P file-sharing protocols are introduced in [3, 4]. For example, the entropy, HSIP , over the source-IP-address space is calculated as [8] HSIP = −

n X

pi log2 pi ,

(1)

i=1

where n is the number of unique source IP addresses and pi is the probability that the ith IP address shows up as source IP within the time window.

3

Feature Selection

Fig. 2. shows the scatter plots in the multi-dimensional space defined by all the adopted discriminators in [3, 4]. It is easy to spot the separability of the two classes within the subspace defined by randomly paired discriminators: along some of the dimensions, data from the positive class (shown as green markers)

Essential Discriminators for P2P Teletraffic Characterization

5

Fig. 2. Scatter plots in the multi-dimensional feature space. RR: normalized SVM-RFE ranking rate. FD: Fisher discriminant value. (#) stands for number and (E) stands for entropy.

and those from the negative class (shown as blue markers) forms perceptible classification boundaries. However, there are also some redundant features could be observed in the graphs. For example, the time range (the y-axis Fig. 2b) could offer little discriminative information in the classification – there is severe overlapping between different classes when all the data are projected onto this axis. Because redundant discriminators could not only introduce much complexity in the learning but also impose unnecessary cost on data collection and processing, it will be helpful to evaluated the significance of all these discriminators with regards to the classification and eliminate the irrelevant/redundant ones. In [4], a Fisher Linear Discriminant [9] based filtering approach is presented to rank the features and remove the redundant ones for reducing the collection and processing cost. Suppose two classes of observations have means µj,+ , µj,− and variances Vj,+ Vj,− along the jth discriminator. The Fisher Discriminant (FD) along this discriminator is defined as the ratio of the variance between the classes to the variance within the classes: Sj =

2 σbetween,j (µj,+ − µj,− )2 = . 2 σwithin,j Vj,+ + Vj,−

(2)

It can be shown that the maximum separation occurs when the two means are widely separated from each other while the variances along the axis are small. Thus the FD in (2) gives us a very illustrative measure of the class separability along the axis; a larger value generally indicates easier separation. The FD values along each axis could help to rank the features, so that the features with small FD values could be removed from the training of the model, using the

6

Essential Discriminators for P2P Teletraffic Characterization

backward selection method [10]. In Fig. 2, The FD values associated with each discriminator is listed along each axis.

4

Non-linear Feature Selection by SVM-RFE

It is pertinent to note the non-linear correlation between some of the features and the class label. For example, the entropy of Source IPs (y-axis in Fig. 2e) is obviously nonlinearly correlated with the class label: while negative samples are with medium values, positive samples are located at both lowermost and uppermost part along this axis. This invites us to apply non-linear feature selection method to explore the nonlinearity in the data and remove nonlinearly correlated redundant features. In the following, we adopt the non-linear SVM-RFE method [11] for this purpose. This section introduces the basic idea of non-linear SVM-RFE following [12]. We first briefly review the formulation of Support Vector Machine (SVM), then we explain SVM-based feature ranking criterion and show its application in the SVM-RFE algorithm. 4.1

Support Vector Machine

Support Vector Machines [13] realize the following idea. Suppose we are given two classes of samples D = {(xi , yi )|xi ∈ Rd , yi ∈ {−1, +1}, i = 1, · · · , `, } as a training set. We first map the input vector xi into a high (possibly infinite) dimensional feature space, F , through a nonlinear mapping function Φ; then construct the optimal hyperplane that realizes the maximal margin in this space. With the so called kernel trick, the mapping Φ is implicitly implemented by some kernel function K(·, ·), which defines an inner product in the feature space. The decision function yielded by an SVM classifier turns to be a linear combination of Φ(x) as f (x) = hw, Φ(x)i + b,

(3)

where w is the normal vector of the decision hyperplane in F and b the bias. A novel sample x with f (x) > 0 is assigned to the positive class, otherwise it is assigned to the negative class. For an SVM classifier with misclassified samples being linearly penalized with a positive soft margin parameter C, the optimization problem can be written as:  ` P   ξi ,  min 21 kwk2 + C i=1 (4) s.t. yi f (xi ) ≥ 1 − ξi ,    ξi ≥ 0, i = 1, · · · , `, where nonnegative slack variables ξi , i = 1, · · · , `, are introduced to guarantee that feasible solutions always exist. The solution of the problem in Equation (4) can be obtained with the Lagrangian theory and w can be computed as: w=

` X i=1

αi∗ yi Φ(xi ),

(5)

Essential Discriminators for P2P Teletraffic Characterization

7

where αi∗ is the solution of the following quadratic optimization problem, usually called a dual problem:  ` ` P P  1  αi αj yi yj K (xi , xj ), max W (α) = α −  i 2   i,j=1 i=1 ` (6) P  s.t. yi αi = 0,    i=1  0 ≤ αi ≤ C, i = 1, · · · , `. 4.2

Ranking Criteria for Feature Selection

SVMs provides us with many statistics to estimate their generalization performance from bounds on the leave-one-out error L. The leave-one-out error is known to be an unbiased estimator of the generalization performance of a classifier trained on ` − 1 examples. One of the most common L error bounds for SVMs is the radius/margin bound (for decision function with non-zero bias b) [13]: L ≤ 4r2 kwk2 ,

(7)

where r is the radius of the smallest hypersphere that contains all the mapped data Φ(x). The geometrical margin δ of a separating hyperplane, defined as the distance between the hyperplane and an unbounded support vector, can be obtained by δ = 1/kwk.

(8)

Thus, for an SVM, maximizing the margin corresponds to minimizing kwk. Given the optimal solution of Equation (6) as α∗ and w∗ , it is easy to check that, kw∗ k2 =

` P i,j=1

yi yj αi∗ αj∗ K(xi , xj ).

(9)

For a linear SVM, Equation (9) can be simplified to w∗ =

` P i=1

yi αi∗ xi .

(10)

For the linear case, we can see that if some elements of w∗ are zero, the deletion of the associated input features will not lead to variation in the decision function. Furthermore, a feature associated with an element near to zero in w∗ may be considered insignificant, and can probably be deleted without degeneration in generalization ability. Thus, the ranking criterion of the kth feature for a linear problem is defined as Rk =

` q X kw∗ k2 − kw∗(k) k2 = | yi αi∗ xik |, i=1

(11)

8

Essential Discriminators for P2P Teletraffic Characterization

where xik is the kth element of xi , and w∗(k) is obtained from w by setting all components xik to 0 for i = 1, . . . , `. We can extend this discussion to the nonlinear case where deletion of an input feature corresponds to deletion of multiple features in the feature space. In this case, features which contribute least to kw∗ k in Equation (9) can be possible candidates for deletion. The contribution of the kth feature to kw∗ k can be evaluated as Rk =

q

kw∗ k2 − kw∗(k) k2 =

` X

(k)

(k)

yi yj αi∗ αj∗ (K(xi , xj ) − K(xi , xj ))1/2 ,(12)

i,j=1

where x(k) is the vector with the kth feature of x set to 0. Note that for the sake of simplicity and speedup of computation, α∗(k) , the solution of the optimization problem with the kth feature deleted, is supposed to be equal to α∗ .

4.3

SVM-RFE Algorithm

The SVM-RFE algorithm [11] was proposed to select relevant genes for cancer classification problems. It follows the backward selection method. One starts with a pool of all the features and train a classifier based on the features in the pool. At each step, one or a subset of features which is least important is removed from the pool. The above process is repeated until a predefined number of features are left or all the features have been ranked. The removed feature at each step is the one whose removal minimizes the variation of kw∗ k2 , i.e. the feature with smallest Rk is eliminated. See Table 1 for the detailed algorithm. Here, a backward sequential selection is used because of its lower computational complexity compared to randomized or exponential algorithms and its optimality in the subset selection problem [14].

Step 1 Initialization: Pf = {1,...,d}. Step 2 Loop for feature selection 1o Train an SVM classifier with the features in Pf ; 2o Compute Rk , (k = 1, ..., |Pf |); 3o Ef = arg mink Rk ; (Ef can be multiple features.) 4o Pf = Pf − Ef ; Step 3 If |Pf | ≥ r, then go to step 2, otherwise stop. Table 1. The backward selection algorithm for SVM-RFE. Pf : the set of preserved features for SVM training, Ef : a set of features to be deleted, Rk : the ranking criterion for the kth feature with the current SVM, r: a predefined number of features, and |Pf |: the cardinal number of feature set Pf .

Essential Discriminators for P2P Teletraffic Characterization

5

9

Experiments

To testify the feasibility and evaluate the performance of the nonlinear SVMRFE, we apply it to the same traffic classification task presented in [5]. For easy comparison, we use the same experimental setting as in previous study. Experimental results are compared with the FD-based method. In the experiment, we perform classification between the background traffic and two most popular P2P protocols, i.e., BitTorrent, which is the world’s most popular P2P file sharing protocol, and PPLive, which is a typical protocol of the new generation P2P applications known as P2PTV. Training and test are both performed on trace data containing the same protocols. The P2P traces are collected within the following network environment. For each of the eight-core servers, we run 6 virtual machines with Windows 2000 installed. QEMU is chosen as the hypervisor software for its good configurability. Corresponding software clients are installed, configured, and run upon the guest OSes. A P2P trace is collected and labeled based on what client is running on the guest OS. The background traffic is captured in a similar network environment where common network operations such as web browsing, FTP/HTTP downloading, and online gaming are permitted. In SVM training, we use the Gaussian kernel which is a universally adaptable kernel function with reliable performance in a wide range of applications. Parameter settings, i.e., width parameter of the kernel, γ, and error penalty parameter, C, are determined by 10-fold cross validation. All the reported results are averaged on 100 runs on the randomly shuffled versions of the same trace set. Discussions in [3, 4] suggest that there are two important parameters, i.e., the time-window size, w, which is closely related to the response performance of the system, and the sampling rate r, which determines the scalability of the data collection and analysis. Numerical study show that (w = 16, r = 18 ) could achieve very high accuracy with a reasonable burden on the monitored network. The following results are based on the above parameter setting. For a given realization of the traces, we perform the nonlinear SVM-RFE by sequentially removing the least significant discriminator from the feature pool. The obtained order of discriminators is shown in Fig. 3a, with the features ranked in descending order of their significances. Then we train the classifiers on the data with the feature at the end of the list removed at each step and record the change in prediction accuracy. According to Fig. 3a, the most important feature is the entropy over destination ports in the 16s time window: we can have an accuracy of 89% using this single feature. This indicates that the scattering of the traffic into different ports is one of the key features that characterizing P2P hosts. The most essential subset of discriminators is apparently entropy over destination ports, entropy over source IPs, and number of TCP flags. Using only these three discriminators we could obtain a classifier as good as the one we can have using all the discriminators. Adding other features to this discriminator set could give little improvement on prediction accuracy. We have to note that some of the entropy based features such as entropy over destination ports, entropy over source IPs, and entropy over source ports provide

Essential Discriminators for P2P Teletraffic Characterization 100

95

95

70

TCP Flags(#)

Dest. Ports(E)

Protocols(#)

Protocols(E)

Payload(E)

TCP Flags(E)

Dest. IPs(E)

Time Range

Dest. Ports(#)

Time(E)

Dest. IPs(#)

Source IPs(E)

Packets(#)

Source Ports(E)

75

Source Ports(#)

80

Source IPs(#)

85 Payload Volume

Dest. Ports(#)

Payload Speed

Payload Volume

Dest. IPs(#)

Source Ports(#)

Packets(#)

Dest. IPs(E)

Time Range

Protocols(E)

Protocols(#)

Time(E)

TCP Flags(E)

Payload(E)

Source Ports(E)

Source IPs(#)

75

TCP Flags(#)

80

Source IPs(E)

85

90

Payload Speed

90

Prediction Accuracy(%)

100

Dest. Ports(E)

Prediction Accuracy(%)

10

70 Feature Ranking

(a)

Feature Ranking

(b)

Fig. 3. Prediction accuracy against ranked features. (a) Result of nonlinear SVM-RFE. (b) Result of FD based filtering.

important discriminant information as they are all ranked high in the list, despite that the first three discriminators seems to perfectly match this classification task. On the other hand, discriminators at the end of this list e.g., number of destination ports, payload volume, payload speed, and number of destination IPs which are presumed to be important are in fact irrelevant/redundant to the task and could be safely discarded without significant loss in prediction accuracy. The result of FD-based filtering is shown in Fig. 3b as a reference. This linear method ranks traditional features such as payload speed, payload volume, and number of source IPs higher than entropy based features. This could be explained by the facts that entropy based features tend to nonlinearly correlated with the class label. That is, nonlinear class boundaries are formed in the subspace defined by these features, which could be shown the scatter plots in Fig. 2.

6

Conclusion

In this paper, we have presented a study on applying machine learning techniques for characterizing P2P content distribution hosts in the network for network engineering purpose. For better lightweightness and adaptability of the system, we apply nonlinear SVM-RFE to identify the most significant features that help to differentiate P2P hosts from other hosts. The numerical experiments show that P2P hosts are best characterized by a group of three features: number of source IPs, entropy of source IPs, and number of TCP flags. This results in an effective monitoring and analysis system for P2P host classification with very low collection and storage cost. On the other hand, some traditional features which are presumed to be discriminative are actually irrelevant to the task and could be discarded without negative influence on the prediction accuracy.

Essential Discriminators for P2P Teletraffic Characterization

11

References 1. http://www.ipoque.com/en/resources/internet-studies 2. T. Ban, R. Ando, and Y. Kadobayashi, Monitoring and analysis of network traffic in P2P environment, NICT Journal, Vol.54, Nos. 2/3, pp. 31–39, 2008. 3. T. Ban, S. Guo, Z. Zhang, R. Ando, and Y. Kadobayashi, Practical network traffic analysis in P2P environment, 2nd International Workshop on TRaffic Analysis and Classification(TRAC2011), July 5–8, 2011. 4. T. Ban, S. Guo, M. Eto, D. Inoue, and K. Nakao, Entropy based discriminators for P2P teletraffic characterization, to appear in Proceedings of 2011 International Conference on Neural Information Processing (ICONIP 2011), Nov. 14–17, 2011, Shanghai, China. 5. T. Ban, S. Guo, M. Eto, D. Inoue, and K. Nakao, P2P network traffic analysis using data mining engines, IEICE Technical Report, Vol. 111, No. 157, pp. 115–118, 2011. 6. T. Karagiannis, A. Broido, M. Faloutsos, and K. Klaffy, Transport layer identification of P2P traffic, Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement, pp. 121–134, 2004. 7. S. Sen and J. Wang, Analyzing peer-to-peer traffic across large networks, Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurement, pp. 137–150, 2002. 8. A. Lakhina, M. Crovella, and C. Diot, Mining anomalies using traffic feature distributions. In: ACM SIGCOMM (August 2005) 9. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, New York, 2001. 10. I. Guyon and A. Elisseeff, An introduction to variable and feature selection, Journal of Machine Learning Research, Vol. 3, pp. 1157–1182, 2003. 11. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, Gene selection for cancer classification using support vector machines, Machine Learning, 46(1-3):389–422, 2002. 12. T. Ban and S. Abe, SVM ensembles for selecting the relevant feature subsets, Proceedings of IEEE International Joint Conference on Neural Networks, Vol.2, pp. 943–948, 2005. 13. V. Vapnik. Statistical Learning Theory. John Wiley and Sons, 1998. 14. C. Couvreur and Y. Bresler, On the optimality of the backward greedy algorithm for the subset selection problem, SIAM Journal on Matrix Analysis and Applications, 21(3):797–808, 2000.