Predicting Dense Regions in Dynamic Graphs K. Ashwin Kumar Abdul Quamar University of Maryland at College Park

ABSTRACT Many of the real-world interactions and processes over the time can be modelled using dynamic graphs. Temporal aspect of dynamic graphs poses lot of interesting problems and has generated lot of interest in research community off-late. We consider solving one such problem, which is prediction of dense regions in dynamic graphs. Density prediction in dynamic graphs involves the study of the growth processes of dynamic graphs and building models to assist in the temporal density prediction. In other words the model would predict the dense regions in a dynamic graph. Given the history of graphs over time, question is, how to predict the graph at a future time t and its dense regions? We present a three phase approach to solve this problem that includes temporal graph summarization, time based node and link prediction at each time step and min-cut partitioning to find dense regions. We also highlight many different applications and real scenarios where our work would find it’s application. At the end, we present our initial results that show effectiveness and efficacy of our proposed approach.

1.

INTRODUCTION

It is well known that many real-world interactions, processes and behavioural structures can be modelled and analysed using graph abstractions. For example: biological networks (protein-protein interaction networks), social-economic networks (friendship and professional networks, organizational hierarchies, citation networks) communication networks (mobilemobile communication, sensor networks), genealogy (family tree), and transportation networks (subway system, road maps). Graph theory is a very strong field and has many decades of research history that can help solve many important problems in the real worlds of business, industry, education, and others. Most of the time, we don’t have to worry about inventing a new theory to solve the problems if modelled using graphs. But, almost all the graph problems that has been looked into are on static snapshots of the data. Graphs captured at a certain time t are called as ‘static’ graphs and graphs that are observed over set of times

T = {t1 , t2 , . . . , tn } is known as ‘dynamic graphs’. Temporal aspect of the graphs has been largely ignored. Ignoring this very crucial information about the data can result into solutions that are not scalable over time, not long lasting and temporally locally optimal. Studying the dynamism of the data helps us understand the process in a better way, we will be able to come up with more accurate solutions without losing the bird’s eye view of the problem. On the other hand, the transient and temporal nature and complexity of most processes makes their experimental characterization extremely difficult. Some of the interesting problems that arise due to temporal nature of the data are keeping track of all the changes over time for example tracking patterns of interest if the graph is evolving over time, learning and predicting temporal sequences, finding similarity between temporal sequences, summarizing the temporal data for further processing, clustering over time, indexing efficiently and querying the temporal data, ranking temporal data/sequences. All of these problems poses challenge in terms of dynamism that data exhibits. In this paper, we consider one such problem that temporal aspect of the data poses, that is predicting dense regions in dynamic graphs or graphs over time. Density prediction in dynamic graphs involves the study of the growth processes of dynamic graphs and building models to assist in the temporal density prediction. In other words the model would predict the dense regions in a dynamic graph. It is interesting to note that the dense regions in the graph can be viewed as community structures in social networks. Community detection in static graphs has received a fair amount of attention in the recent past and there has been a lot of ongoing interesting work in the area. Dynamism in graphs poses a new challenge in the field of community detection and hence the motivation. In this work, we have not made any unrealistic assumptions about the model. We allow new nodes and new edges to be added to the graph at each step as it can be observed in real-life networks. We conduct experiments on the real-world dataset. This problem finds it’s application in variety of problems. For example, in recommendation systems where specific recommendations are done to the users based on groups that he is expected to be a part of in future. Opportunistic advertising, where based on predicted dense regions or communities specific advertisements are targeted to the groups. Network resource allocation, where predicted dense regions can help

in better resource allocation and planning, that is predicted dense regions may indicate increasing demands. In network architecture planning, where graph can be modelled with users as nodes and communication between them over the network as edges, then given the network architecture or topology by predicting the dense regions in future, we can estimate the probability of network clogging or chocking and based on this insight one can plan and deploy better network architecture. Outline: We begin with discussing closely related work (Section 2). We then describe our model and formalize the problem (Section 3). We present our prediction model and related algorithms to solve the problem (Section 4) and present our initial experiments based on real world dataset.

2.

RELATED WORK

Since social networks model real life interactions and interlinked processes, they are inherently highly dynamic in nature. Many a time interactions do not hold long, dense network structure becomes sparse and social communities break or disappear altogether over time. Rapidly growing electronic networks, such as emails, web, blogs, and friendship sites, as well as mobile sensor networks are the examples of such dynamic nature of systems. This temporal dynamism of social networks has been studied recently by various researchers. Mislove et al. [13] study the dynamics of large online social networks. They basically collect and examine detailed growth data from the Flickr online social network, focusing on the ways in which new links are formed. BergerWolf [3] proposed a framework for identifying communities in dynamic social networks, making explicit use of temporal changes. Tantipathananandh et al. [23] propose frameworks and algorithms for identifying communities in social networks that change over time. They basically propose an optimization-based approach for modeling dynamic community structure. On other side, there have been various works that present growth models to capture the dynamism in social networks and their growth rate [1][11]. Cortes et al. [5] introduce a data structure and an updating scheme that captures, in an approximate sense, the graph and its evolution through time. As we discussed earlier, a natural way to incorporate dynamism in graphs is to weigh vertices and/or edges of a graph by their existence probability [7]. A generic problem in the context of probabilistic graphs has been to find the probability that a graph remains connected despite random drop out of vertices and edges. We may consider summarizing the series of graphs over the time with a single probability graph and then detect communities in them. This works good only when we have graph history for time t − 1 and want to predict the graphs structure at time t. We may need to model the growth process and then combine that with the probability graph to be able to predict for any given time. Tylenda et al. [26] developed a graph-based link prediction method that incorporate the temporal information contained in evolving networks. In this they consider time-based weighting of edges. Sharan et al. [21][22] present a two-step process that first summarizes the dynamic graph with a weighted static graph and then incorporates the link weights in a relational Bayes classifier. Leskovic et al. [12] This approach will help us with the concept of summarization of temporal graphs.

On the other hand there has been lot of research in the area of community detection in static graphs. Community consists of a group of nodes that are relatively densely connected to each other but sparsely connected to other dense groups ˝ in the network [8]. The GirvanUNewman algorithm is one of the most popular methods used to detect communities in complex graphs [9]. Graph partitioning techniques [20] were also used where goal is to minimize the cross partition edges or cuts. One of the most widely used methods for community detection is modularity maximization [16]. Modularity presents a metric to asses the quality of the partition, and goal is to maximize the quality of each partition [18][19]. Simple and one of the most effective model is based on random walks [24][25]. Main idea is that when random walk is performed from any node, walk will spend most time in the dense region which is a community.

3.

PROBLEM DEFINITION

Given the history of graphs over time GT = {G1 , G2 , . . . , Gth }. We need to predict the graph Gtp at a future time tp , such that if we do min-cut partitioning of the predicted graph into K regions, then we should get dense regions which would be very similar to the dense regions that we would’ve got in original. This is a very hard problem since it imposes the partial structural similarity between the original graph and the predicted graph. Note that, in this problem predicted and original graphs need not be isomorphic to each other, we are only interested in dense region similarity. We try to solve this problem by using a parametrized prediction model, where we try to learn the parameter from the training data that will essentially guide the generation of prediction graph at any future time tp . Model: We denote graph at a time t as Gt (Vt , Et , Wt ). Wt is t the vector of edge weights where {wij = 1 | (i, j) ∈ Et }. Let th be the time for which we have the history of graphs and tp be the time for which we need to predict the graph Gtp . GT = {G1 , G2 , . . . , Gth } denotes the temporal sequence of S S S graphs. GS t (Vt , Et , Wt ) denotes the summary of temporal sequence of graphs GT at time t which we will discuss in S next section. GP denotes the summary graph at time t with t S node and link predictions. At time tp , we have Gtp = GP tp .

4.

PREDICTION MODEL

We employ three phase approach for prediction. Following are the phases: 1. Summarizing the history of graphs over time. 2. Predicting the missing links based on temporal strength of edges while summarizing. 3. Partitioning the predicted graph to get dense regions.

4.1

Summarizing Temporal Graphs

In dynamic graphs, temporal dynamism exists in number of ways. For example, objects in the graph can appear and disappear over time. Future activity in the graph can depend on the temporal characteristics of the object (both node and edge) fluctuations in the graph. For example, more number of new edges and nodes may form in the regions with recent activity than older events. This is also known as temporal

locality and temporal re-occurrence. Social networks and online news networks exhibit strong temporal locality and re-occurrence. Temporal associations between the objects can determine the further activity in the graph. Attributes of the objects may change with time that may affect the future associations with that objects. For example, when a girl changes her status from ‘single’ to ‘married’ then her future associations change drastically which in turn will change the structure of the region to which she belongs. Therefore, it is very important to model the time-varying links to identify influential relationships in the data. Represent historical behaviour between two nodes in a concise manner, summarizing the relationship into a single edge with attributes. In literature, incorporation of the influence of time varying links into statistical models has been used to improve attribute prediction [21][22]. Similarly, modelling of temporally-varying links is used to improve automatic discovery of communities [10]. Summarization helps immensely in saving processing time and storage, since we need to store and work with only one copy of graph unlike processing the whole temporal sequence of graphs.

given priority, or models strong temporal locality and reoccurrence. Social networks and news networks have θ nearing towards 1. Whereas other non-news web such as network of religious documents etc will have θ nearing 0. There are two main problems with this model that makes this model unfit for prediction purposes:

Summarization of temporal graph sequences was first introduced by Hill et al. [10] in representing the dynamic networks. We share their motivation that property of homophily (The tendency of like to associate with like) in the real networks throws light on the importance of temporal modelling of links and inference in relational domains. More recent links exhibit more homophily than older links. They provide a simple and efficient way to model temporallyvarying relational networks by transforming a temporal sequence of data graphs Gt = {G1 , G2 , . . . , Gt } into a static graph GS by summarizing the sequence of links between any pair of nodes by a single edge with a weight. SumS S S mary of graph is defined as GS t = (Vt , Et , Wt ) at time t as a weighted sum of the snapshot graphs up to time t as follows:

Predictive Model: We address the problems in the temporal graph summary model by Sharan et al., [21][22] by modifying the recurrence relation for exponential kernel. Following is our modified recurrence relation:  PS PS + θWt−1 , t > th + 2  (1 − θ)Wt−2 PS S PS Wt = (1 − θ)Wt−2 + θWt−1 , t = th + 2  S S (1 − θ)Wt−2 + θWt−1 , t = th + 1

VtS = V1 ∪ V2 ∪ · · · ∪ Vt EtS = E1 ∪ E2 ∪ · · · ∪ Et WtS = α1 W1 + α2 W2 + · · · + αt Wt =

t X

K(Gi ; t, θ)

i=1

K is used for kernel smoothing, exponential kernel [5][27] K(Gi ; t, θ) = (1 − θ)t−i θWi is used to model temporal locality and temporal reoccurence. It provides a smooth dynamic evolution of Gt . The iterative nature of the updating allows us to incorporate the information from all previous time periods without incurring the management and storage of graphs for all previous time periods. All that is needed is the graph through time period t−1 and the new set of transactions defined by Gt . Exponential kernel can be evaluated using the following recurrence relation:  S (1 − θ)Wt−1 + θWt , t > to S Wt = θWt , t = t0 θ helps the model to decide the priority that should be given to the recent data and also plays an important role in summarizing the historical data. θ = 1 gives the recent data more priority, as θ approaches 0, more historical data is blended and as θ approaches 1 then more recent data is

1. It only considers edges which it has seen in the data. Predicted graph may have many new edges (lets not consider new vertices at this point). 2. Summary WtS requires the data to have Wt . If we need to predict a graph, then we won’t have this data available. We address these problems by modifying the temporal graph summary model to a graph predictive model, by incorporating link prediction at each summary stage and by removing the dependence of the summary graph WtS on the data graph Wt .

WtS =



S (1 − θ)Wt−1 + θWt , t <= th θWt , t = t0

Where WtP S is the graph obtained by link and node predicS tion in graph GS t based on summarized weights Wt . From PS now we call Wt as the predicted graph. In our case it is th = 10, as we have 20 months of graph data available. In this prediction model, predicted edge weights solely depend on the previous graph predictions if prediction time is far from the time of historical available data. While iterating, as soon as history data is available our model starts considering it and that is obviously better than considering previously predicted graph summaries. This increases the probability of preserving the structure of original graph. Example: G1

G2

G3

G4

...

Gth

Gth +1

Gth +2

...

Gtp

G1

G2

GS 3

GS 4

...

GS th

S GP th +1

S GP th +2

...

S GP tp

This example shows that till Gth summary GS th is calculated using summarization technique with exponential kernel. Gtp is the graph to predict, then from summarized graph GS th model starts predicting new nodes and links at each time S th < t ≤ tp while summarizing GP t . Techniques for prediction of nodes and links are discussed in next sections. There is a serious implementation downside of the above recurrence relation for prediction model. That is, solutions are calculated multiple times, unlike divide and conquer algorithms where each recursive part explores the different search space. This leads to extremely slow execution of the algorithm. For the same reason we implement an iterative algorithm.

Algorithm 1 Iterative Algorithm for Prediction Require: GT = {G1 , G2 , . . . , Gth }, θ 1: for i = 1 to tp − 1 do 2: A = G1 , B = G2 ; 3: if i < th then 4: C = (1 − θ)A + θB; 5: A = C; 6: B = Gi+1 ; 7: else if i == th then 8: C = (1 − θ)A + θB; 9: B = C; 10: else if i ≥ th + 1 then 11: C = addPredictions((1−θ)A+θB, N Vnew , N Enew ); 12: A = B; 13: B = C; 14: end if 15: return C; 16: end for

In the above algorithm, method addPredictions adds the N Vnew and N Enew number of nodes and edges to the summarized graph. Firstly, we will discuss the node prediction in next section.

4.2

Node Prediction

In all the dynamic real world processes addition of new objects causes the most dynamism in the system whereas deletion of existing objects and other perturbations result in the rest. In order to predict the dense regions accurately, it is very important to estimate the number of objects that are to be added. After we have decided on the number of objects to be added, next question would be where to add these objects, such that graph structure does not get disturbed too much or graph properties should reflect the original graph at the prediction time. There have been lot of graph generative models that were studied in the literature. These models are based on various insights gained by studying real world networks. Initially, study of global properties such as the average shortest path length, the average clustering coefficient, or the degree distribution gained focus [1, 6]. These work highlight the fact that most complex networks exhibit small-world property and scale-free degree distribution. These properties although have considerable impact on the behavior of physical processes, but do not provide a sufficient characterization of natural networks. These models result in the graphs where vertices at end points of any given edge exhibit degree correlation and are not independent. Later works led to the classification of networks according to the nature of their degree correlations [15, 17]. This motivated the later works that tried to present a general framework to study the origin of correlations in random networks [2, 6].

4.3

Link Prediction

For predicting link between two nodes, most common intuition is to count the number of common neighbours and link the nodes with the number of common neighbours greater than some value. Following are three most standard measures for measuring the similarity between the two nodes: Common Neighbours: Let Γ(x) denote neighbours of the

node x, the similarity is measured as: |Γ(x) ∩ Γ(y)| Jaccard’s Coefficient: It measures the number of neighbors of both x and y compared to the number of nodes that are either x0 s or y 0 s neighbors. Its given by: |Γ(x) ∩ Γ(y)| |Γ(x) ∪ Γ(y)| Adamic/Adar: A more robust measure was proposed by Adamic and Adar. Closeness between nodes x and y is defined as: X 1 z∈Γ(x)∩Γ(y)

log|Γ(z)|

We plan to use one these measures for link prediction. For every summary graph we would get the weight vector for the edges. Each weight shows the temporal importance of the edge or connection. While predicting new links we plan to consider the temporal strength of the links between the nodes. Greater the weight on the edge shows more temporal locality or strength. For intuition lets consider the example where there are four nodes say u, v, w, x, u and v share 3 neighbours with edge weights 0.5, and w and x share 3 neighbours with edge weights 0.3. Then we would consider predicting new link between u and v as connection to common neighbours show high temporal locality than w and v.

4.4

Partitioning and Evaluation:

Once we get the predicted graph, we will do an min-cut partitioning to partition it into suitable number of dense regions. We are still figuring out the evaluation method. That is to compare the dense regions of predicted graph with the original graph.

5.

DATA DESCRIPTION

We currently have the following data sets for dynamic graphs obtained from the Max Planck Research Institute [13][14][4]: • Flickr Growth • Internet Growth • Wikipedia Growth • Youtube Growth (Directed and Undirected) These datasets are extremely large, except Internet Growth dataset. We consider Internet Growth dataset to evaluate our technique, because this dataset is relatively small and can be managed by a single machine. This dataset contains 48 months of temporal graph data of internet growth. Each month graph captures only the new connections (incremental growth is captured), e.g., if graphs at first and second 0 0 month in the dataset are G1 and G2 then actual graph at 0 0 second month can be thought as G1 ∪ G2 . Dataset contains total 25526 vertices and 104824 edges, spanning over

48 months (4 years). For simplicity, we plan to consider each month graph data as Gt . We can even group the multiple month graph data into a single history at timestamp t, to consider better graph structure at each timestep.

5.1

Baselines

We also plan to consider a baseline to compare our technique. One of the simple baselines is to add new edges randomly at every summarization step. We need to compare our predicted graph with randomly predicted graph.

6.

REFERENCES

[1] A. L. Barabasi and R. Albert. Emergence of scaling in random networks. Science (New York, N.Y.), 286(5439):509–512, October 1999. [2] J. Berg and M. L¨ assig. Correlated random networks. Phys. Rev. Lett., 89(22):228701, Nov 2002. [3] T. Y. Berger-wolf. A framework for analysis of dynamic social networks. In DIMACS Technical Report, pages 523–528. ACM Press, 2006. [4] M. Cha, A. Mislove, and K. P. Gummadi. A Measurement-driven Analysis of Information Propagation in the Flickr Social Network. In Proceedings of the 18th Annual World Wide Web Conference (WWW’09), Madrid, Spain, April 2009. [5] C. Cortes, D. Pregibon, C. Volinsky, and A. S. Labs. Computational methods for dynamic graphs. Journal of Computational and Graphical Statistics, 12:950–970, 2003. [6] S. N. Dorogovtsev and J. F. F. Mendes. Evolution of Networks: From Biological Nets to the Internet and WWW. Oxford University Press, 2003. [7] P. Erd¨ os and A. R´ enyi. On random graphs, i. Publicationes Mathematicae (Debrecen), 6:290–297, 1959. [8] S. Fortunato. Community detection in graphs. Physics Reports, 486(3-5):75 – 174, 2010. [9] M. Girvan and M. E. J. Newman. Community structure in social and biological networks. Proceedings of the National Academy of Sciences of the United States of America, 99(12):7821–7826, June 2002. [10] S. B. Hill, D. K. Agarwal, R. Bell, and C. Volinsky. Building an effective representation for dynamic networks. Journal of Computational and Graphical Statistics, 15:2006, 2006. [11] P. Holme and B. J. Kim. Growing scale-free networks with tunable clustering. Physical Review E, 65(2):026107+, Jan 2002. [12] J. Leskovec, L. Backstrom, R. Kumar, and A. Tomkins. Microscopic evolution of social networks. In KDD ’08: Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 462–470, New York, NY, USA, 2008. ACM. [13] A. Mislove, H. S. Koppula, K. P. Gummadi, P. Druschel, and B. Bhattacharjee. Growth of the flickr social network. In WOSP ’08: Proceedings of the first workshop on Online social networks, pages 25–30, New York, NY, USA, 2008. ACM. [14] A. Mislove, M. Marcon, K. P. Gummadi, P. Druschel, and B. Bhattacharjee. Measurement and analysis of online social networks. In IMC ’07: Proceedings of the 7th ACM SIGCOMM conference on Internet measurement, pages 29–42, New York, NY, USA, 2007. ACM. [15] M. E. J. Newman. Assortative mixing in networks. Phys. Rev. Lett., 89(20):208701, Oct 2002. [16] M. E. J. Newman. Fast algorithm for detecting community structure in networks. Sep 2003. [17] M. E. J. Newman. The Structure and Function of Complex Networks. SIAM Review, 45(2):167–256, 2003. [18] M. E. J. Newman. Detecting community structure in networks. The European Physical Journal B - Condensed Matter and Complex Systems, 38(2):321–330–330, March 2004.

[19] M. E. J. Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23):8577–8582, June 2006. [20] A. Pothen, H. D. Simon, and K.-P. Liou. Partitioning sparse matrices with eigenvectors of graphs. SIAM J. Matrix Anal. Appl., 11(3):430–452, 1990. [21] U. Sharan and J. Neville. Exploiting time-varying relationships in statistical relational models. In WebKDD/SNA-KDD ’07: Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis, pages 9–15, New York, NY, USA, 2007. ACM. [22] U. Sharan and J. Neville. Temporal-relational classifiers for prediction in evolving domains. In ICDM ’08: Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, pages 540–549, Washington, DC, USA, 2008. IEEE Computer Society. [23] C. Tantipathananandh, T. B. Wolf, and D. Kempe. A framework for community identification in dynamic social networks. In KDD ’07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 717–726, New York, NY, USA, 2007. ACM. [24] H. Tong, C. Faloutsos, and J.-Y. Pan. Fast random walk with restart and its applications. In ICDM ’06: Proceedings of the Sixth International Conference on Data Mining, pages 613–622, Washington, DC, USA, 2006. IEEE Computer Society. [25] H. Tong, S. Papadimitriou, P. S. Yu, and C. Faloutsos. Proximity tracking on time-evolving bipartite graphs. In SDM, pages 704–715, 2008. [26] T. Tylenda, R. Angelova, and S. Bedathur. Towards time-aware link prediction in evolving social networks. In SNA-KDD ’09: Proceedings of the 3rd Workshop on Social Network Mining and Analysis, pages 1–10, New York, NY, USA, 2009. ACM. [27] P. R. Winters. Forecasting sales by exponentially weighted moving averages. Management Science, 6:324–342, 1960.