Analysis of Algorithms Graphs BFS, DFS, and Minimum Spanning Trees
Ibrahim Mesecan Page 1
BFS, DFS, and MST
Important Note In this lecture series, I have used some parts and notes from the books: Data Structures and Algorithm Analysis in C++ 2nd editon Author: M.A. Weiss Introduction to Algorithms (2nd Edition) Authors: Cormen / Leiserson / Rivest / Stein And from the Lecture notes of Prof. Charles Leiserson Prof. Erik Demaine. (http://ocw.mit.edu). If I miss referencing anybody please inform me at
[email protected] This material is free for non-commercial and public use. For any reuse or distribution, you must make clear to others the license terms of this work. Ibrahim Mesecan Page 2
BFS, DFS, and MST
Breadth First Search (BFS) Figure below shows an un-weighted graph, G. Using some vertex, s, which is an input parameter, we would like to find the shortest path from s to all other vertices. We are only interested in the number of edges contained on the path, so there are no weights on the edges. 1 2
3
Ibrahim Mesecan Page 3
4 6
5 7
BFS, DFS, and MST
Breadth First Search (BFS) When studying single source shortest path we already studied BFS. To summarize, we will use unweighted shortest path. This is clearly a special case of the weighted shortestpath problem, since we could assign all edges a weight of 1. For now, suppose we are interested only in the length of the shortest paths, not in the actual paths themselves. Keeping track of the actual paths will turn out to be a matter of simple bookkeeping. Ibrahim Mesecan Page 4
BFS, DFS, and MST
Breadth First Search (BFS) Suppose we choose s to be v3. Immediately, we can tell that the shortest path from s to v3 is then a path of length 0. We can mark this information, obtaining the graph in Figure.
1 3
2 4
5
0
6 Ibrahim Mesecan Page 5
7 BFS, DFS, and MST
Breadth First Search (BFS) Now, we can start looking for all vertices that are a distance 1 away from s. These can be found by looking at the vertices that are adjacent to s. If we do this, we see that v1 and v6 are one edge from s. This is shown in figure below. 1 2 1
3
4
5
0
6 Ibrahim Mesecan Page 6
7
1
BFS, DFS, and MST
Breadth First Search (BFS) We can now find vertices whose shortest path from s is exactly 2, by finding all the vertices adjacent to v1 and v6, whose shortest paths are not already known. This search tells us that the shortest path to v2 and v4 is 2. Figure below shows the progress that has been made so far.
1 1
3 0 Ibrahim Mesecan Page 7
2 2
2
4
5
1
6
7
BFS, DFS, and MST
Breadth First Search (BFS) Finally we can find, by examining vertices adjacent to the recently evaluated v2 and v4, that v5 and v7 have a shortest path of three edges. All vertices have now been calculated, and so figure below shows the final result of the algorithm.
1 1
3 0 Ibrahim Mesecan Page 8
2 2
2
4
5
1
6
3
7 3
BFS, DFS, and MST
Breadth First Search (BFS) This strategy for searching a graph is known as breadth-first search (BFS). It operates by processing vertices in layers: the vertices closest to the start are evaluated first, and the most distant vertices are evaluated last. This is much the same as a level-order traversal for trees. 1 2 1
3 0 Ibrahim Mesecan Page 9
2
2
4
5
1
6
3
7 3
BFS, DFS, and MST
Depth First Search (DFS) In fact, if we think BFS is like queues, then the DFS is like stacks. Thus, In BFS we process all unprocessed descendents of the current element and then we go to the next element in the queue. On the other hand, in DFS, whenever we encounter an unprocessed element, that element becomes the active element and we start processing the descendents of that element. Ibrahim Mesecan Page 10
BFS, DFS, and MST
Depth First Search (DFS) void Dfs( Vertex V ) { visited[ V ] = true; for each W adjacent to V if( !Visited[W] ) if (dest(W)) decode_Path(); else Dfs( W );
visited[ V ] = false; Ibrahim Mesecan Page 11
} BFS, DFS, and MST
Depth First Search (DFS) Let’s assume that we want to go from vertex 3 to vertex 5.
1
2
Start
3
4 6
Ibrahim Mesecan Page 12
5 7
BFS, DFS, and MST
Destination
Depth First Search (DFS) We initially sign the source vertex as the distance 0, then we select the first descendent of the active element. This new element becomes the new active element.
1
2
3
4
5
0
6
7 1
Ibrahim Mesecan Page 13
BFS, DFS, and MST
Depth First Search (DFS) Now, we search for the successors of vertex 6. But because there is no vertex that we can reach from 6, after signing vertex 6 as unvisited, it turns back to parent vertex (vertex 3). Then, it checks for the other descendents of vertex 3.
1
1
2
0
3 Ibrahim Mesecan Page 14
4 6
5 7
BFS, DFS, and MST
Depth First Search (DFS) From vertex 1, it checks the first descendent (vertex 4) and then it signs that vertex as the active vertex.
1
Ibrahim Mesecan Page 15
1
2
0
2
3
4 6
5 7
BFS, DFS, and MST
Depth First Search (DFS) And then it starts checking the successors of vertex 4. Again it arrived to vertex 6 and there is no vertex leaving from 6, thus it turns back to the parent vertex (vertex 4).
1
1
2
0
2
3
4
5
3 Ibrahim Mesecan Page 16
6
7 BFS, DFS, and MST
Depth First Search (DFS) Then, it checks if there is any other descendent. Thus, it reaches to the destination. Because it already reached to the destination it finishes and prints the path.
1
Ibrahim Mesecan Page 17
1
2
0
2
3
3
4
5
6
7 BFS, DFS, and MST
Comparison BFS is a simple strategy in which the root node is expanded first, then all the successors of the root node are expanded next, then their successors, and so on. In general, all the nodes are expanded at a given depth in the search tree before any nodes at the next level are expanded. Thus for large graphs we need to keep all the entire graph in the memory. On the other hand DFS directly reaches to the first leaf. If there is no path when reaching to a leaf it removes the previously inserted elements. Thus, if the max depth is d, it requires only d elements in the array. Ibrahim Mesecan Page 18
BFS, DFS, and MST
Comparison But for unknown size elements, our depth can reach very big values. On the other hand, the destination might be the other successor (14) of the source. And, it will check the entire tree to reach to the next successor. 13
14
6
10
4
Ibrahim Mesecan Page 19
1
8
5
2
3
7
11
9
12
BFS, DFS, and MST
Minimum Spanning Trees A minimum spanning tree of an undirected graph G is a tree formed from graph edges that connects all the vertices of G at lowest total cost. A minimum spanning tree exists if and only if G is connected. Although a robust algorithm should report the case that G is unconnected, we will assume that G is connected. 2 2
1
4
3
1 2
Ibrahim Mesecan Page 20
4
6
3 2
10
5
2
1
3
2
4
2 4
4
8
5
1
2
6 1
7
BFS, DFS, and MST
6
1
7
5
Prim's Algorithm for MST One way to compute a minimum spanning tree is to grow the tree in successive stages. In each stage, one node is picked as the root, and we add an edge, and thus an associated vertex, to the tree. At any point in the algorithm, we can see that we have a set of vertices that have already been included in the tree; the rest of the vertices have not. The algorithm then finds, at each stage, a new vertex to add to the tree by choosing the edge (u, v) such that the cost of (u, v) is the smallest among all edges where u is in the tree and v is not. Ibrahim Mesecan Page 21
BFS, DFS, and MST
Prim's Algorithm for MST Initially, the start vertex (1) is selected and there is no other selected vertex. We select an edge whose one side is selected and the other vertex is not. That’s vertex 4.
1
2 1
3
4 6
Ibrahim Mesecan Page 22
5 7
BFS, DFS, and MST
Prim's Algorithm for MST On the next step we again, search for another safe edge is added to the system.
2
1 4
3
1 2
6
Page 23
4
3 2
1
10
5
2
2
1
3
4
5
4
8
5
Ibrahim Mesecan
2
6 1
7
6
BFS, DFS, and MST
7
Prim's Algorithm for MST On every other step we add another vertex which is not in the system. Thus, 5 is not in the system and is one of the cheapest vertex to add. 2
1 4
3
1 2
6
Page 24
4
3 2
1
10
5
2
2
1
3
4
2
4
8
5
Ibrahim Mesecan
2
6 1
7
6
BFS, DFS, and MST
7
5
Prim's Algorithm for MST When we are done, all the vertices are connected with the cheapest edges.
2
1 4
3
1 2
6
Page 25
4
3 2
2
1 10
5
3
2
4
7
2 4
6 1
2
1
4
8
5
Ibrahim Mesecan
2
6
BFS, DFS, and MST
1
7
5
Prim's Algorithm for MST
Ibrahim Mesecan Page 26
We can see that Prim's algorithm is essentially identical to Dijkstra's algorithm for shortest paths. As before, for each vertex we keep values dv and pv and an indication of whether it is known or unknown. dv is the weight of the shortest arc connecting v to a known vertex, and pv, as before, is the last vertex to cause a change in dv. The rest of the algorithm is exactly the same, with the exception that since the definition of dv is different, so is the update rule. For this problem, the update rule is even simpler than before: After a vertex v is selected, for each unknown w adjacent to v, dw = min(dw , cw,v). BFS, DFS, and MST
Kruskal's Algorithm for MST A second greedy strategy is continually to select the edges in order of smallest weight and accept an edge if it does not cause a cycle. Formally, Kruskal's algorithm maintains a forest - a collection of trees. Initially, there are |V| single-node trees. Adding an edge merges two trees into one. When the algorithm terminates, there is only one tree, and this is the minimum spanning tree. Figure next shows the order in which edges are added to the forest.
Ibrahim Mesecan Page 27
BFS, DFS, and MST
Kruskal's Algorithm for MST
Ibrahim Mesecan Page 28
BFS, DFS, and MST
Kruskal's Algorithm for MST The algorithm terminates when all edges are accepted. It turns out to be simple to decide whether edge (u, v) should be accepted or rejected. The appropriate data structure is the Union/Find algorithm can be used. The invariant we will use is that at any point in the process, two vertices belong to the same set if and only if they are connected in the current spanning forest
Ibrahim Mesecan Page 29
BFS, DFS, and MST
Kruskal's Algorithm for MST
Ibrahim Mesecan Page 30
void Kruskal( Graph G ) { int EdgesAccepted; DisjSet S; PriorityQueue H; Vertex U, V; SetType Uset, Vset; Edge E; /* 1*/ Initialize( S ); /* 2*/ ReadGraphlntoHeapArray( G, H ); /* 3*/ BuildHeap( H ); /* 4*/ EdgesAccepted = 0; /* 5*/ while( EdgesAccepted < NumVertex – 1){ /* 6*/ E = DeleteMin( H ); /* E = (U,V) */ /* 7*/ Uset = Find( U, S ); /* 8*/ Vset = Find( V, S); /* 9*/ if( Uset != Vset ){ /*10*/ EdgesAccepted++; /*11*/ SetUnion( S, USet, VSet ); } } }
BFS, DFS, and MST
Conclusion
Ibrahim Mesecan Page 31
There are Depth First Search and Breadth First Search algorithms in graphs. In trees, the DFS is much easier. But, BFS is more common with graphs. BFS requires more space, DFS requires an array with the size d where d is the deepest point in the graph. There are two main Minimum spanning tree algorithms: Prim’s and Kruskal’s Prim’s algorithm is like Dijkstra’s. Kruskal’s algorithm merges two separate sets of graphs in every step. BFS, DFS, and MST
