Puzzle: Maintaining the median

Given a list of n numbers, the median is the number of rank dn/2e, i.e., it is the dn/2eth smallest element. Suppose the list is changing dynamically. That is at every step, a number is added or deleted from the list. The goal is to output the median in the current list as fast as possible. How fast can we do it?

G. Pandurangan

1

Priority Queue (Heap) Data Structure

In many applications, we want to maintain the highest (or lowest) priority item among a set of items that are dynamically changing — i.e., items are added and deleted. Priority queue (also commonly called as Heap) is a data structure that supports 3 operations: • INSERT(P, k): Insert key k (along with its data) into priority queue P . • MIN(P ): Return the smallest key (along with its data) in P . • EXTRACT-MIN(P ): Return and remove the smallest key (along with its data) from P . The above is called a MIN heap. Alternatively, we can define a MAX heap which has INSERT, MAX and EXTRACT-MAX operations. G. Pandurangan

2

Implementing a priority queue

One simple way is using an array: store the keys in an array. There are two ways to store the keys in an array — sorted or unsorted which determines the time needed for the 3 operations. Assume that there are n keys currently in the array A — A[1], A[2], . . . , A[n]. (We will assume that keys are all distinct.) We now perform an INSERT, MIN, or EXTRACTMIN operation. The costs are: • Unsorted array: INSERT (O(1)), MIN (O(1)), EXTRACT-MIN (O(n)) — Why? • Sorted array: INSERT (O(n)), MIN (O(1)), EXTRACT-MIN (O(1)) — Why?

G. Pandurangan

3

Binary heap: A more efficient implementation Store the keys as a complete binary tree. A binary tree is a tree where each node has at most two children. It is complete if all levels are “full” except (possibly) the last level, i.e., nodes in all levels, (except possibly the last) will have 2 children. The last level will be filled from the left. A complete binary tree example:

G. Pandurangan

4

Binary Heap

A binary (min) heap stores its keys in a complete binary tree, i.e., each node stores one key. Additionally, the keys are stored so that they satisfy the following (min) heap property: The key stored in any node will be smaller than the keys stored in its children (if any). Example:

 

  

G. Pandurangan

 







5

Representing a binary heap

A binary heap can be conveniently represented using a vector (or an array). The indices of the vector capture the parent-child relationship. Let A be a vector. The number of the elements in the vector is given by A.size(). The root of the binary heap is stored in V [0]. Given element A[i], the children of this element are stored in A[2i + 1] and A[2i + 2], if they exist. Left child of i denoted as lef t(i) = A[2i + 1]. Right child of i denoted as right(i) = A[2i + 2]. The parent of A[i] is stored in A[b i−1 2 c]. To navigate the tree we follow the corresponding indices of the vector. G. Pandurangan

6

Example

Shows the indices of the vector corresponding the nodes of a binary heap:

 

  

G. Pandurangan

 







7

Maintaining the heap property

We use a recursive algorithm called MIN-HEAPIFY to maintain the heap property. It takes as input a vector Aand an index i into the vector. MIN-HEAPIFY assumes that the binary trees rooted at lef t(i) and right(i) are min-heaps. But A[i] might be larger than the values at its children — violating the min-heap property. MIN-HEAPIFY moves the value of A[i] down in the min-heap so that the subtree rooted at index i obeys the min-heap property. MIN-HEAPIFY will be used to build a heap and also implement the priority queue operations.

G. Pandurangan

8

MIN-HEAPIFY

Input: Vector A and index i, trees rooted at lef t(i) and right(i) are min-heaps. Output: Tree rooted at i is a min heap. Algorithm MIN-HEAPIFY((A, i)) l = lef t(i); r = right(i) if l ≤ A.size() and A[i] > A[l] smallest = l else smallest = i if r ≤ A.size() and A[smallest] > A[r] smallest = r if smallest 6= i Exchange A[i] with A[smallest] MIN-HEAPIFY(A, smallest)

G. Pandurangan

9

Example

 

  







 

 

  







  



  

G. Pandurangan

 







10

Running time of MIN-HEAPIFY

The time taken by MIN-HEAPIFY(A, i) is proportional to the height of node i in the binary tree (Why ?). If there are n elements in the heap, the height of the complete binary tree is O(log n) (why ?). Hence MIN-HEAPIFY takes O(log n) time in the worst case.

G. Pandurangan

11

Building a heap

Suppose we are given n elements in a vector (or an array) and we want to make it into a binary heap. We can do this by calling MIN-HEAPIFY in a bottom-up manner. Input: A vector A of n elements. Output: Binary heap A. Algorithm Build-Min-Heap(A) n = A.size() for i = n down to 1 do MIN-HEAPIFY(A, i) endfor

G. Pandurangan

12

Correctness of Build-Min-Heap

Consider the complete binary tree representation of the array. MIN-HEAPIFY fixes the heap property level by level starting from the last level and going up. Thus when MIN-HEAPIFY is called at a node i, the subtrees rooted at lef t(i) and right(i) already satisfy the heap property. When MIN-HEAPIFY is finally called at the root, then whole array becomes a min heap.

G. Pandurangan

13

Running time of Algorithm Build-Min-Heap

It is easy to see to upper bound the run time of Build-Min-Heap as O(n log n). (Why ?) Actually, by a more careful analysis, it can be shown to be faster — takes O(n) time. We will bound the total time taken by all the MIN-HEAPIFY calls. The time taken by MIN-HEAPIFY(A, i) proportional to the height of node i.

is

G. Pandurangan

14

Running time of Algorithm Build-Min-Heap

Hence total time is bounded by: Pblog nc n h=0 d 2h+1 eO(h) n e nodes at height h. since there are at most d 2h+1 Hence, Pblog nc h P∞ h O(n h=0 2h ) = O(n h=0 2h ). P∞ h The sum h=0 2h = 2 (Show it!) P∞ h Hence O(n h=0 2h ) = O(n).

G. Pandurangan

15

Implementing Priority Queue Operations

The operations INSERT, MIN, and EXTRACT-MIN can be implemented using a binary heap. Given: A binary heap A. • MIN(A) is simply returning A[0]. Takes constant time. • EXTRACT-MIN(A): Takes O(log n) time. 1. min = A[0]. 2. A[0] = A[A.size()]. 3. Delete (last) element A[A.size()] from vector. 4. MIN-HEAPIFY(A, 0). 5. Return min. • INSERT((A, k)): Insert key k into A. Can be done in O(log n) time. (How ?) G. Pandurangan

16

Stack Data Structure

Stack is a Last-in-First-Out (LIFO) data structure that supports the following operations: 1. INSERT(S, k) or PUSH(S, k): Insert or push key k into stack S. 2. DELETE(S) or POP(S): Return and delete the most recently inserted key. 3. EMPTY(S): Return true if S is empty, otherwise return false. A stack can be easily implemented using a vector so that all the above operations take O(1) time. (How ?)

G. Pandurangan

17

Queue Data Structure

Queue is a First-in-First-Out (FIFO) data structure that supports the following operations: 1. INSERT(Q, k) or ENQUEUE(Q, k): Insert or push key k into Queue Q. 2. DELETE(Q) or DEQUEUE(Q): Return and delete the least recently inserted key, i.e, the key that was inserted the earliest. 3. EMPTY(Q): Return true if Q is empty, otherwise return false.

G. Pandurangan

18

Implementing a Queue

A Queue can be implemented using a vector (or an array) so that all the above operations take O(1) time. (How ?) However, one has to be careful about space used. Let’s say we use a vector A to implement a Queue: simply keep the elements in the order of insertion and keep track of the indices of the first and last element. ENQUEUE will insert the element as the last element in the vector. DEQUEUE will remove and return the first element in the vector; update the index of the first element. What is the implementation?

problem

with

the

above

How can we fix it ? Can you give another implementation of a Queue? G. Pandurangan

19