Superpixel based crowd flow segmentation in H.264 compressed videos Sovan Biswas, R. Gnana Praveen and R. Venkatesh Babu Video Analytics Lab, Indian Institute of Science, Bangalore, India Objective

Intermediate Stages

Flow segmentation using H.264 motion vectors ◮ Unsupervised approach for segmentation of high density crowd flows ◮

Problem Formulation

(a)

(b)

(c)

(d)

(e)

(f)

Set of edges at each scale: Ek = {xi : ∀xi ∈ ∂Rk } S S ◮ Complete edge set: E = Ek ◮

k=1 ◮

Object edge set: Ef = {xi : ∀xi ∈ ∂Rg }

Overview: Block Diagram Motion Vectors

Preprocessing of Motion Vectors

Motion Vector Clustering by Superpixel Segmentation

Crowd Flow Segmented Output

Detection of Significant Edges

Intermediate stages: (a) Input Video Sequence (b) Color Coded Version (c) Confidence Score C1 (d) Confidence Score C2 (e) Final Confidence Score Cf (f) Flow Segmentation Quantitative Results Jaccard Index Similarity Measure with respect to ground truth

Final Crowd Flow Segmentation

Jaccard Similarity Measure Video Sequences Ali et al. Proposed Timings (secs) Sequence 1 0.63 0.60 4.96 Sequence 2 0.28 0.67 5.08 Sequence 3 0.57 0.74 4.66 Sequence 4 0.67 0.68 4.49 Sequence 5 0.78 0.24 4.32 Sequence 6 0.41 0.62 5.32 Sequence 7 0.54 0.15 4.95

The Proposed Approach ◮

Pre-processing of motion vectors ◮ median filtering ◮ camera compensation ◮ motion accumulation ◮ color coding

Qualitative Results

(a) Sequence 2 Clustering of motion vectors by super-pixels ◮ Detection of true edges using confidence scores ◮ C1: Confidence occurs across scales Es C1 = S ◮ C2: Confidence based on similarity of flows D(∠θSi − ∠θSj ) C2 = π ◮ Cf : Overall edge confidence Cf = C1 × C2 ◮ Final crowd flow segmentation ◮ Rτ : Edge map after thresholding Cf ◮ Select the scale which best matches Rτ ∗ Eki = argmax J(Rτi , Ek )

(c) Sequence 4

◮

Ek

E-mail: [email protected], [email protected]

(b) Sequence 3 (d) Sequence 6 Experimental results of few of the videos. First column of images shows input video sequence, second column of images show Ali et al. and third column shows the output of the proposed approach Conclusion ◮

Proposed an unsupervised approach for crowd flow segmentation on H.264 motion by combining edge confidence across various scales. Website: val.serc.iisc.ernet.in