Segmented Trajectory based Indexing and Retrieval of Video Data. Faisal I. Bashir, Ashfaq A. Khokhar, Dan Schonfeld. University of Illinois at Chicago, 851 S. Morgan St., Chicago, IL, 60607. {fbashir, ashfaq, ds}

Multimedia Systems Lab, UIC.


Organization of Talk Problem Statement System Design Preprocessing towards Subtrajectory based Modeling  

Segmentation based on Curvature Zero-Crossings Clustering based on Polynomial Coefficients

Dimensionality Reduction  

Principal Component Analysis Singular Value Decomposition

Global PCA Based Modeling Segmented PCA Based Modeling Simulation Results  

Definition of Precision-Recall Metrics Precision-Recall Curves Multimedia Systems Lab, UIC.


Problem Statement & Motivation Index the Video Object Trajectories compactly and retrieve the best matching ones efficiently from the Video Database. For feature space representation of dynamic video content, object motion stands out as the best cue since it captures the rich dynamic content of video clips.

Sample Trajectories in the database. Gridlines imposed for temporal resolution. Multimedia Systems Lab, UIC.


System Design

System Block Diagram Multimedia Systems Lab, UIC.


Segmentation Based on Curvature Zero-Crossings Segmentation is based on Curvature and its dominant zero-crossings. &x&

κ =

[1 + x& ] 2


x& [ n ] = x [ n ] − x [ n − 1]


&x&[ n ] = x& [ n ] − x& [ n − 1]

Threshold & Merge algorithm: Run lengths of positive & negative coefficients.  Segmentation points is declared at Z.C. between two Major runs of opposite signs. 

Segmented Subtrajectories are normalized for spatial invariance.


Segmentation Results Segmentation Based on Curvature Zero-Crossings.

Segmentation of y-components of four sample trajectories. Multimedia Systems Lab, UIC.


Clustering Based on Polynomial Coefficients Subtrajectories are clustered to generate groups of similar subtrajectories for 2-level PCA in next step. Each normalized subtrajectory (coordinate value in 0-1 range) is fit with polynomial of order M. Clustering is performed on M+1 polynomial coefficients for each subtrajectory. This clustering facilitates two-level, coarse-to-fine matching of query trajectory. Multimedia Systems Lab, UIC.


Principal Component Analysis (PCA) for Dimensionality Reduction Data-dependant Orthonormal bases (PCs) as opposed to generic bases in DFT,DWT etc. Let X be a vector of p-random variables:  

linear function α 1′ x of the elements of x with maximum variance. linear functionα 2′ x, uncorrelated withα 1′ x , with maximum variance, and so on.

If Covariance matrix is known then kth PC is its eigenvector corresponding to kth largest eigenvalue. Multimedia Systems Lab, UIC.


PCA (Cont’d) Projection: y = Φ ′q x Y is maximally uncorrelated: det(Σ y ) is maximized. How many PCs to be retained? m

∑λj t m = 100 ×

j =1 p

∑λj j =1

Multimedia Systems Lab, UIC.


Singular Value Decomposition (SVD) Eigenvalue-Eigenvector decomposition of a symmetric matrix: A = QΛQ T For Rectangular matrix, rank r: A = USV T  T σ1 0 0 0V1  0 σ  T  0 0 2 V2  Anxm = [U1 U2 U3 U n ]  0 0 σ r 0V T    3   0 0 0 0V T   m

First r columns of U: Orth. Bases for Col Sp of A. Last n-r … U: … Null-Space of AT First r … V: … Row-Space of A. Last m-r …V: … Null Space of A. Multimedia Systems Lab, UIC.


System Design

System Block Diagram Multimedia Systems Lab, UIC.


Global PCA Based Modeling Let xk denote the ()-dimensional) kth trajectory and there be M total trajectories to be indexed. An )xM size matrix X is then formed with each column representing one trajectory. PCA computation using SVD yields the PC transformation matrix for trajectories: Φ q q-dimensional PCA coefficients for each trajectory are given by: Y = Φ′q X Euclidean Distance is computed between PCA coefficients of query trajectory and all indexed trajectories. Distance measures for x- and y- projections are combined and ranked list displayed to the user. Multimedia Systems Lab, UIC.


Segmented PCA Based Modeling Reduces Dimensionality of PCs estimation problem. Partial Queries can also be evaluated. First-Level PCA is performed on Cluster Centroids for each Cluster of similar subtrajectories. Second-level PCA is then performed on Subtrajectories inside each cluster separately. Multimedia Systems Lab, UIC.


Two-Level PCA Computation Let xk denote the ()-dimensional) kth cluster of subtrajectories and there are M total clusters to be indexed. An )xM size matrix Xc is then formed with each column representing one cluster. PCA computation using SVD yields the PC transformation matrix for clusters Φ qc q-dimensional PCA coefficients for each cluster of subtrajectories are given by: Yc = Φ′qc X c Second-level PCA is similarly performed on trajectories inside each cluster: Yt = Φ′qt X t Multimedia Systems Lab, UIC.


Segmented PCA Query Processing Two lists, Ekx and Eky (matching score of query trajectory’s x- & y-components with kth trajectory’s x- & y-components) are generated as: M

E kx =


∑ ∑

l =1 i =1 M

E ky =


∑ ∑

l =1 i =1

min 2 ( E k i lx , µ )

, k = 1, …P. min 2 ( E k i ly , µ )

Ekx and Eky are combined into a single list E, which is sorted and displayed to user. Multimedia Systems Lab, UIC.


Simulation Results: Ranked List of Trajectories

Five top ranked retrieved trajectories for top-left trajectory as query. Multimedia Systems Lab, UIC.


Precision-Recall Metrics Ground-Truth is established first. Queries are posed. For each value of Recall, Precision is computed from retrieved list. Definition of Prob. Of Detection, Pd and Prob. Of False Alarm, Pf is: P f = yP( y H c )dy Pd = yP( y H T )dy


In terms of ranked list of items retrieved for a query, the Prob. Of Precision and Recall are then derived as: c c T − X ∈ T i X i ∈T Pp = 1 − P f = Pr = Pd = T Tc Multimedia Systems Lab, UIC.


Simulation Results: Full Trajectory Matching

Precision Recall curves for full trajectory query. Multimedia Systems Lab, UIC.


Simulation Results: Partial Trajectory Matching

Precision Recall curves for partial trajectory query. Multimedia Systems Lab, UIC.


Conclusions Two PCA Based approaches for Trajectory indexing and retrieval discussed. Global PCA – Computation intensive, works better when full motion trail is available. Segmented PCA – Reduces dimensionality for PCs estimation problem, works good when only partial trajectory is available (due to occlusion, etc).

Multimedia Systems Lab, UIC.


Questions ??? Contact Information : Faisal I. Bashir. [email protected]

Multimedia Systems Lab, UIC.


Segmented Trajectory based Indexing and Retrieval of Video Data.

Indexing and Retrieval of Video. Data. Multimedia Systems Lab, UIC. 1. Faisal I. Bashir, Ashfaq A. Khokhar, Dan Schonfeld. University of Illinois at Chicago,.

2MB Sizes 1 Downloads 210 Views

Recommend Documents

Segmented Trajectory based Indexing and Retrieval of ...
Multimedia Systems Lab, UIC. 1. Faisal I. Bashir, Ashfaq A. ... Systems Lab, UIC. 4. System Block Diagram .... Of Detection, P d and Prob. Of False Alarm,. P f is:.

A Motion Trajectory Based Video Retrieval System ...
learning and classification tool. In this paper, we propose a novel motion trajectory based video retrieval system. For feature space representation, we use two ...

Real-Time Motion Trajectory-Based Indexing and ...
of the object trajectory in this setting include tracking results from video trackers .... An important application area of trajectory-based indexing is human activity ...

Real-Time Motion Trajectory-Based Indexing and ...
gained significant interest in scientific circles lately. This is primarily due to ...... M.S. and Ph.D. degrees in Electrical and Computer. Engineering from the Johns ...

Trajectory-based handball video understanding
Jul 10, 2009 - timeout) [19]. It relies on Gaussian mixtures and an EM al- gorithm trained on manually labeled sequences. Then, using a partition of the court, ...

Shape Indexing and Semantic Image Retrieval Based on Ontological ...
Retrieval Engine by NEC USA Inc.) provides image retrieval in Web by ...... The design and implementation of the Redland RDF application framework, Proc.

Shape Indexing and Semantic Image Retrieval Based on Ontological ...
Center retrieves images, graphics and video data from online collections using color, .... ular class of image collection, and w(i,j) is semantic weight associated with a class of images to which .... Mn is defined by means of two coordinates (x;y).

Three-Dimensional Segmented Trajectory Optimization ...
Quasi-Static Acoustic Mapping noise database for Blade-Vortex Interaction noise predic- tion and also population density distribution. Flight envelope limits are ...

Curvature Scale Space Based Affine-Invariant Trajectory Retrieval
represented as a trajectory can help mine more information about video data than without. On these lines of object trajectory based video retrieval, Chen et.

Content-based retrieval for human motion data
In this study, we propose a novel framework for constructing a content-based human mo- tion retrieval system. Two major components, including indexing and matching, are discussed and their corresponding algorithms are presented. In indexing, we intro

Video Retrieval Based on Textual Queries
Center for Visual Information Technology,. International Institute of Information Technology,. Gachibowli ... There are two important issues in Content-Based Video Ac- cess: (a) A .... matching algorithm then computes the degree of similarity be-.

Video Retrieval Based on Textual Queries
Audio and the textual content in videos can be of immense use ... An advanced video retrieval solution could identify the text ..... Conference on Computer.

A Comparison of Video-based and Interaction-based Affect Detectors ...
An online physics pretest (administered at the start of day 1) and posttest ... The study was conducted in a computer-enabled classroom with ..... detectors have been built to some degree of success in whole ..... Sensor-Free Affect Detection for a S

Indexing Shared Content in Information Retrieval Systems - CiteSeerX
We also show how our representation model applies to web, email, ..... IBM has mirrored its main HR web page at and canada.

In most of the existing content-based video indexing and re- ... There are 3 major tensor decomposition tools: HOSVD .... Comparison of HOSVD, PARAFAC and Multiple-SVD .... Proceedings of the IEEE International Conference on Im-.

we propose a criterion to select the best combination of parameters for Video-MMR. ... Marginal Relevance can be used to construct multi-document summaries ... is meaningful to compare Video-MMR to human choice. In a video set, 6 videos ...

Video Stream Retrieval of Unseen Queries using ...
Retrieval of live, user-broadcast video streams is an under-addressed and increasingly relevant challenge. The on-line nature of the problem ne- cessitates temporal evaluation and the unforeseeable scope of potential queries motivates an approach whi

Compressed Domain Video Retrieval using Object and ...
The object features such as speed, area and trajectory are then obtained after ... With the advent of broadband networks, high-powered workstations and ... (3) With the availability of hardware encoders and decoders for MPEG, such an.

Enhancing Image and Video Retrieval: Learning via ...
School of Computer Science and Engineering and the Center for Neural Computation. The Hebrew ... on the clustering and retrieval of surveillance data. Our.

Multimedia Systems: Content-Based Indexing and ...
result from single camera operation depicting one event is called a shot, a complete unit ...... Liang, K. C., and JayKuo, C. C. (1999). WageGuide: A joint wavelet-.

Mixed-Resolution Wyner-Ziv Video Coding Based on Selective Data ...
restore the interpolated portions for the spatial domain DVC. In this paper, we ... adaptive line dropping so-called selective data pruning (SDP). SDP is a simple ...