Collaborative Tracking of Objects in EPTZ Cameras Faisal Bashir Fatih Porikli Mitsubishi Electric Research Laboratories, Cambridge, MA 02139.

Overview • Problem Statement and Background – High-resolution wide area coverage – Object detection – Multi-kernel Mean-shift tracking

• Electronic Pan-Tilt-Zoom solution – System-camera interaction – EPTZ tracking algorithm

• Results Mitsubishi Electric Research Labs


Problem Statement and Approach • Problem – Given High Definition (1280x720) Video sequence (of a sports field), perform player detection and tracking generating high-resolution imagery of desired targets (players)

• Approach – Modern HD cameras deliver low resolution thumbnail (LRT) of wide area, as well as high resolution cropped image of the target region – We use these two time-stamped images to collaboratively detect and track players Mitsubishi Electric Research Labs


Challenges of HD Sports Tracking • Wide area coverage at high-resolution • Automatic detection of objects of interest • Tracking of detected objects

Mitsubishi Electric Research Labs


Wide Area Coverage at HighResolution • Traditional solutions – Scale existing narrow FOV systems using HD cameras • In our application, 16 static HD cameras will be needed using this approach!!!

– Master-slave architecture using fixed and PTZ cameras • Multiple independent cameras need to be carefully calibrated to joint world coordinate system Mitsubishi Electric Research Labs


Wide Area Coverage at HighResolution • Our approach – Scalable architecture that can be extended to higher resolution imagery – Single camera providing both wide area coverage and high resolution imagery of target – No need for calibration as relative homography between low resolution thumbnail and high resolution cropped image is trivially known Mitsubishi Electric Research Labs


Object Detection • Build a statistical model-based representation of the background – Model should be robust to variations in background appearances – Easy to update background and detect foreground objects

• Earlier approaches – Mixture of Gaussians (MoG) with expectationmaximization for parameter learning [Stauffer & Grimson ‘99] – Kernel density estimation for non-parametric background modeling [Elgammal et al ‘00] Mitsubishi Electric Research Labs


Object Detection • Recursive Bayesian learning for parameter estimation in MoG-based background representation [Porikli & Tuzel ‘05] – Recent history of each pixel is maintained by k layers (3D multivariate Gaussians) – Estimate the probability distribution of mean and variance for each layer

• Background model maintained at LRT size for low computational cost; projected to HD size for efficient background subtraction Mitsubishi Electric Research Labs


Object Tracking • Multi-kernel mean-shift tracking with background modeling in HRC view – Tracking in HRC view since in LRT view, objects too small for reliable tracking – Better object model (RGB Histogram) update in HRC view because more object data is available – Multi-kernel performs better because rapidly moving objects don’t spend much time in small FOV of HRC Mitsubishi Electric Research Labs


System-Camera Interaction HD






Process -BG Generation - Player Init




-BG Update


Process -Track -Object Model Update -Background Update

Process -BG Update


Process -Track -Object Model Update -Background Update

The original HD image in the camera as well as the two images LRT and HRC transferred to system for processing. Mitsubishi Electric Research Labs




Get Next LRT Image

Get Next LRT and HRC Images

Generate/Update Background

Generate/Update Background Model (LRT Only)

HD Tracking Algorithm



Player Exit/Lost

Flow chart of the algorithm to process LRT and HRC images from camera. It also highlights the role of semi-automatic player initialization and collaborative tracking.

Track in HRC Mean-Shift

NO Update Object Model using LRT Mask and HRC pixels




Mitsubishi Electric Research Labs


HD Detection and Tracking Scenario




(a) HD background maintained from the low resolution background and individual high resolution images. (b) Low resolution thumbnail (LRT) image of the whole FOV. Please note the very small object sizes. (c) EPTZ high resolution cropped (HRC) image of a detected player (pitcher) being tracked. Mitsubishi Electric Research Labs


Results 49












Collaborative HD player tracking in HRC view on base-ball sequence (1280x720). Mitsubishi Electric Research Labs










Collaborative HD human tracking in outdoor environment. (a) HD background image maintained through low resolution background and individual high resolution cropped images. (b) Low resolution thumbnail (LRT) image of the whole FOV. (c) EPTZ tracking result images in high resolution Mitsubishi Electric Research Labs


Results Table 1. Average per frame processing times for tracking in our collaborative solution as compared to HD only tracking. Collaborative LRT+HRC

HD Only

Background Update

50 mSec.

800 mSec.


20 mSec.

25 mSec.


5 mSec.

10 mSec.


75 mSec.

835 mSec.

Mitsubishi Electric Research Labs


References • • • • • • • • • • •

C. J. Needham and R.D. Boyle, Tracking Multiple Sports Players Through Occlusion, Congestion and Scale, British Machine Vision Conference, Manchester, UK, 2001. F. Rafi, S. M. Khan, K. Shafiq and M. Shah, Autonomous Target Following by Unmanned Aerial Vehicles, SPIE Defence and Security Symposium 2006, Orlando FL. F. Bashir, W. Qu, A. Khokhar and D. Schonfeld, HMM-based Motion Recognition System using Segmented PCA, IEEE International Conference on Image Processing, Genoa, Italy, 2005. W. Qu, D. Schonfeld and M. Mohamed, Distributed Bayesian Multiple Target Tracking in Crowded Environments using Multiple Collaborative Cameras, EURASIP J. Applied Signal Processing. 2007 (In print). S. M. Khan and M. Shah, A Multiview Approach to Tracking People in Crowded Scenes using a Planar Homography Constraint, 9th European Conference on Computer Vision ECCV 2006, Graz, Austria, 2006. X. Zhou, R. T. Collins, T. Kanade and P. Metes, A Master-Slave System to Acquire Biometric Imagery of Humans at Distance, ACM International Workshop on Video Surveillance, Nov. 2003. J. Migdal, T. Izo and C. Stauffer, Moving Object Segmentation using Super-Resolution Background Models, Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Oct. 2005. C. Stauffer and E. Grimson, Adaptive Background Mixture Models for Real-Time Tracking, in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Fort Collins, CO, Vol. II, 1999, pp. 246-252. A. Elgammal, D. Harwood and L. Davis, Non-parametric Model for Background Subtraction, in Proc. European Conference on Computer Vision, Dublin, Ireland, Vol. II, 2000, pp. 751-767. F. Porikli and O. Tuzel, Bayesian Background Modeling for Foreground Detection, ACM Int’l Workshop on Video Surveillance and Sensor Networks (VSSN), Nov. 2005, pp. 55-58. D. Comaniciu, V. Ramesh and P. Meer, Kernel-Based Object Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 25, No. 5, pp. 564-575, 2003. Mitsubishi Electric Research Labs


Thank You !!! [email protected] [email protected]

Mitsubishi Electric Research Labs


Collaborative Tracking of Objects in EPTZ Cameras

Given High Definition (1280x720) Video sequence. (of a sports .... Planar Homography Constraint, 9th European Conference on Computer Vision ECCV 2006,.

3MB Sizes 1 Downloads 205 Views

Recommend Documents

Collaborative Tracking of Objects in EPTZ Cameras
A brief training period is required where statistics from a few frames are used to model the background .... meaningful tracking in the LRT view. On the other hand ...

Tracking Across Multiple Cameras with Overlapping ...
line is similar, we count on the color and edge clues which lead us to the correct results. There are .... and Automation, May 2006. [16] S. M. Khan and M. Shah, ...

Tracking Across Nonoverlapping Cameras Based On ...
identification using Haar-based and DCD-based signature,” IEEE Intl. Conf. on ... evaluation of multimodal biometric authentication using state-of-the-art systems ...

Vision substitution and moving objects tracking in 2 ...
Abstract. Vision substitution by electro-stimulation has been studied since the 60's. Camera pictures or movies encoded in gray levels are dis- played via an ...

Continuously Tracking Objects Across Multiple Widely Separated ...
The identities of moving objects are maintained when they are traveling from one cam- era to another. Appearance information and spatio-temporal information.

Adaptive Fragments-Based Tracking of Non-Rigid Objects Using Level ...
Chan-Vese manner, using the framework of level sets to pre- serve accurate ..... ings of the IEEE Conference on Computer Vision and Pattern. Recognition (CVPR), 2006 ... tic space-time video modeling via piecewise GMM. IEEE. Transactions ...

segmentation and tracking of static and moving objects ... - IEEE Xplore
ABSTRACT. In this paper we present a real-time object tracking system for monocular video sequences with static camera. The work flow is based on a pixel-based foreground detection system followed by foreground object tracking. The foreground detecti

Adaptive Vision-Based Collaborative Tracking Control ...
robotic system to move along a predefined or dynamically changing trajectory, the regulation result in [6] was extended in [3] to address the UGV tracking ...

Pay-per-Tracking: A Collaborative Masking Model for ... adnxs.c

An Optimal Approach to Collaborative Target Tracking ...
Semidefinite programming·Second-order cone programming·Multi-agent systems ...... Advanced Optimization Laboratory: Addendum to the SeDuMi User Guide Version 1.1 ... Taipei, Taiwan. (2004).

installation of CC cameras and Biometric atendance in ... - stu telangana
Aug 30, 2016 - Sub:- Model Schools - Telangana – Procurement & Installation of CC TV Cameras and Biometric Attendance Systems to all Model schools ...

Reduced-power GPS-based system for tracking multiple objects from ...
May 12, 2000 - signal is tracked through code-correlation at the receiver. This provides the ...... '51) can be easily recorded by an inexpensive timer located at.

Reduced-power GPS-based system for tracking multiple objects from ...
May 12, 2000 - ping or cargo containers, trucks, truck trailers, automobiles, etc. can be .... include monitoring channel status and control, signal acqui sition and ...

Reduced-power GPS-based system for tracking multiple objects from ...
May 12, 2000 - the propagation time differences of the signals transmitted from the ..... code-time offset for each of the satellite signals and for determining the ...

A Modified KLT Multiple Objects Tracking Framework ...
RGB template model to modify traditional KLT tracker more robust .... the current template model by: Ti(x) = µTi (x) + dn. ∑ j=1. λjE j i (x), for. (µTi ,E j i ) = PCA(Î1.

installation of CC cameras and Biometric atendance in ... - stu telangana
Aug 30, 2016 - PROCEEDINGS OF THE DIRECTOR OF SCHOOL EDUCATION & EX – OFFICIO PD, ... RC.No. 9876/l/B1/MS/TS/2016, dated 25-6-2016 and.

PDF Download Calibration and Orientation of Cameras in Computer ...
Cameras in Computer Vision (Springer Series in. Information Sciences) ... modelling, system calibration, self-calibration and error handling. The chapters have ...

Detecting Junctions in Photographs of Objects
3 junction points and contour points near junctions will theoretically not be detected well for any ... junctions is one way to reason globally about the image. .... call this probability Ci(x, y, θ, r), where Ci is the ith channel (in this case we

Evaluation of Profile Injection Attacks In Collaborative ...
open nature of collaborative filtering allows attackers to inject biased profile data and force ... used for recommendation, rather than the original user data. PLSA is a “fuzzy” ..... on-line sources of the popular horror films:

Transfer learning in heterogeneous collaborative filtering domains
E-mail addresses: [email protected] (W. Pan), [email protected] (Q. Yang). ...... [16] Michael Collins, S. Dasgupta, Robert E. Schapire, A generalization of ... [30] Daniel D. Lee, H. Sebastian Seung, Algorithms for non-negative matrix ...

Experiences with the Impact of Tracking Technology in Mobile ...
mobile phones still employ sensor-based tracking, and typi- cally produce inaccurate .... ment paper maps with digital content retrieved from an on- line source.