J Real-Time Image Proc (2013) 8:285–295 DOI 10.1007/s11554-011-0232-7

SPECIAL ISSUE

Real-time automatic license plate recognition for CCTV forensic applications M. S. Sarfraz • A. Shahzad • Muhammad A. Elahi M. Fraz • I. Zafar • E. A. Edirisinghe

•

Received: 30 September 2010 / Accepted: 21 October 2011 / Published online: 19 November 2011 Ó Springer-Verlag 2011

Abstract We propose an efficient real-time automatic license plate recognition (ALPR) framework, particularly designed to work on CCTV video footage obtained from cameras that are not dedicated to the use in ALPR. At present, in license plate detection, tracking and recognition are reasonably well-tackled problems with many successful commercial solutions being available. However, the existing ALPR algorithms are based on the assumption that the input video will be obtained via a dedicated, high-resolution, high-speed camera and is/or supported by a controlled capture environment, with appropriate camera height, focus, exposure/shutter speed and lighting settings. However, typical video forensic applications may require searching for a vehicle having a particular number plate on noisy CCTV video footage obtained via non-dedicated, medium-to-low resolution cameras, working under poor illumination conditions. ALPR in such video content faces severe challenges in license plate localization, tracking and recognition stages. This paper proposes a novel approach for efficient localization of license plates in video sequence and the use of a revised version of an existing technique for tracking and recognition. A special feature of the proposed approach is that it is intelligent enough to automatically adjust for varying camera distances and diverse lighting conditions, a requirement for a video forensic tool that may operate on videos obtained by a diverse set of unspecified, distributed CCTV cameras. M. S. Sarfraz  A. Shahzad  M. A. Elahi  M. Fraz Computer Vision Research Group, COMSATS Institute of Information Technology Lahore, Lahore, Pakistan I. Zafar  E. A. Edirisinghe (&) Digital Imaging Research Group, Loughborough University, LE11 3TU Loughborough, UK e-mail: [email protected]

Keywords License plate recognition  CCTV video footage  Traffic monitoring  Video indexing  Surveillance

1 Introduction The number of on-road motor vehicles has increased with the rapid growth of world’s economy and with this augmentation the need for security and monitoring of vehicles has also increased. It is necessary for officials to continuously examine the traffic to avoid/control congestion, overspeeding and unlawful activities that involves a vehicle. Many successful commercial systems that employ dedicated camera systems, providing video input captured under control environments to ALPR algorithms, exist at present [1, 3, 6, 9–11, 19, 24, 25]. However, application scenarios in video surveillance and forensics such as tracking down a stolen vehicle or searching for a vehicle involved in a crime, as identified by a bystander to be of a particular registration number, requires the painstaking task of manual search, because the existing ALPR systems are not capable of efficiently working on video footage obtained via non-dedicated (for ALPR) CCTV systems The non-deterministic camera positioning (height and angle), specifications (speed, focus, aperture), lighting conditions, presence of compression artifacts, high levels of noise etc. in CCTV systems pose a significant challenge to computer vision and pattern recognition algorithms used in existing ALPR systems. This paper presents an efficient and robust framework that can perform localization, tracking and recognition of multiple vehicle license plates in a real-time scenario (i.e., incoming video stream from low-resolution surveillance cameras). The major aim of carrying out this work is to make significant contribution to the efficacy improvement

123

286

of video indexing/annotation applications. The system is robust enough to learn any scenario and adjust itself according to any camera angle, height and distance from the road. The main contribution of this paper is a novel technique of detecting license plates in real time from lowquality surveillance video. The method detects the license plate of moving vehicles based on the geometry of its contours present in the foreground. The detection procedure is supplemented by a conventional template matching procedure in the initial learning phase to automatically learn and adjust itself to the plate size in the initial few frames. The detected plate is tracked by using a dynamic displacement method and finally the characters of the license plate are recognized using a simple nearest neighbor classifier. The main intention of tracking a vehicle’s license plate throughout the video stream is to enhance the efficiency of license plate character recognition using majority voting on a set of detected samples of the same license plate and to eliminate the fallaciously detected license plates by continuous assessment of the tracked plates. The idea of tracking a license plate instead of the whole vehicle relies on the fact that there are less chances of occlusion for license plate than for vehicle because of its smaller size when tracked in several frames. The rest of this paper is organized as follows. After a review of related works, the main framework is described in three sections, license plate localization, tracking and recognition. The experimental results are provided in Sect. 4, followed by discussion and conclusion in Sect. 5.

2 Related work In general, license plate recognition systems consist of two major parts, localization within a single frame of traffic video and character recognition. Donoser et al. [10] addressed the issues of detection, tracking and recognition together. They introduced a realtime framework that enabled detection, tracking and recognition of license plates from video sequences. Their detection algorithm is based on the analysis of a maximally stable extremal region (MSER) detection that differentiates the region of interest on the basis of intensity of the region as against the boundary of the region. However, MSER detection approach fails when the intensity of the license plate and/or characters on the plate are akin to the outer region. This effect is very common in variant outdoor conditions and motion blurred videos. Hough transform is another well-known technique for license plate detection that can prove useful in finding the boundary box of a license plate regardless of characters

123

J Real-Time Image Proc (2013) 8:285–295

[11, 23]. This is a quite efficient method, but it has high computational complexity and therefore not suitable for real-time applications. Chang et al. [3] proposed a license plate detection algorithm using color edge and fuzzy logic. However, their algorithm can only be used to detect the license plates with specific colors. Techniques based on learning such as Adaboost [9, 24] are also used for license plate detection. Simplicity and speed are the attractive features for Adaboost learning with respect to other classifiers. However, in comparison to edge-based methods, Adaboost is slow. Adaboost method fails to detect a license plate when the range of variations for distance or viewing angle increases. In the area of vehicle tracking, the standard approach for robust tracking in traffic consists of adopting sophisticated instruments like radars, as for example in [20]. However, this has the drawback of being very expensive in comparison to standard video cameras. Another method is the blob tracking [12]. In this approach, a background model without moving objects is generated for the scene. Each frame is compared with the background model by computing the absolute difference between them and consequently obtains a foreground blob representing the vehicles. The vehicle tracking literature almost universally relies on variants of Kalman filters [7], although particle filters and hybrid approaches have been widely used in other tracking applications. One of the earlier prominent works has been done by Koller et al. [14] who proposed a deformable contour-based vehicle tracking algorithm. The algorithm does not work well if the vehicle entering the scene is partially occluded. This is why the approach proposed in this paper directly focuses on license plate detection instead of detecting a vehicle and then localizing its license plate. Several other real-time techniques have been proposed in literature, for example using a mixture of sonar and vision information [13] or using stereo vision [15]. Another method proposed by [17] suggested mounting a few artificial landmarks on the car to be followed, while [4] using templates of a car’s back to perform the tracking. A number of approaches for recognizing the characters on a license plate after successful detection have been proposed in literature. Shapiro et al. [21] use adaptive iterative thresholding and analysis of connected components for segmentation. The classification task is then performed with two sets of templates. Rahman et al. [18] used horizontal and vertical intensity projection for segmentation and template matching for classification. Dlagnekov and Belongie [8] used the normalized crosscorrelation for classification by analyzing the whole plate, hence skipping segmentation.

J Real-Time Image Proc (2013) 8:285–295

3 Proposed framework This section describes the proposed framework comprising detection/localization, tracking and recognition of license plates in CCTV surveillance videos, as shown in the block diagram in Fig. 1. As a first step, the license plate is localized in the incoming frames. This requires some background learning and pre-processing to differentiate between license plates and other plate-like regions. The located plate is further tracked in each frame by continuous upgradation of background and finding the new location of the license plate. The detected plate(s) is/are enhanced and character recognition procedure is applied to recognize the characters of the license plate in each frame. 3.1 Background learning Efficient background extraction is a key step for moving object detection in a video sequence. A background image is required to represent the base state of the area under examination for detection purposes. The extracted background is subtracted from each frame to obtain the

287

foreground object (moving vehicle in this case). However, it is hardly possible to acquire an image of the observation area that does not contain any vehicles or other foreground objects. Thus, it is crucial to extract the background image from the video stream itself. The accuracy of the detection procedure depends on precise and rapid background estimation. In the proposed approach, background learning is performed by using the exponential forgetting technique [22]. The system begins its background computation by considering the very first frame as background and updates it with impending frames. Every updated background is a weighted sum of the previous background and the new frame. In this way, the background dynamically adapts the changes in the movement of objects or luminance conditions in the frame. Our detailed experiments revealed that the exponential forgetting technique performs efficiently in CCTV footage, in which movement of objects can be rather complex and no control exists over luminance variations. Mathematically, the background learning procedure can be described as follows:

Fig. 1 Overview of the proposed framework

123

288

Bnþ1 ¼ ð1  lÞBn þ lFn

J Real-Time Image Proc (2013) 8:285–295

ð1Þ

where ‘Fn’ is the current frame, ‘B’ is the background and ‘l’ is the background learning coefficient. Background learning procedure is illustrated in Fig. 2. The value of background learning coefficient ‘l’ is empirically set at ‘0.1’, i.e., for each new background estimation, 90% of the previously estimated background and 10% of the new frame values are incorporated. In the same way, for the subsequent frame, 90% of the previously calculated background values are considered that have 90% effect of their previous background values and so on. In this way, the effect of previously calculated backgrounds is not voided straightaway; instead, their effect is abridged frame by frame in an exponential manner. The first row shows a sequence of video frames on which background learning is applied. The second row shows the calculated background by using exponential background learning. The third row illustrates the foreground as calculated by subtracting the background from the original frame. 3.2 Pre-processing The current system is designed taking surveillance cameras, which usually record less frames per second and have

low resolution, into consideration. For this reason, preprocessing is required to minimize noise and for sharpening edge information in the frame. This enhances plate regions and improves detection efficiency. Figure 3 illustrates the effect of pre-processing. It can be seen that the original frame contains significant amount of salt and pepper noise and has poor edge information. Noise present in the foreground is removed by using well-known median filtering: a nonlinear technique that applies a sliding window on the frame pixels and removes noise while preserving the edge information. Morphological operators (erosion and dilation) are applied afterward to further refine the foreground. 3.3 License plate detection/localization After pre-processing, candidate regions (for license plate) are detected by finding the contours and boundaries of connected components in the frame. These connected components are further processed based on template matching criteria to void the false regions and decide about the true region of interest ‘ROI’. The ROI selection is carried out in two steps. First, the identified candidate regions are judged on the basis of their size and aspect ratio. If the size of any region is smaller or larger than a certain threshold, then that region is classified as a false

Fig. 2 Background learning and subtraction by using exponential forgetting method. a–e Sequence of original video frames. f–j Calculated backgrounds for the respective frames. k–o Background subtraction from the respective frames

123

J Real-Time Image Proc (2013) 8:285–295

289

Fig. 3 Effect of pre-processing (median filtering and morphological operators). a, d Original frames. b, e Binary images of foreground objects containing salt and pepper noise. c, f Pre-processed noisefree frames

region and is discarded from further processing. If the ‘width-to-height’ aspect ratio of any region is less than or greater than a certain threshold, then that region is also discarded for further processing. 3.3.1 Region of interest selection After neglecting the false regions on the basis of their size, the remaining candidate regions are checked on the basis of their texture similarity to license plate-like areas. Since we target the application of our approach to videos captured with CCTV camera installations of unknown specification, it is imperative that the method must cope with varying camera distances and/or height from the plane. To achieve this in real time, we propose using a new initial learning mechanism that accounts for and learns automatically the expected size of the number plate in the video. This is important since it reduces a possibly large set of candidate regions to be evaluated, to a very few, which results not only in improved detection accuracy but also in greater computational advantage. 3.3.1.1 Initial learning The initial learning is performed on the first few incoming frames by narrowing down the possible sizes of expected plate-like regions fulfilling the initial geometrical constraints (aspect ratio etc.). The range of possible plate sizes is taken between some upper and lower threshold that is the function of the frame size. The regions formed in this range are then checked on the basis of their texture similarity in a pure template matching fashion and this information is further used to refine the size range for that particular video. More specifically, we set the initial lower and upper thresholds for a rectangular region to be a license plate as

5–20% of the frame size, respectively. The upper limit is set to 20% of the frame size considering the application requirements that the camera view should be at least as wide as the width of a single lane on the road; if the detected plate is less than 5% of the frame size, it is almost impossible to extract the characters with acceptable accuracy. However, this initial threshold is automatically updated to narrow down the range on the basis of learning performed by the system. The learning procedure starts from the very first, frame in which the lower and upper threshold values are set at 0.05 and 0.2, respectively, of the frame size. For the initial few frames, these threshold values are kept unchanged by considering the possibility that true candidate regions may be of any size in this range. The candidate regions formed on the basis of these threshold values are processed by matching with the stored templates of example plates. The regions for which the respective normalized cross-correlation is above a defined threshold are considered and their size information is further used for updating upper and lower bound values. The process keeps updating the threshold values with each new candidate that fulfills the correlation criteria. It stops when the size does not change between few successive frames. Figure 4 illustrates the effect of the learning procedure. It is apparent that license plate sizes are learned and the range is narrowed with passing frames. For any candidate region, if the cumulative sum of correlation is more than a certain threshold, then that region is finalized as a license plate. Offline training has been done for finding the optimized correlation threshold (see Sect. 4.1). Figure 5 illustrates the output of various stages of the ROI selection stage. The implemented technique is robust

123

290

J Real-Time Image Proc (2013) 8:285–295

compute a nine-bin histogram of gradient magnitudes over gradient orientations; concatenating the histograms of all the partitions, we obtain a 144-dimensional feature vector for each candidate region. The final classification is carried out using a simple nearest mean classifier as given in Eq. 2, trained on an offline training set of license plate and nonlicense plate positive and negative examples. ( ) ffi rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi X X 2 2 C ¼ sign ðxi  xi Þ  ðxi  xiþ Þ ð2Þ i

Fig. 4 Threshold learning in various videos. In video 1, the camera was mounted quite close to the passing vehicles on the road. In video 2, the camera was mounted far away from on-road vehicles

enough that it automatically decides and adapts the threshold for the size of license plate in a video with respect to the overall frame size on the basis of learning. This results in improved computational efficiency, as only a few plate-like regions, formed on the basis of the learned size, will be evaluated for the final detection/localization purpose in the rest of the video frames. 3.3.2 License plate localization The final selection of the license plate is achieved by extracting relevant features and classification. For this purpose, we use a histogram of oriented gradients HOG [5] for feature description. The HOG features are extracted from the candidate plate-like regions in a slightly adapted manner for our purpose. Each candidate region is partitioned into 16 non-overlapping blocks. For each block, we Fig. 5 False candidate region rejection. a, d Show that a number of false candidate regions detected. b, e Are obtained by applying the limiting criteria of window sizes. c, f Are finalized true candidate regions after applying the correlation criteria

123

i

where x is the HOG feature vector of the test region and xþ and x are the mean vectors of the positive and negative examples, learned from the training set. The term under square root determines the Euclidean distance. This relatively naive classification strategy is adopted for simplicity keeping in view the real-time requirements of the intended application. The overall detection scheme gives very promising results in terms of accurate localization and computational efficiency. 3.4 License plate tracking After detection of the license plate in the video sequence, the next step is to keep track of that license plate with the movement of a vehicle in the consecutive frames. The real idea of applying a tracker is to provide a supplementary resource for the localization and extraction processes. The tracking of license plate can serve wider practical issues as well. The most important of them is the traffic parameter estimation. The tracking of license plates also helps in video indexing by using the plate characters. When a new car appears in the frame, its license plate is detected and passed on to an efficient tracker to keep track of the vehicle by its license plate in the video sequence. The tracker returns a number of shifted samples of the

J Real-Time Image Proc (2013) 8:285–295

291

same license plate that are testified on the same correlation criteria used in localization. If the plate retains the minimum threshold limit, it is extracted and the result is finalized by majority voting; otherwise, the region is considered as a false candidate and rejected in the next frames. In every frame, the license plates are detected independently. Tracking in this situation is simply connecting these detections in successive frames. For every license plate, the coordinates in the current frame are passed to a modified Lucas Kanade’s tracker that predicts the position of the respective license plate in the next frame by computing the displacement using a constant acceleration dynamic model [2, 16] as follows: I ðx; y; tþsÞ ¼ I ðx  n; y  g; tÞ

ð3Þ

where, ‘I’ is a window of pixels, ‘x’ and ‘y’ are new x and y coordinates of the object (license plate), and ‘n’ and ‘g’ are the displacement values for the previous x and y coordinates, respectively. The later image taken at time t ? s can be obtained by moving every point in the current image, taken at time t, by a suitable amount. The amount of motion d = (n, g) is called the displacement of the point at X = (x, y) between time instants t and t ? s, and is in general a function of x, y, t and s [2, 16]. A window of size N is used to gather more information of the texture around the feature point, as the value of a single pixel can change due to noise. In our approach, the detected license plate’s top left coordinate is the feature point used. In this way, the gradient matrix M is computed as:  N  2 X Ix Ix Iy M¼ ð4Þ Ix Iy Iy2 i¼1

Note that the gradient matrix M denotes the standard Hessian matrix of the window centered on the point to be tracked (the top left corner of the detected license plate in our case). Ix and Iy in Eq. (4) are therefore the directional derivatives at each pixel in that window. The sum over subscript i represents that the final Hessian M is obtained by summing these directional derivatives on all the i pixels in that window. The displacement d = (n, g) of a feature is computed to minimize the residue error (e). The displacement d is calculated by iteratively solving the following equation for Dd (where Dd = d): M:e ¼ d

ð5Þ

where N X e¼ ðIi  Ji Þ½Ixi  Iyi T

ð6Þ

i¼1

I and J are the two consecutive images. This is done to minimize the nonlinear error. Each time a new Dd is calculated, the value for d is updated by:

dði þ 1Þ ¼ d ðiÞþDd

ð7Þ

Until the condition norm (Dd) \ e is satisfied. Then the coordinate of the feature point is updated by adding d to the formal coordinate x(t): x ð t þ 1Þ ¼ x ð t Þ þ d

ð8Þ

where, x (t ? 1) is the updated coordinate of the feature point for the next frame. Figure 6 illustrates the operation of tracking in six consecutive frames of a surveillance video sequence, where multiple license plates are accurately detected. 3.5 License plate recognition The final step of the framework is to recognize the characters of the detected plates. The localized plate is preprocessed for noise removal and enhancement of edge information. After pre-processing, connected regions are identified in the enhanced image, and bounding boxes are drawn around them using minimum boundary rectangle ‘MBR’. This way, each character is separately enclosed by a bounding box. Each separated character is then classified using a simple nearest neighbor classifier. Note that the classifier requires the binary maps of the segmented characters. The recognition process is illustrated in Fig. 7. The classifier is trained using an offline training set made from a set of standard characters as well as manually cropped characters (10 examples each) from few of the videos (not used in the evaluation). A key point of consideration is, due to poor lighting conditions or low-quality video, significant chances of error may arise in the recognition of the plate characters in a single iteration. To reduce the probability of error, each number plate is read in every frame and compared with its previously recognized result; if a recognition error is found in the previous result, then it is corrected. Since we are tracking the license plate, the result is finalized by majority voting of results from all the instances of the license plate images. Figure 8 shows the majority voting for license plates, as shown in Fig. 7.

4 Experiments and results The proposed framework is evaluated on a set of CCTV road surveillance videos obtained for general purpose and manual inspection, i.e., the cameras used to capture footage are not dedicatedly selected or set up for automatic license plate detection and recognition. The resolution varies from a maximum of (1,024 9 768) to a minimum of (360 9 288). The framework processes the specified videos for all three operations (detection, tracking and

123

292

J Real-Time Image Proc (2013) 8:285–295

Fig. 6 Tracking of license plates in six consecutive video frames

Fig. 7 Recognition of license plate character: a binary map of detected license plate; b character segmentation using minimum boundary rectangle on connected components; c recognized characters using nearest neighbor search 20 18

Occurrence

16 14 12 10 8 6 4 2 0 N

J

L

S

5

U

6

E

8

9

R

B

H

Z

A

Characters

Fig. 8 Effect of majority voting. Various characters are recognized in 30 consecutive frames, and the recognition result is finalized depending on the occurrence of each character

recognition of license plates) at a maximum of 35 ms per frame, which means approximately 28 fps. This includes the majority voting of detected license plates in each frame for improving recognition efficiency. The system is implemented using C on a 2.3-GHz dual core machine. The comparison of execution time with contemporary methods is presented in Table 1. It is noted that the

123

performance time figures of the benchmark algorithms were obtained from the relevant articles and scaled to the same frame size (360 9 288) and frame rate (25 fps). The results achieved by the proposed approach are significantly better when compared with those reported by existing state-of-the-art approaches. For instance, when the proposed framework’s execution time is compared with that of [10] which reports minimal detection, tracking and recognition time, it can be seen that the execution time for tracking and recognition are almost identical but the localization performed by the proposed framework is significantly faster. This is due to the fact that the proposed approach has the ability to learn operating conditions, due to which overall detection operation becomes faster with time. It takes about 1 s for the system to learn and adapt operating conditions after initializing. Also, in [10], the reported resolution of test videos is 352 9 288, which is lesser than our test videos. 4.1 Detection/recognition results To evaluate the performance accuracy of our framework, we execute it for [10 good to worst quality videos with [200 vehicles passing through them. A number of these videos contained pedestrians and other unwanted objects as well.

J Real-Time Image Proc (2013) 8:285–295 Table 1 Comparison of the proposed framework with various methods

293

Method

Localization

Tracking

Recognition

Donoser et al. [10]

0.070 s

0.005 s

0.006 s

Zhange et al. [24]

0.05 s

–

–

Rastegar et al. [19]

2.3 s

–

0.4 s

Zheng et al. [25]

5.03 s

–

–

Arth et al. [1]

–

0.039 s

0.0198 s –

Line-wise filters DFTs [19]

2.1 s

–

Edge image improvement [19]

1.97 s

–

–

Row-wise and column-wise [19]

2.44 s

–

–

The proposed method

0.025 s

0.006 s

0.0045 s

1

True Positives (sensitivity)

0.9

0.3

0.8

0 -0.1

0.1

0.2

0.4

0.7

0.5

0.6 0.5

0.6

0.4 0.3

0.7 0.2

0.8

0.1 0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Positives (1-specificity)

Fig. 9 ROC curve for license plate detection. The graph shows that false positives increase with the increase in correlation threshold

Figure 9 shows the ROC curve for different operating points (correlation threshold). The number of false positives increases with the decrease in the value of the normalized threshold. We have selected 0.4 as the operating

point threshold in our initial learning stage. With this value, approximately 80% true positives and 3% false positives occur that are rejected afterward by the successive detection steps. The overall average detection/localization accuracy is 94%. For good-quality videos, the license plate recognition results are 100% which is promising. For extremely lowquality videos with high blur factor in which the license plates are even hard to be read by a human observer, the recognition procedure has shown 91% efficiency. The results for low-quality videos with high blur factor are shown in Fig. 10. Figure 10 shows false candidate detections due to low-quality video. This is why majority voting is used to eliminate such false candidate regions. In recognition of characters, a common problem is the similarity of appearance of letters/characters. Some letters look similar to numbers, e.g., O and 0, B and 8, I and 1, S and 5, etc. If letters and numbers have designated positions on the license plates, which is usually the case for license plate formats worldwide, for example, there are seven characters on a license plate and the first two and last two characters contain only letters and the remainder contains numbers. The problem stated is illustrated in Fig. 11.

Fig. 10 False detection due to low-quality video

Fig. 11 Recognition results of detected plates from consecutive frames. a Detected plates from consecutive video frames. b Respective recognized characters

123

294

The proposed system has been thoroughly trained, so that the chance of misclassification of a character is very low.

5 Conclusion An intelligent license plate localization, tracking and recognition method has been implemented for real-time video streams captured from road surveillance cameras. We have addressed these issues for a range of low- to high-resolution video streams with variant conditions and motion blur. The realized system intelligently performs all three operations in 35 ms per frame with exceptional accuracy. It means that the method can easily work at 28 fps. This is due to its capability to learn and adjust itself with different camera positions and distances. The framework uses a novel and comparatively faster technique for license plate detection that is a key operation for the whole system. The system works well for multiple license plates in a single frame. The method can be improved further by incorporating super-resolution techniques on the obtained multiple low-resolution instances of the license plate for enhanced recognition

References 1. Arth, C., Limberger, F., Bishof, H.: Real-time license plate recognition on an embedded DSP-platform. Institute for Computer Graphics and Vision, Graz University, Austria (2006) 2. Baker, S., Matthews, I.: Lucas–Kanade 20 years on: a unifying framework. Int. J. Comp. Vis. 56, 3221–3255 (2004) 3. Chang, S.L., et al.: Automatic license plate recognition. IEEE Transact. Intell. Transp. Sys. 5(1), 42–53 (2004) 4. Chateau, T., Lapreste, J.: Robust real time tracking of a vehicle by image processing. IEEE Intell. Vehicles Symp. 315–318 (2004). doi:10.1109/IVS.2004.1336401 5. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. CVPR IEEE Comp. Soc. Conf. 1, 886–893 (2005) 6. Deb, K., Chae, H.U., Jo, K.H.: Vehicle licence plate detection method based on sliding concentric windows and histogram. Int. J. Comp. 4(8), 771–777 (2009) 7. Dellaert, F., Thorpe, C.: Robust car tracking using Kalman filtering and Bayesian templates. Conference on Intelligent Transportation Systems (1997) 8. Dlagnekov, L., Belongie, S.: Recognizing cars. Technical Report CS2005–0833. UCSD University of California, San Diego (2005). Accessed 19 June 2010 9. Dlagnekov, L.: license plate detection using AdaBoost, computer science & engineering dpt., La Jolla, San Diego. http://www. cse.ucsd.edu/classes/fa04/cse252c/projects/louka.pdf 10. Donoser, M., Arth, C., Bischof, H.: Detecting, tracking and recognizing license plates ACCV 2007, Part II. LNCS 4844, 447–456 (2007) 11. Kamat, V., Ganesan, S.: An efficient implementation of the hough transform for detecting vehicle license plates using DSPs. In: Proceedings of the IEEE International Conference RTAS. 58–59 (1995)

123

J Real-Time Image Proc (2013) 8:285–295 12. Kanhere, N.K., Birchfield, S.T., Sarasua, W.A.: Vehicle segmentation and tracking in the presence of occlusions. Transp. Res. Rec. 1944, 89–97 (2006) 13. Kim, S., Oh, S. Y., Kang, J., Ryu, Y. K., Park, C., Park, K.: Front and rear vehicle detection and tracking in the day and night-times using vision and sonar sensor fusion. IEEE/RS J. Int. Conf. Intell. Robots Sys. 2173–2178 (2005) 14. Koller, D., Weber, J., Malik, J.: Robust multiple car tracking with occlusion reasoning. In Proc. Third Europeon Conference on Computer Vision, Stockholm, Sweden. LNCS 800, pp. 189–196. Springer, 2–6 May 1994 15. Li, X., Yan, X., Murphey, Y., Karlsen, R., Gerhart, G.: A realtime vehicle detection and tracking system in outdoor traffic scenes. ICPR 2004, Proceedings of the 17th International Conference 2. (2004) 761-764 16. Lucas, B. D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence. (1981) 17. Marmoiton, F., Collange, F., Alizon, J., Derutin, J.: 3d localization of a car observed through a monocular video camera. In: Proceedings of the IEEE International Conference on Intelligent Vehicles. (1998) 18. Rahman, C., Badawy, W., Radmanesh, A.: A real time vehicle’s license plate recognition system. In: AVSS, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, IEEE Computer Society Press, Los Alamitos 163–166. (2003) 19. Rastegar, S., Ghaderi, R., Ardeshir, J., Asadi, N.: An intelligent control system using an efficient license plate location and recognition approach. Int. J. Image Process. (IJIP) 3(5), 252–264 (2009) 20. Sergi, M., Shankwitz, C., Donath, M.: Lidar-based vehicle tracking for a virtual mirror. Intelligent Vehicle Symposium. Proc. IEEE. 333–338 (2003). doi:10.1109/IVS.2003.1212932 21. Shapiro, V., Gluhchev, G., Dimov, D.: Towards a multinational car license plate recognition system. Mach. Vis. Appl. 17(3), 173–183 (2006) 22. Tan, X., Li, J., and Liu, C. A video-based real-time vehicle detection method by classified background learning. World Transact. Eng. Technol. Educ. 6(1) (2007) 23. Yanamura, Y., Goto, M., Nishiyama, D., Soga, M., Nakatani, H., Saji, H.: Extraction and tracking of the license plate using Hough transform and voted block matching. IEEE IV2003 Intelligent Vehicles Symposium, Proceeding (Cat. No. 03TH8683), 243–246 (2003) 24. Zhang, H. et al.: Real-time license plate detection under various conditions. Springer-Verlag Berlin, Heidelberg, 192–199 (2006) 25. Zheng, D., Zhao, Y., Wang, J.: An efficient method of license plate location. Pattern Recogn. Lett. 26, 2431–2438 (2005)

Author Biographies Dr M. Saquib Sarfraz born in 1977, received an M.S. degree in Electrical and Computer Engineering from National University of Sciences and Technology, Pakistan, in 2003 and his PhD degree in Computer Vision at Technische Universita¨t Berlin, Germany in 2008. He is currently working as head of Computer Vision Research Group, COMSATS Institute of Information Technology Lahore, Pakistan. His research interests include pattern recognition, statistical machine learning, face recognition, and multimodal biometrics. Atif Shahzad is currently working as Research staff in Computer Vision Research Group, COMSATS Institute of Information

J Real-Time Image Proc (2013) 8:285–295 Technology Lahore, Pakistan. He received his MSc degree in Electrical & Electronics Engineering; United Kingdom. His research interests include Object Identification and Classification in Videos. Muhammad Adnan Elahi is currently working as Research staff in Computer Vision Research Group, COMSATS Institute of Information Technology Lahore, Pakistan. He received his MSc degree in Embedded Digital Systems, United Kingdom. His research interests include Microprocessor & FPGA based design, GPS based Tracking/ Navigation systems. Muhammad Fraz is currently working as Research staff in Computer Vision Research Group, COMSATS Institute of Information Technology Lahore, Pakistan. He received his MSc degree in Embedded Digital Systems, United Kingdom His research interests include Video Processing on DSPs.

295 She received her MSc degree in Computer Sciences from the International Islamic University, Islamabad, Pakistan, in October 2000, her degrees in MSc Multimedia and Internet Computing (with distinction) in October 2004 and her PhD in Computer Science in 2008 at Loughborough University. Her research interests include Computer and Machine Vision, Pattern Matching and Recognition, Image Analysis, machine learning and Automated Surveillance. Eran. A. Edirisinghe is a Reader in the Department of Computer Science at Loughborough University UK. He is the research coordinator and a member of the Visual, Imaging & Autonomous system Research Division (VIAS) within the Department. His research interests include image processing, computer vision, pattern recognition, video coding, signal processing, stereo image coding, image and video watermarking.

Iffat Zafar is currently working as a Research Associate in the Department of Computer Science at Loughborough University UK.

123