PhD Thesis Final Presentation

Robust Image Feature Description, Matching and Applications Supervised by Dr. Rajat Kumar Singh

& Dr. Satish Kumar Singh

12/11/2016

Presented by – Shiv Ram Dubey Research Scholar (RS136), IIIT Allahabad

Outline

• Introduction & Motivation • Literature Survey & Problem Formulation

• • • • •

Interleaved Order Based Local Descriptor Local Image Descriptors Color Image Descriptors Illumination Compensation Boosting the Performance of Local Descriptors

• Conclusions and Future Directions • References • Publications 2

Introduction & Motivation 3

 Image matching  Automatically recognize whether two images contain the

similar content.  Comparing the image pixels as they are, will not work.

Comparing pixels of two regions (images are taken from Corel-database [1])

4

Challenges • Scale change, • Rotation, • Viewpoint variations, and • Illumination changes, etc.

Hence, the comparison using descriptor becomes necessary. 5

 Descriptors allow certain differences between the images.

Comparing using descriptor function (images are taken from Corel-database [1]) 6

 Image descriptor is being used in many applications [2], for

example:  Image Retrieval  Biomedical Image Analysis  Texture Classification  Image Correspondence  Face Analysis  Biometrics  Building Panorama  And many more…

7

Image descriptor

• Descriptions of the visual features • Described by appearance based characteristics such as color, shape, etc.

A descriptor must be

• Distinctive • Robust • Low Dimensional 8

 Issues 1. Where to compute the descriptors?  Over interest regions.  Interest region may be Grid [3], Key-Points [4 - 10] or Global [12-21] based.

Extracting regions using (a) Grid, (b) Key-points, and (c) Global approach [22]

2.

How to compute the descriptors?  Depends upon requirement.

3.

Similar ?

How to compare two descriptors?  Generally descriptors come with its similarity measure criteria.

9

Literature Survey & Problem Formulation

10

Method

Key-point Global Based Approach

Type

Scale Rotation Illumination Invariance Invariance Invariance

Viewpoint Invariance

Color/ Gray-scale

SIFT

[4]





Gradient

Y

Y

N

N

G

Edge-SIFT

[5]





Binary

Y

Y

N

N

G

[11]





Order+ Gradient

P

Y

N

Y

G

HRI

[6]





Order

N

Y

P

N

G

LIOP

[7]





Order

N

Y

Y

N

G

MRRID

[8]





Order

P

Y

Y

Y

G

EOD

[9]





Order

N

Y

Y

N

G

LBP

[12]





Difference

N

N

Y

N

G

CS-LBP

[13]





Difference

N

N

Y

N

G

LTP

[14]





Difference

N

N

Y

N

G

[6]





Difference

N

N

Y

N

G

LDP

[15]





Gradient+ Difference

N

N

Y

N

G

LTrP

[16]





Direction

N

N

Y

N

G

LEBP

[17]





Filter+Difference

N

N

Y

N

G

SEH

[20]





Structure

N

N

N

N

C

CDH

[23]





Gradient

P

Y

N

N

C

MROGH

CS-LTP

11

Method

Key-point Global Based Approach

Type

Scale Rotation Illumination Viewpoint Color/ Invariance Invariance Invariance Invariance Gray-scale

LMeP

[24]





Difference

N

N

Y

N

G

LTCoP

[25]





Difference

N

N

Y

N

G

SS-3D-LTP

[26]





Filter+Difference

N

N

P

N

G

SLBP

[27]





Filter+Difference

N

N

Y

N

G

SOBEL-LBP [28]





Filter+Difference

N

N

Y

N

G

DBC

[29]





Difference

N

N

Y

N

G

SSLBP

[30]





Difference

N

N

Y

N

G

GCH

[31]





Color Histogram

N

Y

N

N

C

CCV

[32]





Color Histogram

N

Y

N

N

C

BIC

[33]





Color Histogram

N

Y

N

N

C

cLBP

[34]





Difference

N

N

Y

N

C

mscLBP

[35]





Difference

Y

N

Y

P

C

mCENTRIST [36]





Difference

N

N

Y

N

C

12

 Local ordering based descriptor such as LIOP [7] –  Dimension increases exponentially with number of neighbors.

 In recent years, the local descriptors proposed [12, 14, 26] –  Did not explored the relationship among the local neighbors.  In some literature, it is explored at the cost of increased dimension [16, 25].

 The color descriptors proposed in the literature [18-20, 23, 34-56] –  The performance suffered due to lack of inter-channel relationship and robustness.

 Most of the existing illumination invariant methods [7, 12, 14] –  Fail in case of drastic illumination changes.

 The local descriptors with some pre-processing –  SOBEL-LBP [28] has used only two filters.  Performance may be further improved using multiple filters or other techniques.

13

Interleaved Intensity Order Based Local Descriptor (IOLD)

14

15

Comparison between the pattern dimension using LIOP [7] and proposed approach

16

Illumination – leuven Image Blur – bikes

JPEG Compression – ubc Scale and Rotation – boat Viewpoint change graf Viewpoint change wall

Complex Illumination – corridor

Complex Illumination – destop 17

Image is taken from [22]

All the matching experiments are conducted using a computer having Intel(R) Core(TM) i5 CPU [email protected] GHz processor, 4GB RAM, and 32-bit Windows 7 Ultimate operating system.

18

(a)

(d)

(b)

(e)

(c)

(f)

(g) Descriptors performance for kd=14, 24, 15, 25 and 16 when B=1 and C=1 over Oxford dataset 19 for sequence a) leuven, b) bikes, c) ubc, d) boat, e) graf, f) wall, and g) the matching time.

Comparison of IOLD with LIOP [7], SIFT [4] and HRI-CSLTP [6] over Complex illumination change dataset in terms of (a) recall-precision and (b) matching time.

20

Local Image Descriptors for Biomedical Image Retrieval

21

The computation of 𝐿𝐷𝐸𝑃𝑖,𝑗 pattern for center pixel 𝑃𝑖,𝑗 (intensity value 𝑖,𝑗) using the flow diagram with an example.

22

23

24

The transformation of an 𝑁-dimensional vector using 1-D Haar wavelet.

to another 𝑁-dimensional vector

at 𝑙𝑡h level 25

 Precision and recall for any query image is given as follows,

 Average retrieval precision (ARP) & Average retrieval rate (ARR)

 F-score or F-measure is given as follows,

 Average normalized modified retrieval rank (ANMRR) 26

 We compared the proposed

 Local Diagonal Extrema Pattern (LDEP)  Local Bit-plane Decoded Pattern (LBDP)

 Local Bit-plane Dissimilarity Pattern (LBDISP)  Local Wavelet Pattern (LWP)

descriptors with following descriptors –  Local Binary Pattern (LBP) [12] – IEEE PAMI 2002  Local Ternary Pattern (LTP) [14] – IEEE TIP 2010  Center Symmetric Local Binary Pattern (CSLBP) [13] - Pattern Recognition 2009  Center Symmetric Local Ternary Pattern (CSLTP) [6] - IEEE CVPR 2010  Local Derivative Pattern (LDP) [15] – IEEE TIP 2010  Local Tetra Pattern (LTrP) [16] – IEEE TIP 2012  Local Ternary Co-occurrence Pattern (LTCoP) [25] – Neurocomputing 2013  Local Mesh Pattern (LMeP) [24] – IEEE JBHI 2014  Spherical Symmetric 3-Dimensional Local Ternary Pattern (SS3DLTP) [26] –

Neurocomputing 2015 27

a) OASIS-MRI Database [39] - Open Access Series of Imaging Studies (OASIS) has released 421 cross-sectional images. Partitioned into four categories having 106, 89, 102 and 124 images on the basis of the ventricular shape inside the images. b) Emphysema-CT Database [40] - Emphysema-CT database comprises of the 3 classes namely, Normal Tissue (NT), Centrilobular Emphysema (CLE), and Paraseptal Emphysema (PSE) with 59, 50 and 59 images respectively.

c) NEMA-CT Database [41] - The National Electrical Manufacturers Association (NEMA) [41] created the digital imaging and communications in medicine (DICOM) standard for research purpose. This database consists of the 499 images from different body parts with 8 classes having 104, 46, 29, 71, 108, 39, 33 and 69 images respectively. d) TCIA-CT Database [42] - The cancer image archive (TCIA) provides the 604 Colo_prone 1.0B30f CT images of the DICOM series number 1.3.6.1.4.1.9328.50.4.2 of study instance UID 1.3.6.1.4.1.9328.50.4.1 for subject 1.3.6.1.4.1.9328.50.4.0001. According to the size and structure of Colo_prone, the database is having 8 categories with 75, 50, 58, 140, 70, 92, 78, and 41 images respectively. e) EXACT09-CT Database [43] - Extraction of Airways from CT 2009 (EXACT09) is a database of chest CT scans. Used 675 CT images of CASE23 of testing set and categorized into 19 categories having 36, 23, 30, 30, 50, 42, 20, 45, 50, 24, 28, 24, 35, 40, 50, 35, 30, 28 and 55 CT images respectively. 28

The performance comparison of LDEP, LBDP, LBDISP and LWP descriptors with LBP, LTP, CSLBP, CSLTP, LDP, LTrP, LTCoP, LMeP and SS3DLTP descriptors over OASIS-MRI database using D1 [24-26] distance measure.

29

The performance comparison Emphysema-CT, NEMA-CT, TCIA-CT and EXACT09-CT databases using D1 [2430 26] distance measure.

Local Color Image Descriptors for Natural and Texture Image Retrieval

31

 We proposed four color descriptors namely –  Local Color Occurrence Descriptor (LCOD)  Rotation and Scale Invariant Hybrid Descriptor (RSHD)  Multichannel Adder Based Local Binary Pattern (maLBP)  Multichannel Adder Based Local Binary Pattern (mdLBP)

 LCOD and RSHD – Based on the Color Quantization  maLBP and mdLBP – Based on the Multi-channel LBP

32

Red (R) channel

Green (G) channel

Blue (B) channel

250

15

56

50

115

156

134

210

13

120

78

220

200

187

120

34

226

70

176

90

154

67

9

4

100

215

165

Quantization 0 to 63 = 1 64 to 127 = 2 128 to 191 = 3 192 to 255 = 4 Look Up Table Rq

Gq

Bq

o/p

4

1

1

1

2

3

3

4

1

1

1

1

1

2

2

4

4

3

2

1

4

2

1

1

2

2

3

2

3

2

1

1

2

4

3

1

1

3

3

1

1

4

4

1

2

1

5

1

2

2

6

4

4

3

63

4

4

4

64

Color quantized shades

51

8

9

29

28

54

38

20

35

33

An illustration to compute the local colour occurrence binary pattern for D = 2, where D is the distance of local neighborhood. The number of shades is considered as 5 in this example.

34

Five structure element containing (a) only one, (b) two consecutive, (c) two non-consecutive, (d) three consecutive, and (e) four consecutive same quantized shade.

Six patterns derived from the five structure elements, representing (a) no structure, (b) type 1 structure, (c) type 2 structure, (d) type 3 structure, (e) type 4 structure, and (e) type 5 structure.

Three examples to illustrate the computation of q patterns over each quantized shade for a particular 35 pixel; In this example, the number of quantized color shade is set to 4.

36

 We compared the proposed  Local Color Occurrence Descriptor (LCOD)  Rotation and Scale Invariant Hybrid Image Descriptor (RSHD)  Multichannel Adder-based Local Binary Pattern (maLBP)

 Multichannel Decoder-based Local Binary Pattern (mdLBP)

descriptors with following descriptors –  Local Binary Pattern (LBP) [12] – IEEE PAMI 2002  Color Local Binary Pattern (cLBP) [34]- IEEE ICIP 2010  Multi-Scale Color Local Binary Pattern (mscLBP) [35] – IEEE ICPR 2010  mCENTRIST [36]- IEEE TIP 2014  Structure Element Histogram (SEH) [20] – JVCIR Elsevier 2013  Color Difference Histogram (CDH) [23] – Pattern Recognition Elsevier 2013 37

a) Corel-1k database [44] - Containing 1000 images from 10 categories having 100 images each category. b) Corel-10k database [1] - A total of 10800 images from 80 categories having different details ranging from natural scenarios and outdoor sports to animals are present. c) MIT-VisTex database [45] - Consisting of 640 images from 40 categories with 16 images per category. d) STex-512S database [46] - Consists of the 7616 numbers of color images from 26 categories. e) Corel-1k-Rotate database - Synthesized by rotating first 25 images of each category of Corel-1k with angle 0, 90, 180, and 270 degrees. f) Corel-1k-Scale database - Synthesized by scaling first 20 images of each category of Corel-1k at the scales of 0.5, 0.75, 1, 1.25, and 1.5. g) Corel-1k-Illumination database - Synthesized by adding -60, -30, 0, 30, and 60 in all channels of the first 20 images of each category of Corel-1k database.. 38

The result comparison of LCOD, RSHD, maLBP and mdLBP descriptors with LBP, cLBP, mscLBP, mCENTRIST, 39 SEH and CDH descriptors over Corel-1k and Corel-10k databases in terms of the ARP and ANMRR.

The result comparison of LCOD, RSHD, maLBP and mdLBP descriptors with LBP, cLBP, mscLBP, mCENTRIST, 40 SEH and CDH descriptors over MIT-VisTex and STex-512S databases in terms of the ARP and ANMRR.

Top 10 retrieved images using each descriptor for a query image from Corel-1k database. Note that 10 rows corresponds to the different descriptor such as LBP (1st row), cLBP (2nd row), mscLBP (3rd row), mCENTRIST (4th row), SEH (5th row), CDH (6th row), LCOD (7th row), RSHD (8th row), maLBP (9th row) and mdLBP (10th row).

41

Top 10 retrieved images using each descriptor for a query image from MIT-VisTex database. Note that 10 rows corresponds to the different descriptor such as LBP (1st row), cLBP (2nd row), mscLBP (3rd row), mCENTRIST (4th row), SEH (5th row), CDH (6th row), LCOD (7th row), RSHD (8th row), maLBP (9th row) and mdLBP (10th row).

42

The results comparison of different descriptors over Corel-1k-Rotate, Corel- 1k-Scale and Corel-1kIllumination database.

43

Illumination Compensation from Color Channels

44

RGB image

HSI image

RIGIBI image

R channel

I channel

RI channel

Mapping

RICGICBIC image

RIC channel

G channel

B channel

I channel

I channel

GI channel

Mapping

GIC channel

BI channel

Mapping

BIC channel

45

Global Color Histogram (GCH) [31],

GCH

GCHIC

Color Coherence Vector (CCV) [32],

CCV

CCVIC

Border-Interior Classification (BIC) [33],

BIC

BICIC

Color Difference Histogram (CDH) [23],

CDH

CDHIC

Structure Element Histogram (SEH) [20] and

SEH

SEHIC

Square Symmetric Local Binary Pattern (SSLBP) [30]

SSLBP

SSLBPIC

46

Phos natural illumination database [47]

15 categories

15 images in each category (9 images of uniform illumination and 6 images of non-uniform illumination).

47

Results in terms of ARP and ARR curves for different features with and without illumination compensation using dSEH [20] and dCDH [23] similarity measures over Phos illumination benchmark dataset.

48

Retrieval results using each descriptor from Phos dataset 49

0.6

0.5

0.5

ARP

ARP

0.6

0.4

0.3

0.3 LCOD LCODic

0.2 0.1

0.4

0.15

0.2

0.25

0.3

0.35

RSHD RSHDic

0.2 0.1

0.4

0.15

0.2

ARR 0.35

maLBP maLBPic

0.35

0.4

mdLBP mdLBPic

0.3

ARP

ARP

0.3

0.35

0.3

0.25

0.2

0.15 0.1

0.25

ARR

0.25

0.2

0.12

0.14

ARR

0.16

0.18

0.15

0.1

0.12

0.14

0.16

0.18

0.2

ARR

Performance evaluation of the proposed illumination compensation approach using the LCOD, RSHD, maLBP and mdLBP descriptors over Phos illumination database. 50

Boosting the Performance of Local Descriptors with BoF and SVD

51

The framework of the proposed Content Based Image Retrieval (CBIR) system using Bag-of-Filters (BoF) and Local Binary Pattern (LBP).

The five types of filters used in this paper as the Bag-of-Filters, (a) Average filter, i.e. 𝐹1, (b) Horizontal-vertical difference filter, i.e. 𝐹2, (c) Diagonal filters, i.e. 𝐹3, (d) Sobel edge in vertical direction, i.e. 𝐹4, and (e) Sobel edge in horizontal direction, i.e. 𝐹5.

(a) An example image from Corel-1k database [44], (b-f) the image obtained after applying the 5 filters with mask 𝐹|𝑖=1,2,3,4,5 respectively over the example image of (a). 52

The performance comparison BoF-LBP descriptor with LBP, SLBP, SOBEL-LBP, LTP, LDP, LTrP, and SS-3D-LTP descriptors over Corel-1k, Corel-10k, MITVis-Tex and Stex-512S databases using ARP (%).

53

Proposed framework for NIR face retrieval using SVD and local descriptors.

54

Illustration of the sub-band formation (i.e. S, U, V and D sub-bands) from the SVD factorization [48] of any input PL using an example of size 4×4. • 𝐴 and 𝐶 are the 2×2 matrices containing the orthonormal column vectors, and • 𝐵 is a 2×2 diagonal matrix having singular values at the main diagonals 55

 Extracted the following descriptors – 

Local Binary Pattern (LBP) [12],



Semi-structure Local Binary Pattern (SLBP) [27],



Directional Binary Code (DBC) [29], and



Local Gabor Binary Pattern (LGBP) [51].

 The LBP descriptor computed over S, U, V, and D sub-bands are denoted as the SVD-S-LBP, SVD-U-LBP, SVD-V-LBP and SVD-D-LBP respectively.

56

Two widely adopted and benchmark NIR face databases, namely PolyU-NIR [49] and CASIA-NIR [50] is used for the face retrieval experiments.

 We have considered the set-1 of the PolyU-NIR face database which consists of the total

7277 images from 55 subjects.

 CASIA-NIR face database is comprised of the total 3940 images from the 197 subjects

having 20 faces each.

57

The performance comparison of the local descriptors (i.e. LBP, SLBP, DBC, and LGBP in the 1st, 2nd, and 3rd column respectively) over different sub-bands of the SVD in terms of the ARR (%) over PolyUNIR (in the 1st row) and CASIA-NIR (in the 2nd row) face database.

58

Retrieval results from CASIA-NIR face database using LBP (1st row), SVD-S-LBP (2nd row), SLBP (3rd row), SVD-S-SLBP (4th row), DBC (5th row), SVD-S-DBC (6th row), LGBP (7th row) and SVD-S-LGBP (8th row) descriptors. The first image in each row is the query face and rest images are retrieved faces. The faces in rectangles are the false positives. 59

Conclusions & Future Directions

60

 The interleaved intensity order reduces the dimension while consideration of more

number of local neighbors improves the discriminative ability.  The relationship of center pixel with its neighbors along with the relationship

among the local neighbors are exploited by LDEP, LBDP, LBDISP, and LWP.  The diagonal neighbors, bit-planes and wavelet decomposition are making the descriptors

more discriminative while the dimensions are very low as compared to the state-of-the-art descriptors.  The color quantization based descriptors LCOD and RSHD are rotation and scale

invariant.  These descriptor are more suitable for the color natural databases.  The dimension of these descriptors is also very reasonable.

 The multichannel decoded local binary patterns maLBP and mdLBP have shown

the great improvement in the retrieval performance over color texture databases.  The dimension of these descriptors is relatively high but still low as compared to some

recently introduced descriptors.

61

 The illumination compensation mechanism are useful for both uniform as well as

non-uniform illumination.  It can be used as a pre-processing in many applications.  It has also been noticed that the results of illumination invariant descriptors improved with

illumination compensation.  The performance of local descriptor such as LBP has been improved significantly

when applied with the BoF in CBIR over the gray scale natural and textural databases.  The performance of local descriptors are improved when extracted over S sub-band

of SVD. The local descriptors with SVD have been experimented over the NIR face databases under

image retrieval framework and observed satisfactory performance.  The use of SVD with local descriptors does not affect the size of the local descriptors.

62

 The region based descriptors mostly designed for the gray scale images; it

can be explored for the color images also.  The performance of local descriptors can be further improved by utilizing

the local neighborhood information more effectively.  The color descriptors are still having the dimensionality problem which can

be tackled further.  The sensitivity to the illumination problem of color descriptors can be

explored to introduce more illumination robust descriptors.  The performance of descriptors proposed for the one type of images can be

explored with the other kind of images.  Nowadays, deep learning is being used very actively to develop the

features at the intermediate layers. This may also one of the future works to work with descriptors and deep learning. 63

References

64

[1] [2] [3] [4] [5]

[6] [7] [8] [9] [10]

[11] [12] [13] [14] [15]

[16] [17] [18] [19] [20] [21] [22] [23] [24]

https://sites.google.com/site/dctresearch/Home/content-based-image-retrieval. M. Pietikainen, G. Zhao, A. Hadid, and T. ShonenComputer, “Vision Using Local Binary Patterns”, Computational Imaging and Vision, Vol. 40, 2011. V. Takala, T. Ahonen, M. Pietikainen, Block-Based Methods for Image Retrieval Using Local Binary Patterns, Proceedings of the 14th Scandinavian conference on Image Analysis (2005) 882-891. D. G. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision 60 (2) (2004) 91–110. S. Zhang, Q. Tian, K. Lu, Q. Huang, W. Gao, Edge-SIFT: Discriminative Binary Descriptor for Scalable Partial-Duplicate Mobile Search, IEEE Transactions on Image Processing 22 (7) (2013) 2889-2902. R. Gupta, H. Patil, A. Mittal, Robust order-based methods for feature description, Proceedings of the 23rd IEEE International Conference on Computer Vision and Pattern Recognition (2010) 334 –341. Z. Wang, B. Fan, F. Wu, Local Intensity Order Pattern for feature description, Proceedings of the 13th IEEE International Conference on Computer Vision (2011) 603-610. B. Fan, F. Wu, Z. Hu, Rotationally Invariant Descriptors Using Intensity Order Pooling, IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (10) (2012) 2031 –2045. B. Kim, H. Yoo, K. Sohn, Exact order based feature descriptor for illumination robust image matching Pattern Recognition 46 (12) (2013) 3268-3278. S. Zambanini and M. Kampel, A Local Image Descriptor Robust to Illumination Changes, Proceedings of the 18th Scandinavian conference on Image Analysis (2013) 11-21. B. Fan, F. Wu, Z. Hu, Aggregating Gradient Distributions into Intensity Orders: A Novel Local Image Descriptor, Proceedings of the 24th IEEE International Conference on Computer Vision and Pattern Recognition (2011) 2377–2384. T. Ojala, M. Pietikanen, T. Maenpa, Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (7) (2002) 971–987. M. Heikkila, M. Pietikainen, C. Schmid, Description of interest regions with local binary patterns, Pattern Recognition 42 (3) (2009) 425–436. X. Tan and Bill Triggs, Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions, IEEE Transactions on Image Processing 19 (6) (2010) 1635-1650. B. Zhang, Y. Gao, S. Zhao, J. Liu, Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor, IEEE Transactions on Image Processing 19 (2) (2010) 533-544. S. Murala, R.P. Maheshwari, R. Balasubramanian, Local tetra patterns: a new feature descriptor for content-based image retrieval, IEEE Transactions on Image Processing 21 (5) (2012) 2874–2886. J. Sun, G. Fan, X. Wu, New local edge binary patterns for image retrieval, Proceedings of the 20th IEEE International Conference on Image Processing (2013) 4014-4018. G.H. Liu, L. Zhang, Y.K. Hou, Z.Y. Li, J.Y. Yang, Image retrieval based on multi-texton histogram, Pattern Recognition 43 (7) (2010) 2380–2389. G.H. Liu, Z.Y. Li, L. Zhang, Y. Xu, Image retrieval based on micro-structure descriptor, Pattern Recognition 44 (9) (2011) 2123–2133. W. Xingyuan, W. Zongyu, A novel method for image retrieval based on structure elements descriptor, Journal of Visual Communication and Image Representation 24 (1) (2013) 63–74. G. Zhao, T. Ahonen, J. Matas, M. Pietikäinen, Rotation-Invariant Image and Video Description With Local Binary Pattern Features, IEEE Transactions on Image Processing 21 (4) (2012) 1465-1477. O. Shahar and G. Levi, Image and video descriptors, Weizmann Institute of Science, Israel, 2010. G.H. Liu and J.Y. Yang, “Content-based image retrieval using color difference histogram,” Pattern Recognition, vol. 46, no. 1, pp. 188-198, 2013. S. Murala and Q. M. J. Wu, “Local Mesh Patterns Versus Local Binary Patterns: Biomedical Image Indexing and Retrieval,” IEEE Journal of Biomedical and Health Informatics, vol.18, no.3, pp. 929-938, 2014. 65

[25]

[26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40]

[41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51]

S. Murala and Q. M. J. Wu, “Local ternary co-occurrence patterns: A new feature descriptor for MRI and CT image retrieval,” Neurocomputing, vol. 119, pp. 399-412, 2013. S. Murala and Q. M. Jonathan Wu, “Spherical symmetric 3D local ternary patterns for natural, texture and biomedical image indexing and retrieval”, Neurocomputing, vol. 149, pp. 1502-1514, 2015. K. Jeong, J. Choi, and G. Jang, “Semi-Local Structure Patterns for Robust Face Detection,” IEEE Signal Processing Letters, vol. 22, no. 9, pp. 1400-1403, 2015. S. Zhao, Y. Gao and B. Zhang, “SOBEL-LBP,” In Proceedings of the 15th IEEE International Conference on Image Processing, pp. 2144-2147, 2008. B. Zhang, L. Zhang, D. Zhang, and L. Shen, “Directional binary code with application to PolyU near-infrared face database”, Pattern Recognition Letters, vol. 31, no. 14, pp. 2337-2344, 2010. Shi Z, Liu X, Li Q, He Q, Shi Z (2012) Extracting discriminative features for CBIR. Multimedia Tools and Applications 61(2):263-279. Gonzalez RC, Woods RE (2007) Digital Image Processing (3rd Edition). Prentice Hall. Pass G, Zabih R, Miller J (1997) Comparing images using color coherence vectors. In: 4th ACM international conference on Multimedia, pp 65-73. Stehling RO, Nascimento MA, Falcão AX (2002) A compact and efficient image retrieval approach based on border/interior pixel classification. In: 11th international conference on Information and knowledge management, pp 102-109. J.Y. Choi, K.N. Plataniotis and Y.M. Ro, “Using colour local binary pattern features for face recognition,” In 17th IEEE International Conference on Image Processing (ICIP), pp. 4541-4544, 2010. C. Zhu, C.E. Bichot and L. Chen, “Multi-scale Color Local Binary Patterns for Visual Object Classes Recognition” In Proceedings of the IEEE International Conference on Pattern Recognition, pp. 3065-3068, 2010. Y. Xiao, J. Wu and J. Yuan, “mCENTRIST: A Multi-Channel Feature Generation Mechanism for Scene Categorization,” IEEE Transactions on Image Processing, Vol. 23, No. 2, pp. 823-836, 2014. http://www.robots.ox.ac.uk/~vgg/research/affine/. http://vision.ia.ac.cn/Students/wzh/datasets/illumination/Illumination_Datasets.zip. D.S. Marcus, T.H. Wang, J. Parker, J.G. Csernansky, J.C. Morris and R.L. Buckner, “Open access series of imaging studies (OASIS): Crosssectional MRI data in young, middle aged, nondemented, and demented older adults”, J. Cogn. Neurosci., vol. 19, no. 9, pp. 1498-1507, 2007. L. Sørensen, S. B. Shaker, and M. de Bruijne, “Quantitative Analysis of Pulmonary Emphysema using Local Binary Patterns,” IEEE Transactions on Medical Imaging, vol. 29, no. 2, pp. 559-569, 2010. NEMA–CT image database, Available from [Online]: 〈ftp://medical.nema.org/ medical/Dicom/Multiframe/〉. K. Clark, B. Vendt, K. Smith, J. Freymann, J. Kirby, P. Koppel, S. Moore, S. Phillips, D. Maffitt, M. Pringle, L. Tarbox, and F. Prior, “The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository,” Journal of Digital Imaging, vol. 26, no. 6, pp. 1045-1057, 2013. P. Lo, B. Van Ginneken, J. M. Reinhardt, T. Yavarna, P. A. De Jong, B. Irving, ... and M. De Bruijne, “Extraction of airways from CT (EXACT'09),” IEEE Transactions on Medical Imaging, vol. 31, no. 11, pp. 2093-2107, 2012. Corel Photo Collection Color Image Database, online available on http://wang.ist.psu.edu/docs/realted/. MIT Vision and Modeling Group, Cambridge, „Vision texture‟, http://vismod.media.mit.edu/pub/. Salzburg Texture Image Database, http://www.wavelab.at/sources/STex/. http://utopia.duth.gr/~dchrisos/pubs/database2.html. S.K. Singh and S. Kumar, “Singular value decomposition based sub-band decomposition and multi-resolution (SVD-SBD-MRR) representation of digital colour images”, Pertanika Journal of Science and Technology, vol. 19, no. 2, pp. 229-235, 2011. “PolyU-NIR Face Database, http://www.comp.polyu.edu.hk/~biometrics/NIRFace/polyudb_face.htm”. “CASIA-NIR Face Database, http://www.cbsr.ia.ac.cn/english/NIR_face%20Databases.asp”. W. Zhang, S. Shan, W. Gao, X. Chen, and H. Zhang, “Local gabor binary pattern histogram sequence (lgbphs): A novel non-statistical model for face 66 representation and recognition,” In Proceedings of the 10th IEEE International Conference on Computer Vision, 2005, pp. 786-791.

Publications

67

•Journals •S.R. Dubey, S.K. Singh, R.K. Singh, "Multichannel Decoded Local Binary Pattern for Content Based Image Retrieval," IEEE Transactions on Image Processing, 25(5):4018-4032, 2016. •S.R. Dubey, S.K. Singh, R.K. Singh, "Local Wavelet Pattern: A New Feature Descriptor for Image Retrieval in Medical CT Databases," IEEE Transactions on Image Processing, 24(12):5892-5903, 2015. •S.R. Dubey, S.K. Singh, R.K. Singh, "Rotation and Illumination Invariant Interleaved Intensity Order Based Local Descriptor," IEEE Transactions on Image Processing, 23(12):5323-5333, 2014. •S.R. Dubey, S.K. Singh, R.K. Singh, "Local Bit-plane Decoded Pattern: A Novel Feature Descriptor for Biomedical Image Retrieval," IEEE Journal of Biomedical and Health Informatics, 20(4):1139-1147, 2016. •S.R. Dubey, S.K. Singh, R.K. Singh, "Local Diagonal Extrema Pattern: A New and Efficient Feature Descriptor for CT Image Retrieval," IEEE Signal Processing Letters, 22(9):1215-1219, 2015. •S.R. Dubey, S.K. Singh, R.K. Singh, "Local neighbourhood-based robust colour occurrence descriptor for colour image retrieval," IET Image Processing, 9(7): 578-586, 2015. •S.R. Dubey, S.K. Singh, R.K. Singh, "Novel local bit-plane dissimilarity pattern for computed tomography image retrieval," IET Electronics Letters, 52(15):1290-1292, 2016. •S.R. Dubey, S.K. Singh, R.K. Singh, "Rotation and scale invariant hybrid image descriptor and retrieval," Computers & Electrical Engineering, Elsevier, 46: 288-302, 2015. •S.R. Dubey, S.K. Singh, R.K. Singh, "A multi-channel based illumination compensation mechanism for brightness invariant image retrieval," Multimedia Tools & Applications, Springer, 74(24): 11223-11253, 2015. •S.R. Dubey, S.K. Singh, R.K. Singh "Boosting Performance of Local Descriptors with SVD Sub-band for Near-Infrared Face Retrieval," IET Image Processing. (Submitted in Revised Form)

•Conferences •S.R. Dubey, S.K. Singh, R.K. Singh, "Boosting Local Binary Pattern with Bag-of-Filters for Content Based Image Retrieval," IEEE UP Section Conference on Electrical, Computer and Electronics (UPCON), 2015. 68 (Best paper award)

Thank You

69