Multichannel Decoded Local Binary Patterns for Content Based Image Retrieval IEEE Transactions on Image Processing, 2016 Shiv Ram Dubey, Satish Kumar Singh and Rajat Kumar Singh Indian Institute of Information Technology, Allahabad

Method

TABLE I Truth Table of Adder and Decoder map with 3 input channels 𝑳𝑩𝑷𝒏𝟏 𝒙, 𝒚 𝑳𝑩𝑷𝒏𝟐 𝒙, 𝒚 𝑳𝑩𝑷𝒏𝟑 𝒙, 𝒚 𝒎𝒂𝑴𝒏 (𝒙, 𝒚) 𝒎𝒅𝑴𝒏 (𝒙, 𝒚)

The framework of feature description of color image using adder and decoder based multichannel LBP is shown in Fig. 1. Color Image Red (R) Channel

Green (G) Channel

Blue (B) Channel

LBP1

LBP2

LBP3

0 0 0 0 1 1 1 1

Decoder

Adder maLBP1

° ° °

maLBP4

mdLBP1

° ° °

mdLBP8

maLBP1 Histogram

° ° °

maLBP4 Histogram

mdLBP1 Histogram

° ° °

mdLBP8 Histogram

maLBP Feature Vector

mdLBP Feature Vector

0 0

0 1 1 2 1 2 2 3

1

a

0

1

0 0

0

0 0

a 0

1 0

1

1 0

1

0

LBP 2

0 0

a

1 0

LBP 3

0 1

a

0

0

0

0

0 1

MIT-VisTex database 100 LBP cLBP mscLBP mCENTRIST maLBP mdLBP

95 90 85 80 75 65 1

95 90 LBP cLBP mscLBP mCENTRIST maLBP mdLBP

85 80 75

2

3

4

5

6

7

8

Number of Retrieved Images

9

10

70 1

2

3

4

5

6

7

8

9

10

Number of Retrieved Images

Fig.4. The performance comparison of proposed maLBP and mdLBP descriptor with existing approaches such as LBP, cLBP, mscLBP, and mCENTRIST descriptors over Corel-1k and MIT-VisTex databases.

0

0

0 0

1 0

0

𝜃

a

0 1

𝑟

0 0

a

0

0

0

0

0 0

a 0

0 1

0

0

0

0

0 0

pixel

0

0

0

f (c) Weighting function

72

161

20

2

maLBP 1

maLBP 2

maLBP 3

maLBP 4

72

128

32

0

0 0

mdLBPn4

mdLBP 1

mdLBP 2

mdLBP 3

mdLBP 4

0

a

0

4

0

a

0

0

0 0

0

0

a

0 0

0

1

16

4

2

1

0

0

1

0

mdLBPn5

mdLBPn6

mdLBPn7

mdLBPn8

(f) Eight output decoder LBPs

𝐼𝑡𝑁−1 (𝑥, 𝑦)

0

0 0

a

1

0

0 0

mdLBPn3

0

𝐼𝑡𝑁 (𝑥, 𝑦)

0 0

mdLBPn2

6

DM

1 2

0

a

0

0

0

𝐼𝑡𝑛+1 (𝑥, 𝑦)

0

2

AM

8

(e) Multichannel adder based local binary pattern for each output channels 0

mdLBPn1 0

𝐼𝑡1 (𝑥, 𝑦)

1

7

128

a

16

1

maLBPn4

0 1

0

4

0

maLBPn3

0 0

3

a

5

32

0

a

0

0 1

maLBPn2

1

64 1

0 0

a

1

0

a

2

2

(b) Adder map and Decoder map

(d) Four output adder LBPs

a

Corel-1k database 100

70

0 1

0

(a) Three input LBPs for three channels

maLBPn1

of

0 1 2 3 4 5 6 7

0 1

1

1

1

𝐼𝑡2 (𝑥, 𝑦)

Fig.2. The local neighbors 𝐼𝑡𝑛 (𝑥, 𝑦) 𝐼𝑡 𝑥, 𝑦 in 𝑡 𝑡ℎ channel for 𝑛 ∈ [1, 𝑁] and 𝑡 ∈ [1,3].

1

LBP 1

1

ℛ 𝐼𝑡 𝑥, 𝑦

a 0

0

𝐼𝑡𝑛 (𝑥, 𝑦)

0 0

1

𝐼𝑡3 (𝑥, 𝑦)

2𝜋 𝑁

0 1 0 1 0 1 0 1

Image retrieval experiments are performed to test the performance of proposed descriptors in terms of average retrieval precision (ARP). Precision is the percentage of correct number of retrieved images out of total number of retrieved images. Results are compared with basic LBP [1] and other color based local descriptors such as cLBP [2], mscLBP [3], and mCENTRIST [4]. Here, results are presented over natural Corel-1k [5] and textural MIT-VisTex [6] databases in Fig.4.

The computation of 𝑚𝑎𝐿𝐵𝑃𝑡 1 for ∀𝑡1 ∈ [1,4] and 𝑚𝑑𝐿𝐵𝑃𝑡 2 for ∀𝑡2 ∈ [1,8] from input 𝐿𝐵𝑃𝑡𝑛 𝑥, 𝑦 using an example of three LBP patterns is illustrated in Fig. 3 for 𝑁 = 8.

Fig.1. The flowchart of computation of multichannel adder based local binary pattern feature vector (i.e. maLBP) and multichannel decoder based local binary pattern feature vector (i.e. mdLBP) of an image from its Red (R), Green (G) and Blue (B) channels.

𝐼𝑡𝑛 −1 (𝑥, 𝑦)

0 0 1 1 0 0 1 1

Performance Evaluation

ARP (%)

 Local binary pattern (LBP) [1] is widely adopted for simplicity and efficient image feature description.  To describe the color images, it is required to combine the LBPs from each channel of the image.  We introduced adder and decoder based two schemas for the combination of the LBPs from more than one channel.  The introduced descriptors significantly improve the retrieval performance and outperform the other multichannel based approaches.

The position of local neighbours of any pixel of the image is depicted in Fig.2. A local binary pattern 𝐿𝐵𝑃𝑡 (𝑥, 𝑦) for a pixel (𝑥, 𝑦) in 𝑡𝑡ℎ channel is generated as follows, 𝑛 𝑛 𝐿𝐵𝑃𝑡 𝑥, 𝑦 = 𝑁 ∀𝑡 ∈ 1,3 (1) 𝑛=1 𝐿𝐵𝑃𝑡 𝑥, 𝑦 × 𝑓 , where, 1, 𝐼𝑡𝑛 𝑥, 𝑦 ≥ 𝐼𝑡 𝑥, 𝑦 𝐿𝐵𝑃𝑡𝑛 𝑥, 𝑦 = (2) 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 and 𝑓 𝑛 is a weighting function defined by the following equation, 𝑓 𝑛 = (2)(𝑛−1) , ∀𝑛 ∈ [1, 𝑁] (3) 𝑛 The truth map of adder map (i.e. 𝑚𝑎𝑀 (𝑥, 𝑦)) and decoder map (i.e. 𝑚𝑑𝑀 𝑛 (𝑥, 𝑦)) are shown in Table 1.

ARP (%)

Introduction

mdLBP 5

mdLBP 6

mdLBP 7

mdLBP 8

(g) Multichannel decoder based local binary pattern for each output channels

Fig.3. An illustration of the computation of the adder/decoder based local binary pattern maps, adder/decoder based local binary pattern bits, and adder/decoder local binary pattern decimal values from three 8-bit input LBPs. Green and Red circles represent 0 and 1 respectively.

Fig.5. Top 10 retrieved images (columns) using LBP (1st row), cLBP (2nd row), mscLBP (3rd row), mCENTRIST (4th row), maLBP (5th row) and mdLBP (6th row) descriptors from Corel-1k database. Images in the 1st column are query images as well as the top most similar images.

References [1]T. Ojala, M. Pietikainen and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE TPAMI, 24(7): 971-987, 2002. [2]J.Y. Choi, K.N. Plataniotis and Y.M. Ro, “Using colour local binary pattern features for face recognition,” 17th IEEE ICIP, 2010. [3]C. Zhu, C.E. Bichot and L. Chen, “Multi-scale Color Local Binary Patterns for Visual Object Classes Recognition” IEEE ICPR, 2010. [4]Y. Xiao, J. Wu and J. Yuan, “mCENTRIST: A Multi-Channel Feature Generation Mechanism for Scene Categorization,” IEEE TIP, 23(2): 823-836, 2014. [5]Corel Photo Collection Color Image Database taken from: http://wang.ist.psu.edu/docs/realted/. [6]MIT Vision and Modeling Group, Cambridge, „Vision texture database‟ taken from: http://vismod.media.mit.edu/pub/.