A Procedure for Comparing Color 2-Dimensional Images Through their Extrusions to the 5-Dimensional Colorspace Antonio Aguilera

Santos Gerardo Lázzeri Menéndez

Ricardo Pérez-Aguila

Centro de Investigación en Tecnologías de Información y Automatización (CENTIA) Universidad de las Américas, Puebla (UDLAP) Ex-Hacienda Santa Catarina Mártir México, 72820, Cholula, Puebla [email protected] [email protected] [email protected]

Abstract

This paper presents a procedure for comparing color 2-Dimensional images through their extrusions to the 5-Dimensional colorspace. Some key operations to perform our comparison process include the computation of 5D hypervolumes and 5D Boolean operations (intersection and union). Finally, it describes an application of the proposed procedure under the context of the comparison and classification of a volcano's fumaroles.

1. Introduction

The topic related to comparing color 2D-images has been widely considered in several works by proposing specific methods to achieve this process, see for example Huttenlocher [5], Jurisica [7] or Pass [9]. In the next sections we will describe our proposed method for comparing color 2D-images which can be resumed in the following way: a) Extruding color 2D-images towards the 5D colorspace (section 2). b) Computing the 5D hypervolumes of the extruded images (section 3). c) Determining if two color 2D-images are “initially similar” (section 4). d) Computing the intersection between two extruded images (section 5). e) Determining if two color 2D-images are similar (section 6). Finally, section 7 presents the algorithm to perform the proposed comparison method and, section 8, briefly describes an application.

2. Extruding Color 2D-images Towards the 5D Colorspace

The color 2D-images are extruded towards the 5D colorspace: where X1, X2, X3, X4 and X5 coordinates correspond to the pixels’ values x1, x2, R (the red component), G (the green component) and B

(the blue component), respectively [4]. This way the extrusion of each pixel will be a 5D hyperprism h and n will indicate the total number of hyperprisms obtained for a color 2D-image. It is recommended to avoid zero values for components R, G and B in order to obtain for each pixel its corresponding 5D hyperprism. j

3. Computing the 5D Hypervolume of the Extruded Images

Let H the set of 5D hyperprisms for a color 2D-image. Now, we will compute the total 5D hypervolume HV of this set, i.e. the sum of the 5D hypervolume of each one of its hyperprisms: HV =

∑ hypervolume h ∈ H n

(

i

i

)

=1

The hypervolume of a 5D hyperprism can be easily computed through the product of its values * 2 1 = 2 correspond to the dimenwhere 1 sions of a pixel in the original 2D-image. x Side

x Side

x Side * R * G * B

x Side = 1

4. Determining if Two Color 2D-images are “Initially Similar”

Ha Hb be the corresponding sets of 5D hyperprisms for two color 2D-images a and b with their respective computed 5D hypervolumes HV and HV . If we assume that the color components R, G and B are inside the range [1, 256] (where 1 indicates the least intensity), then we can expect that the hyperprism of a white pixel (R=256, G=256, B=256) will have the maximum 5D hypervolume (in fact 2563 u ), while the hyperprism of a black pixel (R=1, G=1, B=1) will have the minimum 5D hypervolume (13 u ). If the majority of the pixels of an image are dark then its associated 5D hypervolume will be less than the associated 5D hypervolume of a image whose pixels are lighter and therefore, both images will have numeric differences related with the color of their pixels. These differences Let

and

a

b

Proceedings of the 15th International Conference on Electronics, Communications and Computers (CONIELECOMP 2005) 0-7695-2283-1/05 $20.00 © 2005 IEEE

5

5

if HVa < HVb

The images from Fig. 2 were classified, according to our proposed procedure, as not “initially similar”. Let ε1=0.05, HV is equal to 10,742,439 u , HV is 9,819,038 u and Q (HV ,HV ) ≈ 0.085. Therefore the condition Q ≤ ε1 is not fulfilled.

if HVb < HVa if HVa = HVb

5. Computing the Intersection Between Two Extruded Images

can be determined through the computation of the function Q between the total hypervolumes according to: a,b

 HVa 1 − HV b   HVb Qa b ( HV a , HVb ) = 1 −  HVa  0   Let ε1 be an arbitrary value ,

b

a,b

For example, consider the images presented in Fig. 1 and ε1 = 0.05 (both images have a size of 320 × 240 pixels). Then HV (according to our implementation) is 5,146,844 u5 (where u5 stands for 5D hypercubical units) and HV is 4,996,787 u5. Therefore Q (HV ,HV ) ≈ 0.029 which implies that Q ≤ ε1 and thus the images are “initially similar”. a

b

a,b

a

b

a,b

Image a

a

b

a,b

a,b

Q ≤ ε1

b

a,b

such that 0 ≤ ε1 ≤ 1. Then, we will propose that two images a and b are “initially similar” (a second comparison will be considered in section 6) if the Q of the 5D hypervolumes of the corresponding sets H and H satisfies the inequality (in fact ε1 is an allowed difference): a

5

a

5

Now we will compute the intersection between Ha and Hb (the corresponding sets of 5D hyperprisms for color 2D-images a and b) which were classified as “initially similar”. If the sets of hyperprisms (i.e. sets of 5D Orthogonal Polytopes) are represented through schemes as Hyperoctrees [12], Hypervoxelizations [6] or the Extreme Vertices Model [1], then this Boolean operation would be performed through their corresponding algorithms by intersecting the corresponding 5D hyperprisms. However, this process can be achieved in a very simple way by considering only two points of each 5D hyperprism: one of the points will be (x1, x2, 0, 0, 0) while the other will be (x1 + 1, x2 + 1, R, G, B). These two points will define a segment which is the main diagonal that connects the bases of a 5D hyperprism (see Fig. 3).

Image b

Figure 1. Two images classified as “initially similar” (see text for details; images obtained from the CENAPRED [3]).

Image a

Image b

Figure 2. Two images classified as not “initially similar” (see text for details; images obtained from the CENAPRED [3]).

Proceedings of the 15th International Conference on Electronics, Communications and Computers (CONIELECOMP 2005) 0-7695-2283-1/05 $20.00 © 2005 IEEE

(x+1,y+1,R,G,B)

coordinates of the points (x +1, x +1, R , G , B ) and (x +1, x +1, R , G , B ), that is, we have the new point: (x + 1, x + 1, R , G , B ) Where R = min{R , R } G = min{G , G } B = min{B , B } The new segment’s vertices (x , x , 0, 0, 0) and (x + 1, x + 1, R , G , B ) will correspond to the main diagonal of the 5D hyperprism h which is the intersection between 5D hyperprisms h and h (see Fig. 4). The final set H of hyperprisms h will correspond to the intersection between the 5D colorspace’s extrusions of image a and image b. In order to get the resultant color 2D-image of the intersection operation, we only have to consider again its 5D hyperprisms’ main diagonals. The main diagonal’s first point (x1, x , 0, 0, 0) will indicate the coordinates of the original pixel (obviously x and x ) while the last three coordinates of the second point (x + 1, x + 1, R, G, B) will indicate their appropriate color. In the Fig. 5 the color 2D-image, that is the result of intersecting the 5D colorspace’s extrusions of images presented in Fig. 1 is shown. 1

1

2

j

j

2

1

2

k

K

k

i

K

i

2

i

i

j

K

i

j

k

k

K

j

1

1

i

j

2

k

k

i

c

(x,y,0,0,0)

Figure 3. The main diagonal of a hyperprism in the 5D colorspace.

Let h ∈ H and h ∈ H be two 5D hyperprisms with the same x1 and x coordinates. The points’ coordinates of the diagonal associated to h are then (x , x , 0, 0, 0) and (x + 1, x + 1, R , G , B ); while the points’ coordinates of the diagonal associated to h will be (x , x , 0, 0, 0) and (x + 1, x + 1, R , G , B ). i

a

j

b

2

i

1

2

1

2

i

i

i

j

1

2

1

2

j

j

j

The required Boolean operation, intersection, can be performed by selecting only the minimum (

2

1

1

2

2

,

x + 1, y + 1, R , G , B ) i

i

i

(

x + 1, y + 1, R , G , B j

∩

j

j

)

(

x + 1, y + 1, Rk , Gk , Bk )

=

( x, y ,0,0,0)

( x, y ,0,0,0)

( x, y ,0,0,0)

hj ∈ Hb hi ∈ H a hk ∈ H c Figure 4. Computing the intersection between two 5D hyperprisms in the 5D colorspace.

∩ Image a

j

k

=

Image b

Figure 5. Computing the intersection between the 5D colorspace’s extrusions of two color 2D-images “initially similar” (from Figure 1; images a and b obtained from the CENAPRED [3]).

Proceedings of the 15th International Conference on Electronics, Communications and Computers (CONIELECOMP 2005) 0-7695-2283-1/05 $20.00 © 2005 IEEE

6. Determining if Two Color 2D-images are Similar

We will compute the 5D hypervolume HVc of the set of prisms which are the result of the intersection between Ha and Hb (the 5D extrusions of the images being compared). The intersection between Ha and Hb will imply that the set of prisms Hc is composed by the 5D hypervolume that is common to Ha and Hb. Obviously there is 5D hypervolume of Ha not included in Hc and there is 5D hypervolume of Hb not included in Hc. We will compute the proportion of the 5D hypervolume that belongs to Ha but not included in Hc by the following function:

Q a c = 1 − HVc HV a ,

In a similar way, the proportion of the 5D hypervolume that belongs to Hb but not included in Hc can be computed by:

Qb c = 1 − HVc HVb ,

be an arbitrary assigned value such that 0 ≤ εa ≤ 1. εa will indicate the allowed proportion of Let

εa

5D hypervolume of H that is not included in H . In a similar way, let εb be an arbitrary value such that 0 ≤ εb ≤ 1 where εb will indicate the allowed proportion of 5D hypervolume of H not included in H . We will assume that two images and are similar if their satisfy both inequalities: and a

c

b

c

a

b

Qa,c

Qb,c

Qa,c Qb,c

≤ εa ≤ εb

For example, consider the images and their intersection presented in Fig. 5. Since section 4, we presented that H = 5,146,844 ( stands for 5D = 4,996,787 . The 5D hypercubical units) and of the intersection between and hypervolume is 3,744,778 . Then ≈ 0.272 and Q ≈ 0.25. Let ε =ε = 0.30, then we have, that through our procedure, both ≤ ε and ≤ ε are satisfied. Therefore, color 2D images and in Fig. 1 are classified as similar. u

Va

5

u

5

u

HVb

5

Ha

HVc u

Hb

a

5

Qa,c

b,c

b

Qa,c

a

Qb,c

a

b

b

7. The Algorithm

The whole proposed procedure (sections 2 to 6) for comparing two color 2D-images can be summarized through the following Algorithm:

Boolean imagesAreSimilar(Image a, Image b, float e_1, float e_a, float e_b) { // We get the sets of 5D hyperprisms (extruded pixels) for images a and b. Set Ha = getExtrudedPixelsFromImage(a); Set Hb = getExtrudedPixelsFromImage(b); // We calculate the 5D hypervolumes of the sets of the hyperprisms in Ha and Hb. Integer Hva = calculateHypervolume(Ha); Integer Hvb = calculateHypervolume(Hb); /* We calculate the numeric difference between the 5D hypervolumes of the sets of hyperprisms. */ float Qab = calculateQab(Hva, Hvb); /* If the numeric difference is less or equal than the allowed difference indicated by the input value e_1 then the images are “initially similar”. */ if(Qab <= e_1) { /* We get the set of 5D hyperprisms which is the intersection between the 5D hyperprisms in Ha and Hb. */ Set Hc = intersection(Ha, Hb); /* It is calculated the 5D hypervolume of the intersection between the sets of hyperprisms Ha and Hb. */ Integer Hvc = calculateHypervolume(Hc); // It is calculated the proportion of hypervolume in Ha not included in Hc. float Qac = calculateQac(Hva, Hvc); // It is calculated the proportion of hypervolume in Hb not included in Hc. float Qbc = calculateQbc(Hvb, Hvb); /* If both proportions of hypervolume not included in Hc are less or equal than the allowed proportion indicated by input values e_a and e_b then */ if((Qac <= e_a) && (Qbc <= e_b)) return true; // The images are similar. return false; // The images are not similar. } return false; // The images are not “initially similar”. }

Proceedings of the 15th International Conference on Electronics, Communications and Computers (CONIELECOMP 2005) 0-7695-2283-1/05 $20.00 © 2005 IEEE

a)

b)

Figure 6. Two possible situations related with the visualization of a volcano’s silhouette. a) The silhouette covered by snow. b) The silhouette visualized as a dark region (Images obtained from the CENAPRED [3]).

8. Application

The above proposed method for comparing images has been used in an experimental application related to the Popocatépetl volcano (located in the limits of Puebla state in México; and active and under monitoring since 1997) in order to evaluate its fumaroles [12]. We selected a total of 76 images from CENAPRED archives [3] which compose a case base. These images represent some of the Popocatépetl volcano fumaroles during the year 2003. The selected images have a resolution of 640 × 480 pixels and 24-bits color under the format JPG. The objective of our application is given an input image, to obtain a subset of the 76 selected images which are classified as similar to that input image. In order to normalize the images in the case base, they were scaled down to a resolution of 320 × 240 pixels and segmented through a process of Multilevel Thresholding [8]. Through this procedure we converted each 24-bits color image (with 16,777,216 possible colors or 256 possible values for each one of its color components) to a 4,096 colors image (where red, green and blue components have each one only 16 possible values). The following is an example of the function of Multilevel Thresholding over one of the color components (where g(x,y) is the value of green component for a pixel in (x,y) and G(x,y) is its new assigned value): G( x, y ) = 16 g ( x, y )  + 8  16  Since we are interested in comparing images by considering only the volcano’s fumaroles, we will try to eliminate some “noise” that could have impact on the retrieval process. We will consider the volcano’s silhouette as a source of “noise” because it has been observed that some images present that silhouette covered by snow while others don’t present that situation and therefore the volcano’s silhouette is visualized as a darker region (For example, see Fig. 6). As commented in section 4, the 5D hypervolume of an extruded lighter (darker) pixel will have a greater (lesser) value. The situation related to the silhouette’s visualization is closely linked to the final 5D

hypervolume of an extruded image, because two images with similar fumaroles could have different silhouettes and therefore their total 5D hypervolumes could have important differences that could classify them as not “initially similar”. We will eliminate this “noise” by assigning to all our selected images in the cases base (and the input image) the same volcano’s silhouette. This will be performed by computing the union of each one of the 76 extruded images with a image (also extruded to 5D colorspace) that contains a completely white volcano’s silhouette and a black background (see Fig. 7).

Figure 7. The volcano’s silhouette to assign to the images in the case base.

The union of two extruded images can be performed in a very simple way by considering all the intersection’s process (commented in section 5) except in the new 5D hyperprism’s determination. In the intersection’s case, we determined one of the points in the new 5D hyperprism’s main diagonal as: (X1 + 1, X + 1, R , G , B ) Where R = min{R , R } G = min{G , G } B = min{B , B } If we are performing the union’s case, then we change the new point’s determination by: R = max{R , R } G = max{G , G } B = max{B , B } Independently of the silhouette’s coloration in the original images, the union of them with image of Figure 7 will assign the white silhouette because all its color components are the highest possible, while the black background of Figure 7 won’t affect the original images’ background because its color components are the lowest possible (see two examples in Figure 8). 2

k

k

k

i

k

i

k

i

j

j

k

i

k

i

k

i

Proceedings of the 15th International Conference on Electronics, Communications and Computers (CONIELECOMP 2005) 0-7695-2283-1/05 $20.00 © 2005 IEEE

j

j

j

j

k

∪

=

∪

=

Figure 8. Two examples of the assignation of the same volcano’s silhouette by performing the union of images 9. Conclusions and Future Work

In this paper we have presented the first results related to our methodology for comparing color 2Dimages by analyzing some relations in a 5-dimensional context. As mentioned before, some of the key operations include the computing of 5D hypervolumes and 5D Boolean operations (intersection and union). However, our perspectives for future research include the consideration of a more detailed geometrical and topological analysis over the 5D Orthogonal Polytopes (these polytopes are the result of the union of the set of 5D hyperprisms that represent the extrusions of the images' pixels to the 5D colorspace) in order to determine whether two images are similar or not. In the topic related to the geometrical and topological analysis of these 5D polytopes we can mention some important procedures to be applied in our methodology for comparing images: • Classification of edges, faces and volumes as manifold or non-manifold elements, see [10] and [11]. • Application of a similarity metric described by Jurisica [7] that considers two images a and b as similar according to the number of geometric transformations applied to the first one in order to obtain the second one. In fact, we will extend that procedure by considering 5D geometric transformations: translations, reflections and rotations around volumes (the rotations in the n-dimensional space are around a (n-2)-dimensional subspace, for more details about this topic see [4]). 10. References

[1] Aguilera, Antonio. Orthogonal Polyhedra: Study and Application. Doctoral Thesis. Universitat Politècnica de Catalunya, 1998. [2] Aguilera, Antonio; Lázzeri Menéndez, Santos Gerardo & Pérez-Aguila, Ricardo. Image Based Reasoning Applied to the Comparison of the Popocatépetl Volcano’s Fumaroles. To appear in the IX Ibero-American Conference on Artificial Intelligence IBERAMIA 2004; Workshop on Deduction and Reasoning Techniques DRT’04. Puebla, México, November 22-26, 2004.

[3] Centro Nacional de Prevención de Desastres (CENAPRED), México. Web Site (october 2003): http://www.cenapred.unam.mx [4] Duffin, Kirk & Barnett, William. Spiders: A new user interface for rotation and visualization of n-dimensional points sets. Proceedings of the 1994 IEEE Conference on Scientific Visualization. [5] Huttenlocher, D. P. & Klauderman, G. A. and Rucklidge, W. J. Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 15 pp. 850-863, 1993. [6] Jonas, Amnon & Kiryati, Nahum. Digital Representation Schemes for 3-D Curves. Technical Report CC PUB #114, The Technion-Israel Institute of Technology, Haifa, Israel, July 1995. [7] Jurisica, Igor & Glasgow, Janice I. Extending Case-Based Reasoning by Discovering and Using Features in IVF. Proceedings of the 2000 ACM Symposium on Applied Computing, pp. 52-59. Villa Olmo, Como, Italy, March 19-21, 2000. [8] Kurmyshev, Evguenii & Sánchez-Yáñez, Raúl. Particiones Difusas para la Umbralización Multinivel de Imágenes. Proceedings of the 12th International Conference on Electronics, Communications and Computers CONIELECOMP 2002, pp. 224-227. February 25 to 27, 2002. Acapulco, Guerrero, México. [9] Pass, G.; Zabih, R. and Miller, J.. Comparing images using color coherence vectors. In Proceedings of ACM Multimedia 96, pp. 65-73, Boston MA USA, 1996. [10] Pérez Aguila, Ricardo & Aguilera Ramírez, Antonio. Classifying Edges and Faces as Manifold Or Non-Manifold Elements in 4D Orthogonal Pseudo-Polytopes. Posters Papers Proceedings of the 11th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision WSCG 2003. Editor: Skala, V. ISBN: 80903100-2-8. February 3 to 7, 2003. Plzen, Czech Republic. [11] Pérez Aguila, Ricardo. The Extreme Vértices Model in the 4D space and its Applications in the Visualization and Análisis of Multidimensional Data Under the Context of a Geographical Information System. M.Sc. Thesis. Universidad de las Américas, Puebla, 2003. [12] Srihari, Sargun N. & Yau, Mann-May. A Hierarchical Data Structure for Multidimensional Digital Images. Communications of the ACM, July 1983, Volume 26, Number 7, pp. 504-515.

Proceedings of the 15th International Conference on Electronics, Communications and Computers (CONIELECOMP 2005) 0-7695-2283-1/05 $20.00 © 2005 IEEE