51

Retinal Visual System based Contrast Measurement in Images Debashis Sen and Sankar K. Pal

Center for Soft Computing Research, Indian Statistical Institute, 203 B. T. Road, Kolkata, India 700108. E-mail: {dsen_t.sankar}@isical.ac.in.

Abstract-A new method of measuring image contrast based on local band-limited approach and center-surround retinal receptive field model is devised in this paper. Various evidences from physiological studies corresponding to contrast perception is considered. This method works at multiple scales (frequency bands) and combines the contrast measures obtained at different scales using LP-norm• Demonstrations of contrast measurement in grayscale and color images using the new method are consid­ ered to show its suitability.

Index Terms-Band pass filter, center-surround retinal recep­ tive field, local band-limited contrast, low pass filter, multiple frequency channels

I.

INTRODUCTION

Contrast enhancement is an important image processing technique that makes various contents of images easily distin­ guishable through suitable increase in contrast. In general, per­ formance of contrast enhancement is evaluated through visual observations. Quantitative evaluation of contrast enhancement is rarely considered as it is a non-trivial task. The non-triviality often arises due to the difficulty in appropriately measuring image contrast. As mentioned in [1], [2], there does not exist any universally accepted measure of contrast. Measures of dispersion (local and global) such as variance, standard deviation and entropy have been used to represent contrast for quantitative evaluation of contrast enhancement. Contrasts calculated according to Weber's and Michelson's definition [3], [4] have also been em­ ployed for the same. Contrast enhancement has been evaluated using contrast representing measures based on local gradient magnitude such as the Tenengrad value [5], and also using average distance between pixels on the gray scale [5]. These aforesaid measures hardly represent the actual image contrast viewed by the eye. Appropriate measurement of contrast at all image pixels is essential to quantitative evaluation of contrast enhancement. As mentioned in [1], [2], any reasonable measure of contrast should be at least crudely tuned according to the retinal visual system and such a measure would then probably be more credible and universally acceptable. A few crude attempts of contrast measurement in grayscale and color images based on difference of Gaussian model of the retinal receptive field have been made in [6], [7]. In this paper, a novel method of image contrast measurement based on several available studies on the retinal visual system is presented. The new method of measuring image contrast

is based on local band-limited approach [4] and center­ surround retinal receptive field model [8], [9]. The concept of quantifying local contrast in complex images given in [4] and antagonistic center-surround models such as the difference of Gaussian (DoG) [10] are brought together in the approach of image contrast measurement. Evidences from physiological studies corresponding to contrast sensitivity in achromatic and color vision [3], [11], [12], and sub- and supra-threshold contrast perception [13], [14] are considered in the proposed approach. Considering the existence of multiple frequency channels in the visual system [15], the proposed method works at multiple scales (frequency bands) and combines the contrast measures obtained at different scales using LP-norm. The organization of the paper is as follows. The new approach of contrast measurement in the achromatic compo­ nent of an image is presented in Section IT and that in the chromatic components of an image is described in Section ITI. In Section IV, the proposed method of contrast measurement is demonstrated on a few real-life images. Section V concludes the paper. II.

CONTR AST DUE TO ACHROM ATIC SIGNAL

We base our method of contrast measurement on Peli's [4] local band-limited approach. As considered in [4], we assume that contrast at a grayscale image pixel should be expressed as the dimensionless ratio of the local change and the local average. Analysis on the applicability of Weber's and Michelson's contrast definition to complex images has been carried out in [4] and the related drawbacks were addressed leading to a definition of local band-limited contrast for complex images. The local band-limited contrast is defined in [4] as f3(x,y) (1) c(x,y) A(X,y) =

where f3(X,y) A(X,y)

f(x,y) * b(x,y) f(x,y) * l(x,y)

(2) (3)

In the above, f is the grayscale image under consideration, b is a band pass filter, l is a low pass filter such that it passes all energy below the pass band of b and * represents the convolution operator. Let us now briefly consider the phenomenon of perception in the retina of an eye. Photoreceptor cells, which are capable

978-1-4244-9799-7/111$26.00 ©20 11 IEEE

52

of phototransduction, are present in the retina and two main types of photoreceptors are referred to as cones and rods [8], [9]. The signals generated by phototransduction in the retina pass through such pathways that both excitatory and inhibitory fields are generated [8], [9]. It is widely accepted that the exci­ tatory and the inhibitory fields organize in the retina such that center-surround retinal (ganglion) receptive fields are formed, where the center and the surround fields are antagonistic (see Figure l(a)). The center-surround organization is called on­ center when the center is excitatory (+) and the surround is inhibitory (-) and it is called off-center in the other case. Now, it is shown in [10] that the difference of Gaussian (DoG) can be used to model center-surround retinal receptive field appropriately in the case of achromatic signal. The application of DoG based center-surround model on a grayscale image, which represents an achromatic signal, is represented as O(x,y)

=

C(x,y) - 8(x,y)

where C(x,y) 8(x,y)

=

f*

1

(g1

- g2)

f(x,y) * g (x,y) f(x,y) * g2(X,y)

(4)

(5) (6)

In the above, 0 is the output of the DoG based center-surround operator, C and 8 represent the signals from the center and the surround, respectively, and g1 and g2 represent the two Gaussian functions of the form exp( � (x2 +y2)) that make up the DoG operator g - g2. Note that the standard deviation «(7) of g1 is smaller than that of g2. We shall now bring together the concept of band-limited local contrast in complex images [4] and the concept of DoG based center-surround retinal receptive field model [10]. Observe that the DoG operator g - g2 is a band pass filter and both the Gaussian functions g1 and g2 are low pass filters. Another interesting observation is that g2 is a low pass filter that passes all energy below the pass band of the DoG operator g - g2· Therefore, we can readily use the output (0) of the band pass filter (DoG operator) g - g2 and the output (8) of the low pass filter g2 in the local band-limited contrast definition given in (1) as follows:

1

1

1

1

c(x,y)

=

f3(x,y) ,x(x,y)

=

O(x,y) 8(x,y)

(7)

The quantity Ic(x,y)1 gives a local band-limited contrast measure at the image pixel (x,y). In the calculation of c(x,y), we consider that the standard deviations of g1 and g2 are related as (792 M x (791' where M is an arbitrary real value greater than unity. Now, evidence of presence of multiple spatial frequency selective channels in the retina has been found [15]. The measure Ic(x,y)1 is obtained from a single spatial frequency channel/band. We consider the value of M 3 and in order to mimic the presence of multiple frequency channels in the retina, we consider multiple values of (791. Multiple standard deviation values of g1 give multiple spatial band pass filters g - g2 having different pass bands and hence we get =

(a) Center-surround reti- (b) Multiple nal receptive field pass channels

&

supra­ contrast

Fig. 1. Pictorial representation of some vision related phenomena of the retina during perception of an achromatic signal

multiple Ic(x,y)1 values which we represent as ICa91 (x,y)l. We consider 24 values of (791 as follows: (791

=

210g(�

v2(1

_

) M2 ) ' V

=

727r 697r . . . 67r 37r , 80' 80 80 ' 80 ,

(8)

where v E [O , 7r] is the frequency at which the pass band of the underlying band pass filter peaks (center/peak frequency). Note that, 7r (27r x !) corresponds to the maximum possible spatial frequency ! cycles per pixel width and the unit of (791 is pixel width. The pass bands (frequency channels) of one dimensional equivalents of the 24 band pass filters (normalized such that the value at peak frequency is unity) corresponding to the 24 values of (791 are shown in Figure l(b). Note that the application of the DoG operator g - g2 on a grayscale image considering multiple standard deviation values of g1 in order to measure contrast essentially means that the contrast is measured at multiple scales. In order to get a single contrast value at a pixel (x,y), we need to combine the contrast measures (lca91 (x,y)l) obtained at multiple scales. In order to perform the aforesaid combination, we consider findings from the study of sub- and supra-threshold contrast perception reported in [13]. While sub-threshold contrast per­ ception corresponds to contrast sensitivity at contrast levels near to the minimum contrast required for detection of a pattern, supra-threshold contrast perception corresponds to contrast sensitivity at much higher contrast levels. In sub­ threshold contrast perception, it has been found that contrast sensitivity is lower when the underlying pattern has low and high spatial frequencies compared to when the pattern has spatial frequencies in between low and high. Whereas, in supra-threshold contrast perception, contrast sensitivity char­ acteristics show very little variation across spatial frequencies [13]. The illustration in Figure l(c) reproduced from [13] demonstrates the aforesaid phenomena. We find that LV-norm can be used to perform the combination of all the ICan (x,y) ls in such a way that the aforesaid phenomena about sub­ and supra-threshold contrast perception are mimicked. We combine the measures ICan (x,y)I, \:1(791 as follows:

1

cdx,y)

=

1

band (c) Sub­ threshold [l3]

where _

p

=

(2::(p-1 an

� 1 and

P - exp

(IOg(�)) -

1- M2

X

ICan (X,Y) IY

)

(IOg(�)

xM

- exp -

1- M2

(1.)

(9)

p

)

2

(10)

53

(a) Image with sinusoidal pattern and (b) One dimensional gray value its different contrast measures (nor- and contrast profiles across malized to the range [0, I)) at all pixels columns Fig. 2. Contrast measures obtained in an image, which has sinusoidal pattern with spatial frequency decreasing from left to right, considering different values of p; p =00, 1, 2, 5, 15 [top-middle to bottom-right]

In the above, the normalization by P ensures that the underlying pass band's magnitude value at peak frequency is unity (see Figure I(b)). The measure cdx,y) gives the contrast measure of the grayscale image j, which represents an achromatic signal, at a pixel (x,y) . When p 1 (L1 norm) is considered in (9), the sub-threshold contrast perception is best mimicked and when p 00 (maximum norm) is considered in (9), the supra-threshold contrast perception is best mimicked. Figure 2(a) shows an image having sinusoidal pattern with varying magnitude (across rows) and spatial frequency (across columns), and the contrast measure CL obtained at all pixels using different values of p, and Figure 2(b) shows their cor­ responding one dimensional profiles across columns. Observe the cases of p 1 and p 2 in Figure 2(b), where contrast sensitivity is higher when the spatial frequency is in between low and high, and the cases of p 15 and p 00, where con­ trast sensitivity hardly varies across spatial frequencies. Hence, it is evident in Figure 2 that with increase in the value of p, the contrast sensitivity characteristics gradually changes from that of sub-threshold contrast perception to supra-threshold contrast perception. Contrast levels in an image may vary from the minimum contrast level required for detection of a pattern to much higher contrast levels. Therefore, the choice of p in (9) is not an obvious one and any analysis based on cdx,y) should at least involve the extreme cases p 1 and p 00. =

=

=

=

=

=

=

=

III. CONTR AST DUE TO CHROM ATIC SIGNALS As mentioned in Section II, signals generated by phototrans­ duction in the retina create excitatory and inhibitory fields, which organize such that center-surround retinal receptive fields are formed with antagonistic center and surround. In the case of chromatic signals, the center-surround retinal receptive fields are formed in color-opponent organization [8], [9]. The color-opponent organization of center-surround retinal receptive field is characterized by the red, green, blue and yellow components of color. A center-surround retinal receptive field in color-opponent organization would be any one of the following four; a red component center with an antagonistic green component surround, a green component center with an antagonistic red component surround, a blue component center with an antagonistic yellow component surround, a yellow component center with an antagonistic blue

component surround. During perception of chromatic signals, all the aforesaid four types of center-surround retinal receptive fields are formed in the retina. We now need an operator that mimics the center-surround retinal receptive field in color-opponent organization for appli­ cation on color components of color images to calculate con­ trast. We consider CIE L *a*b* color components (CIE stan­ dard illuminant 065), where L * represents lightness (achro­ matic component), a* represents the red-green opponent color component (red-positive, green-negative) and b* represents yellow-blue opponent color component (yellow-positive, blue­ negative) [3]. The operator mimicking the center-surround retinal receptive field in color-opponent organization would be applied on the a* and b* components of color images. The DoG operator model of center-surround receptive field would not be appropriate in this case. From the findings reported in [11], [12], we infer that the operator to be applied on a* and b* components of color images should represent a low pass filter and not a band pass filter like the DoG operator. Therefore, we suggest that instead of the DoG operator gl- g2, the operator gl +g2 be considered. Hence, we have: O (x,y)

=

O(x, y) +S(x, y)

=

j * (gl +g2)

(11)

where 0 is now the output of the operator gl +g2, which is applied on the a* and b* color components of the underlying color image and j is a opponent color component. Note that as mentioned in Section II, both the Gaussian functions gl and g2 are low pass filters and hence the operator gl +g2 is also a low pass filter. We shall now see how the operator gl + g2 mimics the center-surround retinal receptive field in color­ opponent organization. Consider the operation 0 +S (notation ( x,y) dropped for simplicity) in red-green (a*) opponent color component, where we have 0 ==Re- Ge and S==Rs - Gs and hence we get: O+S==( Re- Gs)+(-Ge+Rs)

(12)

where Re, Ge, Rs and Gs represent red component center, green component center, red component surround and green component surround, respectively. From (12), it is evident that we get a red component center with an antagonistic green component surround and a green component center with an antagonistic red component surround. Similarly, considering the operation 0 + S in yellow-blue (b*) opponent color component, where we have 0 ==Ye- Be and S==Ys - Bs and hence we get: O+S==( Ye- Bs)+(-Be+Ys)

(13)

where Ye, Be, Ys and Bs represent yellow component center, blue component center, yellow component surround and blue component surround, respectively. From (13), it is evident that we get a yellow component center with an antagonistic blue component surround, a blue component center with an antagonistic yellow component surround. The four center­ surround retinal receptive field mimicked by applying gl +g2 (0 +S) on the a* and b* opponent color components is given in Figure 3(a).

54

o

• +

p-J

(a) Center-surround retinal receptive fields in color (b) Multiple low pass opponent organization channels Fig. 3. Pictorial representation of some vision related phenomena of the retina during perception of chromatic signals a* & b*

(a) Image with sinusoidal pattern and (b) One dimensional color com­ its different contrast measures (nor- ponent value and contrast profiles malized to the range [0,1]) at all pixels across columns

We calculate the local band-limited contrast in the opponent color components of a color image as

Fig. 4. Contrast measures obtained in an image representing an opponent color component, which has sinusoidal pattern with spatial frequency decreas­ ing from left to right, considering different values of p; p =00, 1, 2, 5, 15 [top-middle to bottom-right)

C(X,y) _

O(x,y) - f(x,y) S(x,y)

(14)

follows:

1 0.5

c<;(x,y) where !(x,y) represents local average calculated considering a k x k window, where k » (J92' The quantity O(x,y) f(x,y) approximately represents output from a band pass filter with pass band almost same as that of the low pass filter gl +g2 except at zero frequency, where the pass band magnitude is zero. The quantity Ic(x,y) 1 gives the contrast measure at a pixel (x,y). Similar to Section II, we consider here the multiple values of (J91 given in (8) in order to mimic the presence of multiple frequency channels in the retina. We consider the same values of (J91 as used in Section II, because it is reported in [12] that the retinal responses to chromatic and achromatic signals are of the same order. Multiple standard deviation values of gl give multiple spatial low pass filters gl + g2 having different pass bands and hence we get multiple !c(x,y)1 values which we represent as !c0'91 (x,Y) I. The pass bands (frequency channels) of one dimensional equivalents of the 24 low pass filters (normalized such that the maximum pass band magnitude value is unity) corresponding to the 24 values of (J91 are shown in Figure 3(b). Similar to Section II, we need to combine the contrast measures ICO'n (x,y)1 in order to get a single contrast value at a pixel (x,y) and we perform the combination considering findings from the study of sub- and supra-threshold contrast perception in the case of chromatic signals reported in [14]. In the case of chromatic signals, it has been found that during sub-threshold contrast perception, contrast sensitivity is lower when the underlying pattern has high spatial frequencies compared to when the pattern has low spatial frequencies and contrast sensitivity characteristics show very little varia­ tion across spatial frequencies during supra-threshold contrast perception [14]. It is very interesting to find that similar to the case of achromatic signal, LP-norm can be used in the case of chromatic signals in order to perform the combination of all the !c0'91 (x,y)ls in such a way that the aforesaid phenomena about sub- and supra-threshold contrast perception is mimicked. We combine the measures !c0'91 (x,y)I, '
=

(2:)0.5 0'91

X

IC0'91 (X,Y) IY

)

(l) P

(15)

where p 2: and ( represents one of the two opponent color components a* ( R- G) and b* ( Y- B). In the above, the scaling by ensures that the maximum magnitude value of the pass band of the underlying low pass filter is unity (see Figure 3(b)). The measure c<;(x,y) gives the contrast measure in the opponent color component f of the underlying color image at a pixel (x,y). When p (L1 norm) is considered in (9), the sub-threshold contrast perception is best mimicked and when p 00 (maximum norm) is considered in (9), the supra-threshold contrast perception is best mimicked. Figure 4(a) shows an image, which represents an opponent color component of a color image, having sinusoidal pattern with varying magnitude (across rows) and spatial frequency (across columns), and the contrast measure c( obtained at all pixels using different values of p, and Figure 4(b) shows their corresponding one dimensional profiles across columns. Notice the same wavy envelope in all the one dimensional profiles of the contrast measure c( obtained using different values of p. The wavy envelope is due to the usage of the quantity !(x,y) in (14) in order to make the underlying pass band's magnitude zero at zero frequency and hence we can ignore the wavy envelope while analyzing Figure 4(b). Observe the cases of p and p 2 in Figure 4(b), where contrast sensitivity is lower when the spatial frequency is high, and the cases of p and p 00, where contrast sensitivity hardly varies across spatial frequencies. Hence, it is evident in Figure 4 that with increase in the value of p, the contrast sensitivity characteristics gradually changes from that of sub­ threshold color contrast perception to supra-threshold color contrast perception. Color contrast levels in a color image may vary from the minimum contrast level required for detection of a pattern to much higher levels and hence the choice of p is not obvious like in Section II. =

1

=

=

15

=

1

=

=

IV. CONTR AST MEASUREMENT AND DEMONSTR ATION Once the contrasts in L* (achromatic signal), a* and b* (chromatic signals) components corresponding to an image are

55

calculated, the contrast measure CI of the image at a pixel (x, y) is obtained as follows: CI(X, y)

=

cdx, y)+C(R-G)(X, y)+C(Y_B)(X, y)

(16)

where CL is the contrast in L* (lightness) component, C(R-G) and C(Y -B) are the contrasts in the opponent color components a* and b*. Note that, we consider the same value of p while using LP norm in the calculation of contrasts due to achromatic and chromatic signals corresponding to an image. Figure 5 shows the contrast CI calculated at all pixels of a grayscale image and a color image using different values of p.

r-m)

p- I

p-J

p-JJ

(a) A grayscale image and its dif- (b) A color image and its different contrast measures ferent contrast measures Fig. 5. Contrast measures (normalized to the range [0, I]) calculated at all pixels of a grayscale image and a color image considering different values of P; P =00, 1, 2, 5, 15 [top-middle to bottom-right)

Consider the two images in Figures 6 and 7, which are shown at three different levels of overall contrast. It is visually evident that in both the figures 'image 3' has the highest overall contrast. Let us now consider the proposed method of contrast measurement. The sum of measured contrasts at all pixels corresponding to the image in Figure 6are 1.2537x 105, 1.9059x 105, 2. 1917x 105 for 'image 1', 'image 2' and 'image 3', respectively. The sum of measured contrasts at all pixels corresponding to the image in Figure 7 are 2.0475 x 104, 3.5780 X 104, 3.9753 X 104 for 'image 1', 'image 2' and 'image 3', respectively. The plots of the contrast distributions given in Figures 6and 7 show that more pixels have higher contrast in case of 'image 3' compared to the other two. From the aforesaid observations, we find that the proposed method of contrast measurement is consistent with visual observation. • - ,I

. I (a) image I

�

G

-

"

,

(b) image 2

(c) image 3

...""

l-ilMgel

3000

......

-imlIge2

"00

"

°0

15

--, 1-""2 -,

II

Contrast Value

(d) contrast distributions

Fig. 6. A grayscale image at three contrast levels and their contrast distributions

V. CONCLUSION

A new method of measuring image contrast based on local band-limited approach and antagonistic center-surround retinal receptive field model has been devised in this paper.

(a) image I Fig. 7.

(b) image 2

(c) image 3

(d) contrast distributions

A color image at three contrast levels and their contrast distributions

Various evidences from physiological studies corresponding to contrast perception along with the existence of multiple frequency channels in the visual system have been consid­ ered. In accordance to them, the proposed method has been allowed to work at multiple scales (frequency bands) and combination of the contrast measures obtained at the different scales has been performed using LP-norm. The new method of contrast measurement in images has been found suitable in a few demonstrations involving grayscale and color images. The proposed method of image contrast measurement is a general approach, and hence, it can be used for various image processing tasks. REFERENCES [I) A. Polesel, G. Ramponi, and V. 1. Mathews, "Image enhancement via adaptive unsharp masking," IEEE Trans. Image Process., vol. 9, no. 3, pp. 505-510, 2000. [2) v. Vonikakis, I. Andreadis, and A. Gasteratos, "Fast centre-surround contrast modification," lET Image Processing, vol. 2, no. I, pp. 19-34, 2008. [3) A. K. Jain, Fundamentals of Digital Image Processing. New Delhi, India: Prentice Hall of India, 200I. [4) E. Peli, "Contrast in complex images," Journal of the Optical Society of America A, vol. 7, no. 10, pp. 2032-2040, 1990. [5) Z. Y. Chen, B. R. Abidi, D. L. Page, and M. A. Abidi, "Gray-level grouping (GLG): an automatic method for optimized image contrast enhancement-part I: the basic method," IEEE Trans. Image Process., vol. 15, no. 8, pp. 2290-2302, 2006. [6) Y. Tadmor and D. 1. Tolhurst, "Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes," Vision Research, vol. 40, no. 22, pp. 3145-3157, 2000. [7) A. Rizzi, G. Simone, and R. Cordone, "A modified algorithm for perceived contrast measure in digital images," in Proceedings of 4th European Conference on Color in Graphics, Imaging and Vision, vol. 4, 2008, pp. 249-252. [8) E. Kandel, 1. Schwartz, and T. Jessell, Principles of Neural Science, 4th ed. McGraw-Hili Medical, 1994, Part V - Chapters 25, 26 and 27. [9) H. Blumenfeld, Neuroanatomy through Clinical Cases. Sinauer Asso­ ciates, 2002, Chapter II. [10) R. W. Rodieck and 1. Stone, "Analysis of receptive fields of cat retinal ganglion cells," Journal of Neurophysiology, vol. 28, no. 5, pp. 833-849, 1965. [II) K. T. Mullen, "The contrast sensitivity of human colour vision to red­ green and blue-yellow chromatic gratings," The Journal of Physiology, vol. 359, pp. 381-400, 1985. [12) 1. M. Rovamo, M. I. Kankaanpiiii, and H. Kukkonen, "Modelling spatial contrast sensitivity functions for chromatic and luminance-modulated gratings," Vision Research, vol. 39, no. 14, pp. 2387-2398, 1999. [B) M. A. Georgeson and G. D. Sullivan, "Contrast constancy: deblurring in human vision by spatial frequency channels," The Journal of Physiology, vol. 252, pp. 627-656, 1975. [14) R. L. P. Vimal, "Spatial color contrast matching: broad-bandpass func­ tions and the flattening effect," Vision Research, vol. 40, no. 23, pp. 3231-3243, 2000. [15) R. L. DeValois and K. K. DeValois, Spatial Vision. Oxford University Press, 1990.