VHF9 - 6L (Late-News Paper)

Saliency-based color accessibility: Theory into practice Satohiro Tajima, Kazuteru Komine NHK, 1-10-11 Kinuta, Setagaya-ku, Tokyo 157-8510, Japan Keywords: Color deficiency, Attention, Kullback–Leibler divergence, Color universal design, Human vision ABSTRACT

We propose a novel methodology to analyze the universality of images, taking into account the diversity of color perception. It quantifies and visualizes expected information loss according to the notion of visual saliency. Psychophysical experiments indicated relationships between the model’s predictions and subjective visual clarity. 1. INTRODUCTION It is important to appropriately design visual contents so that sufficient information is received not only by the majority of the audience but also by minorities (including color-vision-deficient observers) [1]. Developed on the basis of studies of confusing colors (e.g., [2]), certain techniques simulate the visual experiences of color-deficient people [3]. However, it has been difficult to evaluate unstructured visual data—such as natural images—according to the results of simulations, since they do not provide the objective information needed for effective evaluation. In this report, we propose a novel method for analysis of the universality of image viewability based on the notion of visual saliency [4], which has been developed in research on human visual attention. We also conducted psychophysical experiments to validate the plausibility of the proposed methodology. 2. THEORY Computing saliency maps for unstructured visual data is a type of structural analysis. The proposed method relies on the concept that the results of structural analysis can be used for viewability analysis; that is, if we have two saliency maps derived from two different visual processing models (i.e., trichromacy and dichromacy), the structural divergence between the maps will ideally capture the characteristics of the perceptual differences between the two observers. The proposed method analyzes images in three steps, the first two of which can be applied to different visual processing models in parallel (Fig. 1): (i) Early visual system: predicting early sensory responses (luminance, color, orientation, etc.) according to models of the lens, cone distribution, and subsequent neural responses; (ii) Saliency map: computing the distribution of saliency at pixels r for an observer Oi in the form of probability density, P(r|Oi); and

(iii) Information loss: comparing the saliency maps (Fig. 2), by which we define the local information loss L for the observer Oi (as compared with that for another observer Oj) as follows: (1) L(r) P(r|Oi) {ln P(r|Oi) – ln P(r|Oj)} Note that the value computed by integrating L over all pixels r equals the Kullback–Leibler divergence between the two saliency distributions. 3. PSYCHOPHYSICAL EXPERIMENTS 3.1 Method Five color-deficient (CD) participants (including four deuteranopes and one protanope) and 18 control participants with common color vision judged the subjective viewability of presented images. The participants’ visual properties were confirmed by the Ishihara test and by estimation of confusing colors during a color-matching task. In the viewability judgments, colored illustrations were compared to modified ones. We tested two methods of modification: dichromat simulation (an approximation of deuteranopia according to the method proposed in [3]) and saliency-based improvement (empirical modification intended to reduce the information loss described in Eq. [1]). The participants subjectively judged whether the original or the modified image seemed more visually clear in a two-alternative forced choice paradigm. In each trial, the target of judgment was restricted to a specified local region of the image, which was presented within the whole image (“with context”) or in a cropped form (“without context”). Six different local regions within each image pair were judged four times by each participant.

Fig. 1 Computation of visual saliency We extended the conventional saliency model to account for individual differences in the early visual system.

ISSN-L 1883-2490/19/1979 © 2012 ITE and SID

IDW/AD ’12

1979

Fig. 2 Visualized “information loss” The visualization demonstrates that the saliency-based information loss captures typical differences in visual experiences by common trichromats and deuteranopes.

3.2 Results and Discussion The control participants tended to judge the images simulating dichromat vision as less clear than the original ones, while the CD participants rated them as more equal (Fig. 3, left two columns). This result confirms the plausibility of the dichromat simulation. On the other hand, the images to which the saliency-based improvements had been applied were favored more often than the simulated ones by both control and CD participants (Fig. 3, right two columns). These results suggest that the saliency-based approach can effectively improve visual clarity for both common and color-deficient observers. The general pattern and statistical significance of the results were maintained when we eliminated the single participant with protanopia from the analysis. Furthermore, image-wise analysis revealed that the proportion of theoretically predicted information loss was significantly correlated with the subjective judgment of visual clarity (Fig. 4). This provides direct evidence of the relationship between saliency and subjective judgment of visual clarity. 4. APPLICATION The proposed method can be utilized for purposes of visual design in situations when one wants to avoid unintended asymmetry in information received by observers. For example, in the process of creating video graphics, it is possible to apply the aforementioned saliency analysis in order to determine whether and where some observers will lose information. Figure 5 shows an example of saliency-based improvement applied to an actual broadcasting screen. 5. CONCLUSION We have proposed and experimentally validated a new approach for universal color design. The proposed methodological framework for visualizing the loss of visual information according to objective criteria is expected to enhance the efficiency and effectiveness of the design process of a wide variety of visual contents.

1980

IDW/AD ’12

Fig. 3 Judgments of subjective visual clarity The saliency-based approach can effectively enhance visual clarity for both control and color-deficient observers.

Fig. 4 Model vs. human behavior Inter-stimulus variability in terms of clarity judgment can be predicted with statistically significant accuracy by the theoretically derived normalized information loss. The dots indicate individual stimuli (dichromat simulation).

Fig. 5 An example of an improved screen Graphics in weather forecasts were modified according to the proposed method. The modified graphics are in use in some actual broadcasts.

REFERENCES [1] B. Wong: “Color blindness,” Nature Methods, 8(6), pp.441 (2011). [2] F. H. G. Pitt: “Characteristics of dichromatic vision,” Medical Research Council Special Report Series, No. 200. London (1935). [3] F. Viénot, H. Brettel, L. Ott, M. B. A. Ben, and J. D. Mollon: “What do colour-blind people see?” Nature, 376(6536), pp.127-128 (1995). [4] L. Itti, and C. Koch: “Computational modeling of visual attention,” Nature Reviews Neuroscience, 7(3), pp.194-203 (2001).