Linear methods for input scenes restoration from signals of optical-digital pattern recognition correlator Sergey N. Starikov, Mikhail V. Konnik, Edward A. Manykin, Vladislav G. Rodin Moscow Engineering Physics Institute (State University), Russia, Moscow ABSTRACT Linear methods of restoration of input scene’s images in optical-digital correlators are described. Relatively low signal to noise ratio of a camera’s photo sensor and extensional PSF’s size are special features of considered optical-digital correlator. RAW-files of real correlation signals obtained by digital photo sensor were used for input scene’s images restoration. It is shown that modified evolution method, which employs regularization by Tikhonov, is better among linear deconvolution methods. As a regularization term, an inverse signal to noise ratio as a function of spatial frequencies was used. For additional improvement of restoration’s quality, noise analysis of boundary areas of the image to be reconstructed was performed. Experimental results on digital restoration of input scene’s images are presented. Keywords: wavefront coded imaging, digital photo camera, kinoform, image restoration, linear deconvolution methods.

1. INTRODUCTION Correlation signals captured by photo sensor can be regarded as an “intermediate” image of the input scene. Therefore it is possible to digitally restore the “true” image that corresponds to the image of input scene captured by the correlator without inserted correlation filter.1 The diffraction image correlator based on the commercial digital SLR photo camera was described in our previous works.1, 2 The correlator is intended for preliminary recognition of the two- and three-dimensional input scenes during registration in the quasimonochromatic spatially incoherent illumination. The key diagram of the correlator is analogous to Lohmann’s incoherent holographic correlator.3, 4 The described correlator can be related to hybrid optical-digital imaging systems based on “wavefront coding” paradigm.5–8 Optical-digital systems of this type allow to increase a depth of field, to compensate aberrations, reduce an out-of-focus influence, and to transmit the confidential information in the digital form through the open communication channels.? Digital restoration methods are necessary for restoration of input scene’s images from images of correlation signals. Linear non-iterative restoration methods are preferable because of large size of images of correlation signals that can be as large as 3000 × 2000 pixels. Linear methods of images restoration such as Tikhonov, Wiener and evolution are used. A brief survey of linear restoration methods used in this work is given. Results of digital restoration of input scene’s images are presented.

2. DIGITAL RESTORATION METHODS It is necessary to emphasize that in this type of optical-digital systems resulting images are large enough and typical size of images is 3000 × 2000 pixels. That is why only linear restoration methods are acceptable for restoration of the actual images in a reasonable time. Although iterative methods can produce better results when signal-to-noise ratio (SNR) is low, they have significant drawbacks: increased resource consuming and considerable time of images’ restoration. In the recently proposed paper9 was shown that linear filter with Further author information: Mikhail V. Konnik: [email protected], Sergey N. Starikov and Vladislav G. Rodin: [email protected] Edward A. Manykin: [email protected]

power-low regularization produces nearly the same results as an iterative methods. That is why linear restoration methods are used in this work. The model correlator’s work is represented by the following equation: g(x, y) = f (x, y) ⊗ h(x, y) + n(x, y),

(1)

where g(x, y) is a correlation signals (i.e. encoded image), f (x, y) represents the “true” image of the input scene, h(x, y) is the reference image (i.e. point spread function, PSF), and n(x, y) is an additive noise. In Fourier domain Eq. (1) can be presented as: G(u, v) = F (u, v) · H(u, v) + N (u, v). Inverse problem must be solved in order to restore actual images: G(u, v) F˜ (u, v) ≈ = G(u, v) · Y (u, v), H(u, v) where Y (u, v) is a deconvolution filter. The deconvolution problem is ill-posed, hence it is necessary to regularize the solution by introducing a priori limitations. The regularization introduces smoothing in the ill-posed solution and provides an approximation of the exact solution of an inverse deconvolution problem. There are several methods of searching regularization functions such as Tikhonov, Wiener, and evolution methods that can be implemented as digital deconvolution filters. Tikhonov, Wiener, and evolution filters for restoration of images from the correlation signals are described in the further subsections.

2.1 Tikhonov regularization filter The deconvolution method that is developed by Tikhonov10 can achieve optimal result of image’s restoration for every image. Regularization of an ill-posed problem is a process of searching of inverse solution. Such searching depends on numerical regularization parameter α, which allows to control smoothing of inverse solution. There are many regularizators Q(u, v) succeeding Tikhonov’s stabilization conditions: 1. Q(u, v) > 0 when u, v 6= 0, Q(0) ≥ 0; 2. with larger |u, v|, Q(u, v) ≥ C > 0. According to,10 Tikhonov’s deconvolution filter Ytikh (u, v, α) is the following: Ytikh (u, v, α) =

H(u, v)∗ . |H(u, v)|2 + α · Q(u, v)

(2)

In this work we have applied the following regularization with constant: Ytikh (u, v, α) =

H(u, v)∗ . |H(u, v)|2 + α · max|H(u, v)|2

(3)

Such easy-to-implement regularizator produces good results on experimental and modelling images despite its simplicity.

2.2 Wiener filter Weiner’s deconvolution method11 assumes that both an image and a noise are the realisations of a random process. All further formulation of Wiener’s filter is related to the assumption that noise and an original image are non-correlated. The approximation of the desired image f˜ of the ideal image f can be found by minimizing of root mean square error: e2 = E{(f − f˜)2 },

where E is the mean value of the argument. In9 was reported that in the case of relatively high SNR (about 30 and more) it is useful to perform deconvolution with modified Wiener filter: Ywiener (u, v, c) =

H(u, v)∗ |H(u, v)|2 + c ·

<|N (u,v)|2 > Φ0 (u,v)

(4)

,

where c is the regularization parameter and < |N (u, v)|2 > is the power spectrum of noise (we used the averaged Fourier power spectrum of experimental dark noise as the noise approximation in this work). The power spectrum of the object Φ0 (u, v) was approximated by a power law as following:

√

p Φ0 (u, v) = Ar−β .

(5)

where A and β are constants and r = u2 + v 2 is radial component of the spatial frequency. Numerical experiments9 have shown that parameters c = 1, β = 1 ÷ 3, A = 1 produces good results of input scene’s images restoration.

2.3 Evolution filter Application of Wiener or Tikhonov-like regularization often produces relatively good results. To achieve more degrees of freedom it is possible to control inverse power and regularization power independently. Such deconvolution filter is known as an evolution12 filter: Yevol (u, v, σ, µ, θ) =



H(u, v)∗ |H(u, v)|2

σ  ·

|H(u, v)|2 |H(u, v)|2 + θ · Q(u, v)

µ

.

(6)

Eq. (6) contains the most general formula of the non-iterative linear filtration. Such filter is very useful in practical applications because it incorporates Wiener’s filtering and broad variety of parameters: inverse power is controlled by parameter σ, smoothing power is controlled by parameter µ, and the regularization itself is controlled by the parameter θ. Such parameters are independent and hence provide the ability to control precisely the restoration process. In order to prevent uncertainness such as “divide-by-zero” we have added a machine error value eps so our filter is the following:  σ  µ H(u, v)∗ |H(u, v)|2 Yevol (u, v, σ, µ, θ) = · , |H(u, v)|2 + eps |H(u, v)|2 + θ · Q(u, v) where eps is the machine error (floating-point relative accuracy). As a regularizator Q(u, v) in the evolution filter can be used square of “noise-to-signal” ratio. Signal’s mean square Fourier magnitude can be approximated by intermediate Tikhonov’s restored image with optimal value of regularization parameter α0 . Hence the regularizator Q(u, v) is the following: < |N (u, v)|2 > Q(u, v) = 2 . H(u, v)∗ |H(u, v)|2 + α0 · max|H(u, v)|2 · G(u, v)

(7)

In this work, Gauss noise parameters were estimated from the boundary regions of correlation signals’ image. Results of restoration of the input scene’s images from the registered correlation signals are presented in the further section.

3. EXPERIMENTAL RESULTS The experimental setup of the used image correlator based on commercial digital SLR photo camera was described in.1 The input scene of the correlator was illuminated by spatially incoherent radiation of the He-Ne laser. The recognition of the input scene with multiple objects is illustrated in Fig. 1. Plane images of the test objects were printed on a paper and positioned in front of the photo camera (see Fig. 1a). The size of each object on the paper was about 35 mm; each object on the camera’s photo sensor occupies an area of 350 × 350 pixels. The PSF of the kinoform, which is used in experiments as a correlation filter for recognition of test objects, is presented in Fig. 1b. The size of the PSF was also 350 × 350 pixels. The image of the correlation signals is shown in Fig. 1c. Correlation signals are optically evaluated correlation of input objects with reference object and then registered by photo camera. The whole image’s size of the input scene registered by the camera sensor was 1944 × 1296 pixels in each colour channel. Local bright spots indicate positions in the input scene of objects that are similar to reference ones.

a)

b)

c)

Figure 1. Recognition of multiple objects: a) input objects in front of the camera, b) the reference image that is the kinoform’s PSF rotated on 180◦ , c) intensity distribution of correlation signals.

Application of deconvolution filters is required for restoration of the input scene’s image (see Fig. 1a) from registered correlation signals (see Fig. 1c). As it was mentioned above, the direct solution of the ill-posed problem by the inverse filter gives unstable results because the solution is highly sensitive to noise. As seen in Fig. 2, the image of the input scene restored by inverse filter is noised and cannot be well identified. Such example illustrates the necessity of regularization to be applied in deconvolution filters when real correlation signals are processed.

Figure 2. Input scene’s image that is restored by inverse filter.

It is significant to note that large sizes of reference image (about 15% of coded image in each dimension) are typical for pattern recognition correlators. All decoded images were obtained from the coded image (see Fig. 1c) using the PSF presented in Fig. 1b. Results of the images restoration by Tikhonov, Wiener, and evolution filters are presented in further subsections. Normalized root mean squared (RMS) error13 was calculated for quantitative estimation of the restored image’s quality. The stability coefficient χ was calculated as a relative change of regularization parameter’s value when RMS is changed on 20% from minimum for estimation of stability of deconvolution filters. Deconvolution filters were implemented as MATLAB scripts. All images of the correlation signals from the digital camera were stored in RAW format to prevent image degradation. The open source Dave Coffin’s DCRAW converter14 was used for RAW format conversion.

3.1 Tikhonov filter’s deconvolution results For restoration of the input scene’s image from the registered correlation signals we have applied Tikhonov’s restoration filter with constant regularizator as described in Eq. (3). The regularization parameter α was varied within the ambit of 100 to 10−10 to achieve best results in the group of restored images. Results of deconvolution are presented in Fig. 3.

a)

b)

c)

Figure 3. Images of correlator’s input scene restored by Tikhonov method with different regularization parameter α: a) 10−1 - excessive regularization, b) 10−5 - visually best in the group, c) 10−8 - deficient regularization.

Values of RMS error are presented in Fig. 4a for images restored by Tikhonov’s filter. According to Fig. 4a, minimal RMS error is 0.42 that corresponds to the regularization parameter α = 10−4 . The image of the input scene restored with minimal RMS error is presented in Fig. 4b. 1 0.9

Normalized RMS error

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 10−10

10−9

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Regularization parameter, α = 10n

a)

b)

Figure 4. Estimation of optimal regularization parameter α and stability χ of Tikhonov filter: a) RMS error versus the regularization parameter α; b) the image restored with minimal RMS error that corresponds to α = 10−4 . Stability coefficient for Tikhonov filter is found as χ = 12.

It can be seen that stable results of deconvolution by our implementation of Tikhonov’s filter are within the ambit of regularization parameter α ∈ [10−5 ; 10−3 ]. Hence the stability coefficient, which is relative change of parameter α within the ambit of 20% of RMS, is found as χ = 12. That is why it is relatively difficult to use Tikhonov’s method with constant regularization for automated images deconvolution from registered correlation signals in the presence of noise.

3.2 Wiener filter’s deconvolution results In the Wiener deconvolution filter the “noise-to-signal” regularizator was used (see Eq. (4)). As a “signal”, a power-law approximation was applied as it mentioned in Eq. (5); noise in this case was averaged Fourier power spectrum of experimental dark noise. We have varied regularization parameter c (see Eq. (5)) and power-low parameter β (see Eq. (4)) in order to evaluate best values of these parameters. Results of such experiments are provided in Fig. 5. Visual analysis of reconstructed images have shown that the decrease of β to 0 produces noised but sharpen images, and the increase of β beyond 3 produces blurred images. The optimal by RMS value of β is β = 0.5 and c is within the ambit c = 0.1 ÷ 1.

1 0.9

Normalized RMS error

0.8 0.7 0.6 0.5 0.4 0.3 β = 0.5 β=1 β = 1.5 β=2 β = 2.5 β=3 β = 3.5 β=4

0.2 0.1 0 10−5

10−4

10−3

10−2

10−1

100

101

102

103

104

105

Regularization parameter, c

Figure 5. Estimation of Wiener’s filter parameters: RMS error versus regularization parameter c for different β = 0.5 ÷ 4.

Using obtained values of parameters for Wiener deconvolution filter, we have restored images of input scenes from registered correlation signals. In Fig. 6 are presented the restored images with different value of the regularization c with power-low parameter β = 0.5.

a)

b)

c)

Figure 6. Images of correlator’s input scene restored by Wiener method with the power-law parameter β = 0.5 and different regularization parameters: a) c = 102 - excessive regularization, b) c = 10−1 - visually best in the group, c) c = 10−9 - deficient regularization.

From the data presented in Fig. 7a was obtained that minimal RMS error is 0.43 that corresponds to c = 10−1 and stability coefficient is found as χ = 2.5 · 103 . The image of the input scene restored with minimal RMS error is presented in Fig. 7b. 1 0.9

Normalized RMS error

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 10−5

10−4

10−3

10−2

10−1

100

101

102

103

104

105

Regularization parameter, c

a)

b)

Figure 7. Wiener filter parameters estimation: a) RMS error versus regularization parameter c for β = 0.5; b) the restored image with minimal RMS error 0.43 that corresponds to c = 10−1 and β = 0.5. Stability parameter for Wiener filter is found as χ = 2.5 · 103 .

Comparing the RMS diagram (see Fig. 7a) of Wiener filter and Tikhonov’s RMS diagram (see Fig. 4a) it can be noted that the deconvolution filter with power-law regularization produces more stable results (stability coefficient χ = 2.5 · 103 ) than Tikhonov filter (χ = 12).

3.3 Evolution filter’s deconvolution results The evolution filter is the most configurable and generalized filter among non-iterative linear filters. We have varied the following parameters: the regularization parameter θ, the inversion power σ, and the smoothing power µ. The “noise-to-signal” regularizator was used in evolution filter as well. As a “signal”, an intermediate restoration result by Tikhonov filter with experimentally found optimal α0 = 10−4 was used (as mentioned in Eq. (7)). For noise estimation, regions 64 × 64 pixels from corners of correlation signal’s image were taken and Gauss distribution parameters were fitted.

1

1

0.9

0.9

0.8

0.8

0.7

0.7

Normalized RMS error

Normalized RMS error

The process of the estimation of evolution filter’s parameters can be simplified by fixing parameter µ to match Wiener part of the filter or parameter σ to match inverse part of the filter. Then regularization parameter θ and parameter µ or σ were varied and the RMS error was calculated. Results are presented in Fig. 8. Estimation of parameters of the evolution filter allows to say that near to optimal values are σ = 0.90 for inversion power and µ = 1.25 for smooth power. Lower values of σ produce low-contrast restored images and higher values produce noise amplification and artefacts.

0.6 0.5 0.4 0.3 σ = 0.7 σ = 0.9 σ = 1.0 σ = 1.1 σ = 1.3 σ = 1.5

0.2 0.1 0 10−10

−9

10

−8

10

−7

10

−6

10

−5

10

−4

10

Regularization parameter, θ

−3

10

−2

10

0.6 0.5 0.4 0.3 µ = 0.25 µ = 0.50 µ = 0.75 µ = 1.00 µ = 1.25 µ = 1.50 µ = 1.75

0.2 0.1 0 10−10

−1

10

a)

10−9

10−8

10−7

10−6

10−5

10−4

Regularization parameter, θ

10−3

10−2

10−1

b)

Figure 8. Estimation of evolution filter parameters: a) RMS error versus regularization parameter θ at the different values of inversion parameter σ and fixed smoothing parameter µ = 1, b) RMS error versus regularization parameter θ at the different values of smoothing parameter µ and fixed inversion parameter σ = 1.

Restored images with different value of regularization are presented in Fig. 9 with optimal parameters σ = 0.90 and µ = 1.25.

a)

b)

c)

Figure 9. Images of the correlator’s input scene restored by evolution filter with different values of the regularization parameter θ: a) 101 - excessive regularization, b) 10−6 - visually best in the group, c) 10−9 - deficient regularization.

The dependency of the RMS error versus regularization parameter θ corresponds to optimal evolution filter parameters µ and σ is presented in Fig. 10a. The restored image with minimal RMS error 0.43 that corresponds

to regularization parameter θ = 10−7 is presented in Fig. 10b. It is noteworthy that visually best image in the group (Fig. 9b) is similar to the image with minimal RMS error (Fig. 10b). 1 0.9

Normalized RMS error

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 10−10

10−9

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

Regularization parameter, θ = 10n

a)

b)

Figure 10. Estimation of evolution filter parameters (inversion power σ = 0.9, smooth power µ = 1.25): a) RMS error versus regularization parameter θ, b) the restored image with minimal RMS error 0.43 that corresponds to regularization parameter θ = 10−7 . Stability parameter for evolution filter is found as χ = 3.0 · 104 .

Using data that are presented in Fig. 8, we can calculate the stability coefficient χ for the evolution filter that is relative change of regularization parameter θ when RMS is changed on 20% from its minimum. The stability coefficient for the evolution filter in this case is χ = 3.0 · 104 .

From the obtained experimental results we conclude that the evolution filter can produce more stable image reconstruction results (χ = 3.0 · 104 ) than Wiener’s (χ = 2.5 · 103 ) and Tikhonov’s (χ = 12) filters. Moreover, RMS optimal image and visually best image in the group are the same in this case for the evolution filter. Thus the evolution filter can be considered as more suitable for an automated image processing.

4. CONCLUSIONS In the present paper we have compared three linear methods for input scene’s images restoration from correlation signals. Experiments were performed with pattern recognition correlator based the commercial digital photo camera. This correlator can be regarded as optical-digital system based on “wavefront coding” paradigm. Obtained correlation signals can be mentioned as “intermediate” images of the input scene. Digital restoration of the input scene from the correlation signals were provided by three different deconvolution filters: Tikhonov, Wiener, and evolution filter. Methods of linear non-iterative images restoration are preferable in such optical-digital systems because of large size of produced images. Additionally, a reference image (i.e. a coding PSF) is usually extensional in the pattern recognition correlators that leads to narrow optical transfer function. This circumstance makes it difficult to restore images from the correlation signals. Digital deconvolution filters used in this work are characterized by the following features. Tikhonov filter employs simple constant-based regularization. Wiener filter uses power-law regularization produces and experimental noise data. The evolution filter uses “noise-to-signal” regularizator as follows: “noise” is evaluated from the boundary regions of image, and “signal” estimation is carried out by an intermediate image restored by Tikhonov’s filter. Provided analysis allows to say that all three methods give almost the same RMS error from the original image (RMS error ≈ 0.4) and comparable visual quality of restoration of input scene’s images.

It is significant that the evolution filter can produce more stable results of images restoration. Results of experiments allows us to estimate the stability coefficient for the evolution filter as 3.0 · 104 ; for Wiener filter it is equal to 2.5 · 103 and for Tikhonov filter the stability coefficient is 12.

Hence we conclude that the evolution filter is more stable and flexible in comparison with Tikhonov and Wiener filters. Thus evolution filter is more suitable for automated input scenes’ images restoration from the correlation signals.

ACKNOWLEDGMENTS This work was partially supported by the Ministry of education and science of the Russian Federation (Program “The development of the scientific potential of High School”, project RNP.2.1.2.1103).

REFERENCES [1] Sergey N. Starikov, Nikita N. Balan, Mikhail V. Konnik, Vladislav G. Rodin, Ivan V. Solyakin, and Ekaterina A. Shapkarina, “Input scene restoration in pattern recognition correlator based on digital photo camera,” in [Proc. SPIE, Optical Pattern Recognition XVIII ], 6574:65740J(1) (2007). [2] S.N. Starikov, N.N. Balan, V.G. Rodin, I.V. Solyakin, and E.A. Shapkarina, “Pattern recognition correlator based on digital photo camera,” in [Proc. SPIE, Optical Pattern Recognition XVII ], 6245(1), 62450C (2006). [3] Lohmann, A., “Matched filtering with self-luminous objects,” Applied Optics 7(3), 561–563 (1968). [4] Yu. A. Bykovsky, A. A. Markilov, V. G. Rodin, and S. N. Starikov, “Optical information processing with transformation of the spatial coherence of light,” Quantum Electronics 25(10), 1014–1019 (1995). [5] W.T. Cathey and E.R. Dowski, “New paradigm for imaging systems,” Applied Optics 41, 6080–6092 (2002). [6] Prasad, S., Torgersen, T. C., Pauca, V. P., Plemmons, R. J., and van der Gracht, J., “Engineering the pupil phase to improve image quality,” in [Proc. SPIE, Visual Information Processing XII], 5108(1), 1–12 (2003). [7] van der Gracht, J., Pauca, V. P., Setty, H., Narayanswamy, R., Plemmons, R., Prasad, S., and Torgersen, T., “Iris recognition with enhanced depth-of-field image acquistion,” in [Proc. SPIE, Visual Information Processing XIII], 5438, 120–129 (2004). [8] Plemmons, R. J., Horvath, M., Leonhardt, E., Pauca, V. P., Prasad, S., Robinson, S. B., Setty, H., Torgersen, T. C., van der Gracht, J., Dowski, E., Narayanswamy, R., and Silveira, P. E. X., “Computational imaging systems for iris recognition,” in [Proc. SPIE, Advanced Signal Processing Algorithms, Architectures, and Implementations XIV], 5559, 346–357 (2004). [9] Fienup, J. R., Griffith, D. K., Harrington, L., Kowalczyk, A. M., Miller, J. J., and Mooney, J. A., “Comparison of reconstruction algorithms for images from sparse-aperture systems,” in [Proc. SPIE, Image Reconstruction from Incomplete Data II], 4792(1), 1–8 (2002). [10] Tikhonov A. N. and Arsenin V. Y., [Solutions of Ill-posed Problems], V. H. Winston & Sons: Washington, DC (1977). [11] Fienup, J., “Refined wiener-helstrom image reconstruction,” Annual Meeting of the Optical Society of America, Long Beach, CA (October 18, 2001). [12] G. I. Vasilenko and A. M. Taratorin, [Image Recovery [in Russian]], Moscow (1986). [13] Fienup, J., “Invariant error metrics for image reconstruction,” Applied Optics, No 32 36, 8352–57 (1997). [14] Coffin, D., “Raw digital photo decoding,” http://www.cybercom.net/∼dcoffin/dcraw ((Referred 01.01.2009)).