Multiple-Layer Neural Network Applied to Phase Gradient Recovery from Fringe Pattern

1 2 3 4

Weiguang Ding School of Engineering Science Simon Fraser University [email protected]

5

Abstract

6 7 8 9 10 11 12 13 14

In kinesiology research, fringe projection profilometry is used to measure the surface shape and profile of ex-vivo beating animal heart. Deformation of projected fringe pattern will be caused by non-flat shape of surface and thus used to reconstruct the surface. In this course project, multiple-layer neural network (MLNN) is used to recover the gradient information of the surface as an intermediate step of surface reconstruction. The MLNN is trained by the fringe intensity pattern and phase gradient information extracted from synthetic data set. Various evaluation experiments are made on both parameters of MLNN and the properties of synthetic data set.

15 16 17 18 19 20 21 22 23

1

In tro du c ti o n

B a c k g r o u n d : Surface reconstruction of ex-vivo beating animal heart is necessary in some kinesiology researches. Fringe projection profilometry provides a powerful tool to use non-contact method to measure the shape and profile of moving surface. In this system, collimated fringes (usually with sinusoidal intensity pattern) will be projected onto the target surface. Cameras would be placed from a different view angle. Deformation of fringe pattern will appear in the captured images and surface shape can be recovered from it. Fig. 1 illustrates the set-up of fringe projection profilometry system and gives an example of fringe. object camera computer projector

24 25

(a)

(b)

26 27

Figure 1: (a) Illustration of fringe projection profilometry (cited from [1]); (b) is an example of acquired fringe image

28 29 30 31 32

Related works: Fourier Transform Profilometry (FTP) [2] considers the problem as a modulation and demodulation process, which can be solved by analysis on frequency domain. Similar to FTP, methods including wavelet based method[3], phase-locked method[4] are also used in this application. All these methods take ‘global’ view of this problem and try to find the mapping of the whole fringe image and shape of the object.

33 34

In contrast to the above mentioned to other methods, multiple-layer neural network has been proposed to consider this problem ‘locally’ by Cuevas et al. [5]. This course project uses the

35

idea from [5], while have differences in implementation.

36 37

Description of the problem: Fig. 1 (b) shows an example of acquired fringe image. Intensity of pixel , in this image can be expressed as Fourier series. ,

38

,

2

2

,

(1)

39 40 41 42 43 44 45

Where the projected fringe pattern is represent by term 2 2 , and the surface shape information is contained in term , . To recover the surface, mapping from , to , is desired, and this need to be solved from the equation (1). The problem is that , is a ‘global’ property which does not only rely on the local fringe pattern information. Gradient of , , however, can be determined without knowing information in pixels outside of the window. If the phase gradient can be acquired, the surface reconstruction can be done afterwards.

46 47 48 49 50 51

In this course project, finding the relationship between gradient of , and a local window at pixel , is considered to be a regression problem. Efforts on training multiple-layer neural network (MLNN) to build this mapping are made. Also, the algorithm is evaluated with various experiments.

52 53 54

The multiple-layer neural network (MLNN) is used to solve regression problems which are hard to find an explicit model, which is suitable for the mapping between local fringe pattern and phase gradient value. Fig. 2 illustrates the input and output of MLNN in this application.

55 56

Figure 2: Input and output of MLNN (Fig. cited from [5])

2

M u lt i pl e - L ay e r N eur a l N et w o rk

57 58 59 60 61 62 63 64

The input of the MLNN is the intensity value of every pixel inside a local window. The output of the MLNN is the x and y direction phase gradient. For example, if the local windows size is chosen to be 5x5, the MLNN will have 25 inputs for each pixel on the fringe image. The 2 outputs of the MLNN is the x and y direction phase gradient at the corresponding pixel. In this application, a 2-layer MLNN is used. Parameters of the MLNN are discussed in experiment section.

65 66 67 68 69 70

Synthetic data is used in these experiments because of unavailability of real data. Training data and test data are extracted from different surfaces with similar shape and are illustrated in Fig. 3. Variations in fringe direction, wavelength, noise, illumination nonuniformity (IN) and test data surface shape will be made in different specific experiments. If unspecified, the fringe image is clean (without noise and illumination nouniformity), has fixed direction and 20-pixel wavelength sinusoidal fringe on the image. (e) (f) (a) (b)

3

E x p e r i m en t s

(g) 71 72 73 74 75

(h)

(d) (c) Figure 3: example of data used in MLNN training. (a) is the training fringe image; (b) is the training phase surface; (c) is the training phase gradient in vertical direction; (d) is the training phase gradient in horizontal direction; (e) is the test fringe image; (f) is the test phase surface; (g) is the test phase gradient in vertical direction; (h) is the test phase gradient in horizontal direction.

76

For the MLNN implementation, Netlab [6] package is used with slightly modification.

77 78 79 80 81

In the following experiments, both parameters of the neural network and properties of the image are considered. The MLNN method is also compared with the Fourier Transform Profilometry (FTP) method. Due to large amount of tunable parameters and properties, the following experiments are far from complete. Parameters and properties that have not been experimented will be briefly discussed.

82 83 84

3.1 3.1.1

Experiments on MLNN Parameters Tu n a b l e p a r a m e t e r s w i t h o u t e x p e r i m e n t s

85 86 87 88 89 90 91 92 93 94

Optimization method in following experiments is scaled conjugate gradient descent (SCG). However, determination of the optimal optimization method needs more comprehensive evaluations. The termination condition for the training process using SCG optimization is 4000 iterations over the whole training data. Activation function of hidden layer is set as the hyperbolic tangent function tanh x . Activation function of output layer is chosen as linear function, due to the regression problem [6].While logistic sigmoid function is possible for hidden layer, and sigmoid and softmax function are possible for output layer, the evaluation of their performance is left as a future work.

95 96 97 98 99

5 fold cross validation is done on MLNN with different number of hidden neurons. Here, a 5x5 local window is fixed, which means that the number of input is 25. The error-iteration plots in Fig. 4 (a) and error-number of neurons plots in Fig. 4(b) shows that number of hidden neurons does NOT have a significant influence on the accuracy of the algorithm.

3.1.2

100 101

E x p e r i m e n t s o n d i f f e re n t n u m b e r o f h i d d e n n e u ro n s

(a)

(b)

102 103 104 105 106 107 108

3.1.3

109 110 111 112

Choice of local window size is a tradeoff between information amount and ‘locality’. 5 fold cross validation are also done for MLNN with local window size. Square local windows with size from 5 to 13 are tested and shown in Fig. 4 (b) and (d). In the 5 to 13 range, large window size will result in small errors. However, due to consideration of execution time,

(c) (d) Figure 4: cross validation performance of MLNN. (a) is the error-iteration plots of trained MLNNs with different number of hidden neurons; (b) is the error-iteration plots of trained MLNNs with different number of hidden neurons; (c) and (d) the error comparisons after 4000 iterations E x p e r i m e n t s o n d i f f e re n t l o c a l w i n d o w s i z e

113 114 115 116

window size is fixed to be 5x5 in following experiments. MLNN with different window size will be tested with more experiment in the future. 3.2

Experiments on Data Properties

117 118 119 120 121

Figure 5: Learning result of clean test and training fringe image. (a) and (b) is the target vertical and horizontal phase gradient respectively; (b) and (d)is the vertical and horizontal phase gradient calculated by trained MLNN respectively

122 123 124 125 126

Fig. 5 shows the learning result using ‘clean’ fringe image as the training and test input. Clean means no illumination nonuniformity (IN) and no noise on the fringe image. The value of the MLNN output, the phase gradient, lies in [-0.26 0.26]. In the following experiments, root mean square (RMS) error is used. The RMS error corresponding to results shown in Fig. 5 is 0.012.

127 128 129 130 131

The data has many properties, which cannot be evaluated completely in this report. For example, influence of shape variance and fringe pattern other than sinusoidal fringe are not evaluated in this course project.

132 133

In this experiment, speckle noise and illumination nonuniformity (IN) are added to the fringe image to make it dirty. Fig. 2 (e) is an example of dirty fringe image.

134 135 136 137 138 139 140

First experiment use clean training data and dirty test data with various noise and IN levels. As shown in Fig. 6 (a), the algorithm is sensitive to both IN and noise, and especially very sensitive to noise. Fig. 6 (b) shows the errors of MLNN trained by ‘dirty’ training data which have same noise and IN level with test data. The algorithm is much less sensitive to IN, but still quite sensitive to noise. For comparison, the Fourier Transform Profilometry (FTP) method is also evaluated and shown in Fig. 6 (c). Different from MLNN method, FTP method is more sensitive to IN than noise.

141 142 143 144 145 146

Analysis of performance different between MLNN and FTP: In FTP method, after Fourier transform of the fringe image, if the frequency spectrum of useful information overlaps with illumination spectrum, large error would appear. MLNN is able to compensate for the constant illumination pattern, but couldn’t get accurate prediction at the presence of random noise. Therefore, if MLNN is used to process real image with considerable noise, proper denoising method need to be used.

147 148

3.2.2

149 150 151 152 153

Fringe direction and wavelength changes are also considered. In this experiment, input of test data, the fringe images are modified such that it has different fringe wavelength and fringe direction. The result is shown in Fig. 6 (d). It shows that large direction and wavelength difference will result in large error. However, this information is not enough to evaluate local performance. Comparison in finer scale will be done in the future.

3.2.1

Experiments on noise and illumination

Experiment on fringe direction and wavelength

154 155 156

(a)

(b)

(c)

(d)

157 158 159 160 161 162 163 164

4

165 166 167 168 169 170 171 172 173 174 175 176 177

In this course project, multiple-layer neural network (MLNN) is applied to recover phase gradient in fringe projection profilometry techniques. The mapping between local fringe pattern and phase gradient is found by training a MLNN. The inputs of the MLNN are the pixel values in the local window in fringe image and the outputs are the x and y phase direction gradients. Various tests on both parameters of the MLNN and properties of data are experimented. MLNN performance is not significant related to number of hidden neurons give fixed number of inputs. MLNN has higher accuracy with larger size local window for input under certain range, but due to time consideration, performance test of MLNN with large window size on dirty data set is left as a future work. MLNN is sensitive to noise and less sensitive to illumination nonuniformity (IN), which implies that proper denoising method need to be chosen in real application. If direction or wavelength difference in test data is large, the algorithm will have large error, but error with small changes directions and wavelength need to be included in future work.

178

References

179 180

[1] S. Gorthi and P. Rastogi, Fringe projection techniques: whither we are?. Opt Laser Eng, 48 (2010), pp. 133–140.

181 182

[2] M. Takeda, K. Mutoh, Fourier transform profilometry for the automatic measurement of 3-D object shapes., Appl. Opt. 22 (24) (1983) 3977–3982.

183 184

[3] J. Zhong, J. Weng, Spatial carrier-fringe pattern analysis by means of wavelet transform: Wavelet transform profilometry, Appl. Opt. 43 (26) (2004) 4993–4998.

185 186

[4] M. A. Gdeisat, D. R. Burton, M. J. Lalor, Fringe-pattern demodulation using an iterative linear digital phase locked loop algorithm, Opt. Laser Eng. 43 (7) (2005) 31–39.

187 188

[5] F.J. Cuevas, M. Servin and O.N. Stavrodis, et al. Multilayer neural network applied to phase and depth recovery from fringe patterns. Opt. Commun., 181 (2000), pp. 239–259.

189

[6] Nabney I. NETLAB: algorithms for pattern recognition. London: Springer; 2001.

Figure 6: error images of experiments on data set properties. (a) experiment on clean training and dirty test data; (b) experiment on same level dirty training and test data; (c) experiment of FTP method on dirty test data; (d) experiment on data with different fringe direction and wavelength

Co n c l usio n

Multiple-Layer Neural Network Applied to Phase ... - Semantic Scholar

collimated fringes (usually with sinusoidal intensity pattern) will be projected onto the target. 20 surface. Cameras would be placed from a different view angle. Deformation of fringe pattern. 21 will appear in the captured images and surface shape can be recovered from it. Fig. 1. 22 illustrates the set-up of fringe projection ...

281KB Sizes 0 Downloads 339 Views

Recommend Documents

Genetically Evolving Optimal Neural Networks - Semantic Scholar
Nov 20, 2005 - URL citeseer.ist.psu.edu/curran02applying.html. [4] B.-T. Zhang, H. Mühlenbein, Evolving optimal neural networks using genetic algorithms.

Vectorial Phase Retrieval for Linear ... - Semantic Scholar
Sep 19, 2011 - and field-enhancement high harmonic generation (HHG). [13] have not yet been fully .... alternative solution method. The compact support con- ... calculating the relative change in the pulse's energy when using Xр!Ю (which ...

Genetically Evolving Optimal Neural Networks - Semantic Scholar
Nov 20, 2005 - Genetic algorithms seem a natural fit for this type .... For approaches that address network architecture, one termination criteria that has been.

EEG neural oscillatory dynamics reveal semantic ... - Semantic Scholar
Jul 14, 2015 - reflect mainly “domain general” conflict processing mechanisms, instead of ... mented by domain general or domain specific neural (oscillatory) ...

EEG neural oscillatory dynamics reveal semantic ... - Semantic Scholar
Jul 14, 2015 - Accepted: 01 June 2015 ... unconscious conflict can activate the “conflict monitoring system” in ... seem to suggest that there are conflict-specific control networks for ... 1.1 software package (Psychology Software Tools, Pittsbu

A Novel Three-Phase Algorithm for RBF Neural Network Center ...
Network Center Selection. Dae-Won Lee and Jaewook Lee. Department of Industrial Engineering,. Pohang University of Science and Technology,. Pohang ...

Energy Separation Algorithms Applied to Sonar Data - Semantic Scholar
The analysis of the accelerometer data was used to identify frequencies associated with engine and propeller rotation. Frequency components not harmonically ...

Energy Separation Algorithms Applied to Sonar Data - Semantic Scholar
The analysis of the accelerometer data was used to identify frequencies associated with engine and propeller rotation. Frequency components not harmonically ...

The Digital Watermarking Techniques Applied to ... - Semantic Scholar
May 16, 2012 - Introduction. Power supply in the 21st century is facing more and more challenges, e.g., environment protection, energy shortage etc, such that the techniques related to power supply imminently need to be promoted. Complying with the d

Generalizing Unweighted Network Measures ... - Semantic Scholar
real-world, weighted, social networks using one of our generalized measures: the C-degree ... ber of edges between its neighbors and the number of all possible.

Generalizing Unweighted Network Measures ... - Semantic Scholar
dyadicity, and the heterophilicity. As a case study, we analyze four real-world, weighted, social networks using one of our generalized measures: the C-degree.

Social Network Structure, Segregation, and ... - Semantic Scholar
Jun 29, 2006 - keep in touch but have nothing in common with may know none of your current friends. .... that a group with a more random social network will have higher employment .... is if kJ = 4 job 8 is connected to jobs 6, 7, 9, and 10.

Correlations in End-to-End Network Metrics ... - Semantic Scholar
and services running on top of the network infrastructure. Since changes ... ROUTE. CAPACITY. Fig. 1. Correlations in different e2e network metrics along with.

On fair network cache allocation to content providers - Semantic Scholar
Apr 12, 2016 - theoretic rules outperform in terms of content access latency the naive cache ... With the advent of broadband and social networks, the In-.

On fair network cache allocation to content providers - Semantic Scholar
Available online 12 April 2016. Keywords: In-network caching ..... (i.e., possibly offering different prices to different classes) would be easily considered .... x ∈ Rn. + containing the cache space portion to allocate to each CP. (i.e., the value

Two-phase, Switching, Change-point regressions ... - Semantic Scholar
of the American Statistical Association 67(338): 306 – 310 ... Bayesian and Non-Bayesian analysis of ... Fitting bent lines to data, with applications to allometry. J.

Bright phase-stable broadband fiber-based source ... - Semantic Scholar
toward the practical application of these ideas, many chal- lenges remain. ... 100 Hz and limited spectral range [11] made the overall performance demonstrated in .... produced are measured using a polarization analyzer and single-photon ...

Low Phase Noise Fully Integrated VCO - Semantic Scholar
performance phase locked systems such as frequency synthesizers used in .... [12] 'Analysis and design of analog integrated circuits' by Gray and. Mayer, John ...

Low Phase Noise Fully Integrated VCO - Semantic Scholar
On-chip CMOS bandgap Bias Network is implemented using large ... MEM tunable capacitors, Bandgap Bias Network ... J.N.E.College, Aurangabad. L ...

A two-phase growth strategy in cultured neuronal ... - Semantic Scholar
Oct 27, 2004 - developing neuronal networks (see Connors and. Regehr, 1996; Mainen and Sejnowski, 1996; Sporns et al., 2000, 2002). At the early phase of neuronal development functional requirements are dominant: minimizing time to the formation of s

Semantic Role Labeling with Neural Network Factors - Dipanjan Das
‡Department of Computer Science and Engineering, University of Washington. † Google ... coming close to the previous best published results on the other two.

Adaptive Incremental Learning in Neural Networks - Semantic Scholar
International Conference on Adaptive and Intelligent Systems, 2009 ... There exit many neural models that are theoretically based on incremental (i.e., online,.