26/09/2006 12:42:41 PM

Gait Recognition through MPCA plus LDA

Haiping Lu, K.N. Plataniotis and A.N. Venetsanopoulos The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto

Biometrics Symposium 2006

Outline z z z z z z

Motivation Overview of the proposed method MPCA framework Gait recognition through MPCA+LDA Experimental results Conclusions

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

2

1

26/09/2006 12:42:41 PM

Motivation z

z

z

Gait recognition: human identification at a distance in surveillance/monitoring apps Gait (silhouette) sequences: multidimensional (tensor) objects Dimensionality reduction/feature extraction

• PCA: vectorization, break correlation/structure • Directly on tensor representation? z

Objective: direct tensor feature extraction Kostas Plataniotis

BSYM2006, Baltimore, MD

3

Gait sequence as tensor object 3-mode (time)

2-mode (row)

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

4

2

26/09/2006 12:42:41 PM

To apply PCA and/or LDA

Tensor: 128x88x20

Vectorization …... Vector: 225280x1 Kostas Plataniotis

BSYM2006, Baltimore, MD

5

Linear projection

z z

Very high dimensionality Correlation and structure are broken

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

6

3

26/09/2006 12:42:41 PM

Overview of the proposed method z z

z

Input: gait sequences as tensors Algorithm: feature extraction from tensors using multilinear projection Output: features extracted from gait sequences in their natural representation as tensors.

Kostas Plataniotis

BSYM2006, Baltimore, MD

7

Multilinear projection

z z

Operations on lower dimensionality Original correlation and structure are preserved Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

8

4

26/09/2006 12:42:41 PM

The MPCA framework z

Input: M training gait samples

z

Output: multilinear transformation

z

Objective: the projection captures most of the variations Kostas Plataniotis

BSYM2006, Baltimore, MD

9

Definition of tensor variations

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

10

5

26/09/2006 12:42:41 PM

Objective function

z z

No known optimal solution to simultaneously optimize the N matrices Solution – Part I:

• Decompose into N optimization subproblem • Find each maximizing the scatter in each n-mode vector subspace

Kostas Plataniotis

BSYM2006, Baltimore, MD

11

Solution – Part II

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

12

6

26/09/2006 12:42:41 PM

Solution – Part III z

z

z

depends on ,…, , ,…, . Optimization of one projection matrix in one mode depends on the projection matrices in all the other modes No closed form solution and an iterative solution is introduced. Kostas Plataniotis

BSYM2006, Baltimore, MD

13

The (iterative) MPCA algorithm z

z

z

Initialize projection matrices and determine the subspace dimensionality if not given Compute projection matrices one mode by one mode fixing projections in all other modes Repeat until convergence Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

14

7

26/09/2006 12:42:41 PM

Initialization and termination z

Initialization: full projection truncation (FPT)

• Full projection:

for is an identity matrix.

• Eigentensor: • FPT: Keeping the first projection matrix z

columns of the full in n-mode for all n.

Termination: Kostas Plataniotis

BSYM2006, Baltimore, MD

15

Subspace dimensionality determination z

Ratio of variations kept in the n-mode

z

Keep the first mode so that

Kostas Plataniotis

BSYM2006 by Haiping LU

eigentensors in n-

BSYM2006, Baltimore, MD

16

8

26/09/2006 12:42:41 PM

MPCA plus LDA for gait recognition

Kostas Plataniotis

BSYM2006, Baltimore, MD

17

Gait recognition using MPCA+LDA z

z

z

Gait samples: half gait cycles, partitioned through number of foreground pixels Normalization: spatial and temporal interpolation Eigentensor selection by class discriminability

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

18

9

26/09/2006 12:42:41 PM

Experimental data z

USF HumanID “Gait Challenge” data sets v.1.7

• Different conditions: walking surfaces, shoe • • •

types and viewing angles 71 subjects in gallery set: 731 gait samples Seven probe sets Gait sample size:

Kostas Plataniotis

BSYM2006, Baltimore, MD

19

Parameters z

Q=97, 92% of the total variations kept

z

Number of eigenvectors kept in each mode:

z

Number of EigenTensors selected :

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

20

10

26/09/2006 12:42:41 PM

Examples of gait sample and EigenTensorGait (unfolded) A gait sample

Mean gait sample

Three EigenTensorGaits Kostas Plataniotis

BSYM2006, Baltimore, MD

21

Similarity measure z

Feature with subject c:

z

Mahalanobis+angle distance measure

z

Probe sequence p with a gallery sequence g

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

22

11

26/09/2006 12:42:41 PM

CMC Curves

Kostas Plataniotis

23

BSYM2006, Baltimore, MD

Rank 1 comparison Probe BL HMM LTN GEI MPCA-HT MPCA+LDA A(GAL)

79

99

94

100

94

99

B(GBR)

66

89

83

85

76

88

C(GBL)

56

78

78

80

66

83

D(CAR)

29

35

33

30

27

36

E(CBR)

24

29

24

33

36

29

F(CAL)

30

18

17

21

15

21

G(CBL)

10

24

21

29

19

21

Mean

42

53

53

54

48

54

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

24

12

26/09/2006 12:42:41 PM

Rank 5 comparison Probe BL HMM LTN GEI MPCA-HT MPCA+LDA A(GAL)

96

100

99

100

99

100

B(GBR)

81

90

85

85

83

93

C(GBL)

76

90

83

88

81

88

D(CAR)

61

65

65

55

64

71

E(CBR)

55

65

67

55

52

60

F(CAL)

46

60

58

41

53

59

G(CBL)

33

50

48

48

48

60

Mean

64

74

72

67

68

76

Kostas Plataniotis

BSYM2006, Baltimore, MD

25

Conclusions z

z z

z

MPCA framework: multilinear projection for tensors capturing most variations LDA on selected eigentensors for recognition Results: outperform state-of-the-art gait recognition algorithms Future work: extension to other tensor objects and development of other tensor subspace algorithms

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

26

13

26/09/2006 12:42:41 PM

Acknowledgement z

Thank Prof. Sarkar from the University of South Florida (USF) for providing the Gait Challenge data sets.

Kostas Plataniotis

BSYM2006, Baltimore, MD

27

Back up slides

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

28

14

26/09/2006 12:42:41 PM

Notations and basics

z

Vector: lowercase boldface Matrix: uppercase boldface Tensor: calligraphic letter n-mode product:

z

Scalar product:

z z z

Kostas Plataniotis

BSYM2006, Baltimore, MD

29

Notations and basics z

Frobenius norm:

z

Outer product:

z

n-mode unfolding:

• Column vectors of

are n-mode vectors

of Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

30

15

26/09/2006 12:42:41 PM

Unfolding into matrices

1-mode (column) vectors 10x1

Kostas Plataniotis

Unfolded matrix 10x48=10x(8x6)

BSYM2006, Baltimore, MD

31

Notations and basics z

Tensor decomposition (Tucker’s model)

• • z

Equivalent form (sum of rank-1 tensors):

Kostas Plataniotis

BSYM2006 by Haiping LU

is an orthogonal matrix

BSYM2006, Baltimore, MD

32

16

26/09/2006 12:42:41 PM

Multilinear projection in one mode

A 10x8x6

1-Mode Vectors

1-Mode Projection

B(1)T

A x1B(1)T 5x8x6

Rows

5x10

Kostas Plataniotis

BSYM2006 by Haiping LU

BSYM2006, Baltimore, MD

33

17