Supplementary Material: Proofs Laplacian PCA and Its Applications Deli Zhao Zhouchen Lin Xiaoou Tang Visual Computing Group, Microsoft Research Asia, Beijing, China {i-dezhao,zhoulin,xitang}@microsoft.com

1. Proof of Theorem 1 ˜ i be Xi L ˜ i = Pi Di QT . Then the left singular matrix Pi Theorem 1. Let the d-truncated SVD of the tall-skinny matrix Xi L i −1

is the local projection matrix Ui , and the local coordinates Yi can be derived by Yi = (Qi Di )T Wi 2 . ˜ i , it is not hard to see that Xi L ˜ i (Xi L ˜ i )T = Pi D2 PT . Thus the d columns of Proof. From the d-truncated SVD of Xi L i i i Pi span the principal subspace of Sl , which is the projection matrix Ui we expect. From the expressions of Li and Hw , we 1 1 ˜ i = Hw W 2 . So we have UT Xi L ˜ i = UT Xi Hw W 2 = Di QT . Thus, Theorem 1 follows by can write Li = Hw Wi and L i

i

−1

i

i

i

checking Yi = UTi Xi Hw = (Qi Di )T Wi 2 . ¤

2. Proof of Theorem 2

Lemma 1. If AT is a tall-skinny matrix and AAT is invertible, then A† = AT (AAT )−1 . Proof. This can be found in [11]. ¤ 1

˜ i (Q ˜TQ ˜ i )−1 Q ˜ T )HT , where Q ˜ i = W− 2 Qi . Theorem 2. Zi = Hw (I − Q w i i i ˜ †Y ˜ Proof. With Zi given in equation (20), noticing that (I − Y i i ) is a symmetric and idempotent matrix [11], Zi can be written as T ˜ †Y ˜ Zi = Hw (I − Y (1) i i )Hw . 1

˜ i = Di QT W− 2 . By Lemma 1, we get that By Theorem 1, we know that Y i i ˜ i (Q ˜TQ ˜† = Q ˜ i )−1 D−1 , Y i i i

(2)

1

˜ i = W− 2 Qi . So, we can write where we apply Q i

Thus we obtain

˜ †Y ˜ ˜ ˜ T ˜ −1 Q ˜T. Y i i i = Qi (Qi Qi )

(3)

˜ i (Q ˜TQ ˜ i )−1 Q ˜ T )HT . Zi = Hw (I − Q i i w

(4)

¤

3. Proof of Proposition 1 Proposition 1. LPCA-based tangential maps coincides with the LTSA algorithm if Wi = I. Proof. Substituting Wi = I into Zi in Theorem 2 yields Zi = H(I−Qi QTi )H, where we apply the equality (I−Qi QTi )2 = ˜ i = H and hence (Xi L ˜ i )T Xi L ˜ i = (Xi H)T Xi H. It is easy to know that He = 0. I − Qi QTi . In addition, we also have L T T Therefore, (Xi H) Xi He = 0. Thus, e is an eigenvector of (Xi H) Xi H corresponding to the eigenvalue 0. We know that Qi is the matrix consisting of the eigenvectors associated with the d largest eigenvalues of (Xi H)T Xi H. Thus we have QTi e = 0, because for symmetric matrices, eigenvectors associated with different eigenvalues are orthogonal. So, it is straightforward to get that µ ¶ µ ¶ ¡ ¢ 1 1 T T T T HQi Qi H = I − ee Qi Qi I− ee = Qi QTi . (5) K +1 K +1 Thus, we can rewrite Zi as Zi = H − Qi QTi = I − where

1 eeT − Qi QTi = I − Gi GTi , K +1

h √ i Gi = e/ K + 1 Qi .

Thus, we have the exact form of local geometry matrix used in LTSA for alignment.¤

(6)

(7)