SWILA NOTES

15

2.3. The matrix representation of a linear transformation. In Proposition 2.2.3, we showed that we can identify linear transformations Fm → Fl with l × m matrices. Given a linear transformation T : Fm → Fl , we constructed a matrix A ∈ Ml×m such that T (x) = Ax for all x ∈ Fm . In particular, if Sm , Sl are the standard bases for Fm , Fl , respectively, [T (x)]Sl = A[x]Sm

for all x ∈ Fm .

The question is whether we can define a similar correspondence with more general linear transformations V → W . More specifically, given a linear transformation T : V → W , we wish to find a matrix that transforms coordinates of a vector x ∈ V into coordinates of T (x) ∈ W . In this section, we will show that we can do this if V and W have finite bases. Fix an ordered basis B = (v1 , . . . , vn ) for a vector space V and a vector x ∈ V . If we write x = a1 v1 + · · · + an vn , recall that we define the coordinates of x with respect to B to be   a1   [x]B =  ...  . an This defines an identification of Fn with V . (See Remark 1.4.8 and Definition 1.4.9.) Recall S = {e1 , . . . , em } is the standard basis for Fm . Given a matrix A ∈ Ml×m (F), note Aei equals the ith column of A. This motivates the following definition: Definition 2.3.1. Let V and W be vector spaces over F. Let B = {e1 , . . . , em } be a basis for V , and C = {f1 , . . . , fn } a basis for W . Fix a linear transformation T : V → W . We define the matrix representation of T with respect to B and C to be the n × m matrix   | | [T ]C,B =  [T (e1 )]C · · · [T (em )]C  . | | Here, for each i = 1, . . . , m, [T (ei )]C is the column vector of the coordinates of T (ei ) with respect to C. In the case V = W and B = C, we simply write [T ]B for [T ]B,B . We will typically be interested in the latter case. More specifically, given a vector space V and a linear operator T : V → V , we will be looking for an ordered basis B such that the matrix representation [T ]B of T for which the B is “nice”. Proposition 2.3.2. Fix vector spaces V , W with bases B = {e1 , . . . , em }, C = {f1 , . . . , fn }, respectively. Then [T (x)]C = [T ]C,B [x]B for all x ∈ V. In other words, the n × m matrix [T ]C,B transforms coordinates of a vector x ∈ V with respect to B into the coordinates of T (x) ∈ W with respect to C. Proof. The idea of this proof is to prove the identity first when x equals of the vi , and then identity should hold for all x by linearity. This proof strongly uses that the coordinate maps [ ]B : V → Fm , [ ]C : W → Fn are linear.

16

DEREK JUNG

For each i = 1, . . . , m, [T ]C,B ei equals the ith column of [T ]C,B , which by construction is [T (vi )]C . But note ei = [vi ]B . This implies [T (vi )]C = [T ]C,B [vi ]B . It then follows from linearity of T and the coordinate maps, and the definition of basis, that [T (x)]C = [T ]C,B [x]B

for all x ∈ V. 

Example 2.3.3. Let S = {e1 , . . . , en } be the standard ordered basis for Rn . Define the linear transformation T : Rn → Rn by T (a1 , . . . , an ) =

n X

kak .

k=1

As T (ek ) = kek for each k = 1, . . . , n, the matrix representation of T with respect to S is given by the diagonal n × n matrix   1 0 ··· 0  0 2 0    [T ]S =  . ..  . . . .  . . .  0 0 ···

n

We can use Proposition 2.3.2 to prove the following theorem: Theorem 2.3.4. Fix vector spaces V, W, X with finite bases A, B, C, respectively. If S : V → W , T : W → X are linear transformations, then [T ◦ S]C,A = [T ]C,B [S]B,A . Proof. (Sketch) Assume A has m elements and C has n elements. Then [T ◦ S]C,A is an n × m matrix, so [T ◦ S]C,A may be viewed as a linear transformation from Fm to Fn by Proposition 2.2.3. On the hand, [T ]C,B [S]B,A may also be viewed as a linear transformation from Fm → Fn . It follows from linearity that if [T ◦ S]C,A (ei ) = [T ]C,B [S]B,A (ei ) for all 1 ≤ i ≤ m, then then they agree on all of Fm . This fact follows from the formula written in Proposition 2.3.2.  2.4. Change of coordinate matrices. In this section, we will introduce change of coordinate matrices and obtain equivalent matrix representations of linear transformations. Definition 2.4.1. Let B = {e1 , . . . , en }, C = {f1 , . . . , fm } be two ordered bases for a vector space V . We define the change of coordinate matrix of B to C to be   | | | [Id : V → V ]C,B =  [e1 ]C [e2 ]C · · · [en ]C  | | | For all v ∈ V , we have [v]C = [Id]C,B [v]B . In other words, [Id]C,B transforms (by left multiplication) the coordinates of a vector v in B to the coordinates of v in C.

SWILA NOTES

17

Remark 2.4.2. We will show later in Theorem 2.6.16 that if V has a finite basis B, then any other basis has the same number of vectors as B. Applying Theorem 2.3.4 to the identity T = Id ◦ T ◦ Id, we have the following theorem: Theorem 2.4.3. Let B, C be finite bases of a vector space V . If T : V → V is a linear operator, then [T ]C = [Id]C,B [T ]B [Id]B,C . We will use this theorem and the following corollary throughout. Let S = {e1 , . . . , en } be the standard basis for Fn . From Proposition 2.2.3, recall that a linear operator T : Fn → Fn is of the form T (x) = AT x, where the ith column of AT is T (ei ). Corollary 2.4.4. Fix a basis B = {f1 , . . . , fn } for Fn . If T : Fn → Fn , T (x) = Ax, is a linear operator, then S −1 = [Id]B,S and A = S[T ]B S −1 , where S is the n × n matrix with its ith column being fi . Proof. Observe 

S = [Id]S,B

|  f1 · · · = |

 | fn  . |

As we have S −1

IdMn (F) = [Id]S,S = [Id]S,B [Id]B,S , = [Id]B,S . The corollary then follows from the previous theorem.



Example 2.4.5. Let  1 1 ··· 1  0 1 ··· 1    A= . ..  ∈ Mn (F). . . .  . . .  0 0 ··· 1 We can use the previous corollary to easily find the inverse of A. Define      1 1 1  1   1  0            v1 =  0  , v2 =  0  , . . . , vn =  1  ..   ..  ..   .  .   .  

0

0

    .  

1

Then B := {v1 , . . . , vn } can be seen by induction to be a basis for Fn . Let S = {e1 , . . . , en } be the standard basis for Fn . Observe A = [Id]S,B . By the previous corollary, 

A−1 = [Id]B,S

| =  [e1 ]B · · · |

|



[en ]B  . |

18

DEREK JUNG

Then note e1 = v1 and ei = vi − vi−1 for  1   0  −1 A =  ...   0 0

each i ≥ 2. We may conclude  −1 0 · · · 0 ..  1 −1 .   . .. .. . 0  .  0 1 −1  0 ··· 0 1

We conclude this section with the following definition. Definition 2.4.6. We say that a matrix A ∈ Mn (F) is similar to a matrix B ∈ Mn (F) if there exists an invertible n × n matrix S such that A = SBS −1 . From Corollary 2.4.4, we see that two squares matrices are similar if and only if they represent the same linear operator on Fn , but with respect to possibly different bases. 2.5. The trace of a matrix. Definition 2.5.1. We define the trace tr : Mn (F) → F by tr(aij ) := a11 + a22 + · · · + ann . Note that tr is a linear transformation. Note if F = R and A ∈ Mn (R), tr(At ) = tr(A). If F = C and B ∈ Mn (C), tr(A∗ ) = tr(A). A tedious computation gives us the following lemma (without proof here): Lemma 2.5.2. (Invariance of trace) If A ∈ Ml×m (F) and B ∈ Mm×l (F), then tr(BA) = tr(AB). Remark 2.5.3. Observe 

1 1 0 0



0 0 0 1



 =

0 1 0 0





    0 0 1 1 0 0 = . 0 1 0 0 0 0 This is an example of matrices A, B for which tr(AB) = tr(BA), even though AB 6= BA. That’s what makes the previous lemma so remarkable: despite the fact that Mn (F) is noncommutative in general, the trace functional behaves as it were. For fun, consider what these matrices do as operators on R2 to figure out why exactly one product represents the zero transformation. Remark 2.5.4. Lemma 2.5.2 states that tr(AB) = tr(BA) for matrices A, B of appropriate sizes. Equivalently, the trace linear functional is invariant under cyclic permutations of matrices. However, it would be false to state that the trace totally ignores the noncommutativity of Mn (F). For example, let A, B ∈ M2 (R) be given by     1 2 2 3 A= , B= . 3 4 4 5

SWILA NOTES

19

One can compute  AABB =

392 517 856 1129



 and ABAB =

386 507 858 1127

 .

Hence, tr(AABB) = 1521 6= 1513 = tr(ABAB). We have considered the trace of square matrices. We will now show that we can define the trace of linear operators on vector spaces with finite bases. Proposition 2.5.5. Let T : V → V be a linear operator. If B, C are two finite bases of V , then tr([T ]B ) = tr([T ]C ). Proof. By Theorem 2.4.3, [T ]B = [Id]B,C [T ]C [Id]C,B . By Lemma 2.5.2, tr([T ]B ) = tr (([Id]B,C [T ]C )[Id]C,B ) = tr ([Id]C,B ([Id]B,C [T ]C )) = tr ([T ]C ) . We used that [Id]C,B [Id]B,C = [Id]C .  This enables us to make the following definition: Definition 2.5.6. Let T : V → V be a linear operator. If V has a finite basis B, we define tr(T ) := tr([T ]B ). Note that the previous proposition ensures that is well-defined. More precisely, it doesn’t depend on the choice of the finite basis since taking another matrix representation of T would result in the same trace. Remark 2.5.7. In the previous proposition, note we have not shown that B necessarily has the same number of elements as C. This is the content of the next section. We will later show that the trace defines an inner product on Mn (C) and Mn (R) (see Example 6.1.6 and Remark 7.4.8). 2.6. Dimension and isomorphism. Remark 2.6.1. Recall that a map f : V → W is surjective (or onto) if f (V ) = W , and is injective (or one-to-one) if f (v1 ) = f (v2 )

⇒

v1 = v2 .

We say that a map f : V → W is bijective if it is injective and surjective. Observe that a linear transformation T : V → W is injective if and only if ker(T ) := {v ∈ V : T (v) = 0} = {0}. We will use this observation throughout these notes.

20

DEREK JUNG

Random Thought 2.6.2. Oh dear, John, by Nicholas Sparks In the best seller Dear John, we journeyed with Savannah as she fell in love with a soldier John and experienced the death of her husband Tom. Now married to John with kids, Savannah must overcome yet another challenge: Dealing with John’s lame jokes. John: Hey Savannah Banana. Guess what? Savannah: What John? John: Why did the soldier no longer see one-to-one with his pompous, recently promoted general? Savannah: Why John?... John: Because he no longer had a trivial colonel. Savannah: Oh dear, John. Definition 2.6.3. A linear transformation T : V → W is an isomorphism if there exists a linear transformation S : W → V such that T S = 1W and ST = 1V . We say vector spaces V and W are isomorphic if there exists an isomorphism T : V → W . Here, 1V , 1W are the identity transformations on V, W , respectively. Routine algebraic manipulations give us the following lemma (without proof): Lemma 2.6.4. A linear transformation T : V → W is an isomorphism if and only if it is bijective. Remark 2.6.5. This lemma tells us that a map between vector spaces being linear gives us possibly additional structure on the map. This theme comes up commonly in math. For example, all functions f : C → C that are once differentiable are actually infinitely differentiable and have locally convergent power series at each point. This is not true for functions g : R → R. This stems from the fact that one can approach a complex number from a myriad of directions in C while one can only approach a real number in essentially two ways in R. The fact that a linear transformation is additive and allows us to pull out scalars must be a rather strong condition on our functions. Example 2.6.6. Fix a field F, and define Σc (F) to be the collection of all compactly supported F-valued sequences. In other words, for all (ai ) ∈ Σc (F), there exists N such that for all i ≥ N , ai = 0. Observe that Σc (F) has a basis {ei }i∈N , where ei is the sequence for which its ith term is 1 and every other term is 0. Then we have an isomorphism from Σc (F) to the space F[x] of polynomials with coefficients in F, induced by ei 7→ xi−1 and extended linearly (compare with Example 1.4.5). Definition 2.6.7. (Characteristic of a field F) Fix a field F. If n · 1F is nonzero for all n ∈ N, we define the characteristic of F to be 0. Otherwise, we define the characteristic of F to be the smallest n ∈ N such that n · 1F = 0. Remark 2.6.8. The characteristic of a field F is actually the positive generator for the kernel of the map ι : Z → F defined by ι(n) := n · 1F . It’s an exercise left to the awesome reader to show that the characteristic of a field F is either 0 or a prime number. (Hint: Use the Fundamental Theorem of Arithmetic.) Example 2.6.9. Fix a positive prime p. We can define an equivalence relation on Z by proudly declaring a ≡ b if b − a is divisible by p. Given a ∈ Z, the equivalence class of a is

SWILA NOTES

21

[a]p := a + pZ. We define Z/pZ (sometimes written Zp ) to be the collection of equivalence classes of Z with respect to this relation. So, Z/pZ = {[0]p , [1]p , . . . , [p − 1]p }. We can define addition and multiplication on Z/pZ as: [a]p + [b]p := [a + b]p ,

[a]p · [b]p := [ab]p .

These definitions can be shown to be well-defined. This makes Z/pZ into a field of characteristic p. It can be shown that all finite fields are of order pn for some n ∈ N and prime p > 0. Random Thought 2.6.10. Best name for a math acapella group: InVerse. It could even specialize in oldies and its motto could be “Bringing it back!”. Someone please create this for me; it’s about time someone injected some fun onto acapella groups. Theorem 2.6.11. Suppose F is of characteristic 0. If Fm is isomorphic to Fn as vector spaces over F, then m = n. In particular, the result holds when F = Q, R, or C. Proof. This theorem will follow by use of the trace. Suppose we have L : Fm → Fn and K : Fn → Fm such that L ◦ K = 1Fn and K ◦ L = 1Fm . Proposition 2.2.3 showed that we may identify L and K with matrices, and composition with matrix multiplication. Then by Lemma 2.5.2 (for the middle equality), n = tr(1Fn ) = tr(LK) = tr(KL) = tr(1Fm ) = m.  Remark 2.6.12. If F is of characteristic p for some prime p, we may have that m = n in Z/pZ while m 6= n in Z. Corollary 2.6.13. (Invariance of Dimension) Suppose F is of characteristic 0. Suppose B = {e1 , . . . , em } and C = {f1 , . . . , fn } are ordered bases for a vector space V . Then m = n. In particular, this corollary holds when F = Q, R, or C. Proof. From Remark 1.4.8 and Definition 1.4.9, we have the coordinates map ιB : V → Fm by v 7→ [v]B is an isomorphism. Similarly, ιC : V → Fn by v 7→ [v]C is an isomorphism. Thus, n m ιB ◦ ι−1  C : F → F is an isomorphism. Theorem 2.6.11 gives that m = n. Remark 2.6.14. Note that Corollary 2.6.13 applies for when F = R or F = C. An interesting question is whether Corollary 2.6.13 still holds if F is of positive characteristic. And the answer is yes, with a slightly different proof that also works in the case of characteristic 0. I found this proof online by Jack Huizenga, a professor at Penn State [2]. We first prove the following lemma: Lemma 2.6.15. Fix a field F. If v1 , . . . , vm are m linearly independent vectors in Fn , then m ≤ n. Proof. Suppose for contradiction m > n. Define the matrix   | | A =  v1 · · · vm  ∈ Mn×m (F). | |

22

DEREK JUNG

As m > n, there is a nonzero solution to the equation Ax = 0, call it (α1 , . . . , αm )t . Thus, we have α1 v1 + · · · + αm vm = 0, with at least one of the αi nonzero. This is a contradiction.  We can now prove that any two finite bases of V have the same number of elements: Theorem 2.6.16. Let v1 , . . . , vm and w1 , . . . , wn be two bases of a vector space V over F. Then m = n. Proof. Without loss of generality, assume m > n. Since the wi ’s form a basis, we may write vj = a1j w1 + · · · + anj wn ,

j = 1, 2, . . . , m.

The coefficients (a1j , . . . , anj ) of these vectors are m vectors in Fn , so they are linearly independent by the previous lemma. This implies that we may write the coefficient vectors as nontrivial linear combinations of 0. It’s an easy calculation to check that the vectors v1 , . . . , vm must satisfy the same nontrivial linear combination to be 0.  We can now end this section by defining the dimension of a vector space: Definition 2.6.17. Let V be a vector space over a field F. If V has a finite basis, we define the dimension of V to be the order of any basis of V and say that V is finite-dimensional. Otherwise, we say that V is infinite-dimensional. 2.7. Row reduction and the general linear group: Shearing is caring. A matrix J decides to change the names of its rows to pears. After some time, J is despondent over the name change and shears with her companion matrix R. R comforts J, “Don’t diss pear. For that which we call rows by any other name is just as sweet.” Suppose that we have a system of equations: a11 x1 + · · · + a1m xm = b1 a21 x1 + · · · + a2m xm = b2 .. .. . . al1 x1 + · · · + alm xm = bl We can rewrite this system as an equation of matrices:     a11 · · · a1m x1  .. . . ..   ..  =   . . .  .   al1 · · · alm xm

 b1 ..  .  bl

For fixed (aij ) and (b1 , . . . , bl ), we wish to find a solution (x1 , . . . , xn ). It will be much easier to find a solution if (aij ) is in a simple form or we know its inverse. That’s the content of this section. Definition 2.7.1. A matrix is in row echelon form if • The first nonzero entry in each row (if any) is normalized to be 1. This is called the leading 1 for the row. • The leading 1s (if any) appear in echelon form, i.e., as we move down the rows the leading 1s will appear farther to the right. A matrix in row echelon form is said to be in row reduced echelon form if the entries above and below each leading 1 consist of zeros.