6

DEREK JUNG

0. Notation We will use the following notation for the entirety of the workshop: (1) F will always denote a field, typically R or C. (2) F[t] denotes the space of polynomials with coefficients in F. (3) I sometimes use interchangeably ⊂ and ⊆ for set containment. (4) Given k ∈ N, Pk (F) denotes the collection of polynomials with coefficients in F and degree at most k. (5) N = {1, 2, 3, . . .} will denote the set of natural numbers. (6) Ml×m (F) denotes the set of l × m matrices with entries in F. (7) Mn (F) := Mn×n (F) denotes the set of n × n matrices with entries in F. (8) (aij )1≤i≤l 1≤j≤m ∈ Ml×m (F) will denote an l × m matrix for which aij ∈ F is the entry in the ith row and the j th column.   a11 a12 · · · a1m  a21 a22 · · · a2m    1≤i≤l (aij )1≤j≤m =  . ..  .. . . .  . . .  . al1

(9) (10) (11) (12) (13) (14) (15) (16) (17) (18)

al2 · · ·

alm

I will typically omit the subscript and superscript of the matrix when they are implied by the context and simply write (aij ). For a linear transformation T : V → W , we write 0 6= T or T 6= 0 to state that T is not the zero linear transformation, i.e., there exists x ∈ V such that T (x) 6= 0. Given vector spaces V , W over a field F, L(V, W ) denotes the set of linear transformations from V to W . I will use different notations for identity transformations and identity matrices. A few of these include: 1, I, and Id. For a polynomial p(t) ∈ F[t], we write 0 6= p(t) or p(t) 6= 0 to state that p(t) is not the zero polynomial. Vector spaces will be assumed to be possibly infinite-dimensional unless noted otherwise. For A = (aij ) ∈ Ml×m (R), the transpose At ∈ Mm×l (R) is defined by At := (aji ). For A = (aij ) ∈ Ml×m (C), the adjoint A∗ ∈ Mm×l (C) is defined by A∗ := (aji ). The adjoint of a linear transformation T : V → W between finite-dimensional vector spaces will be denoted by T ∗ . Given a linear operator T : V → V and x ∈ V , the cyclic subspace generated by x will be denoted by Cx . Ap will denote the companion matrix of a monic polynomial p(t).

SWILA NOTES

7

1. Vector spaces: C’mon, it’s just algebra (except not quite) You can go on all sorts of adventures with Legos. You can sail the seas as a pirate, go back millions of years to walk with the dinosaurs, and blast into outer space with a rocket ship. And there’s all sorts of new pieces now, too. There’s the little one-by-one nub block, a plastic staff, and wigs for your little Lego people. But there’s one main piece that everything is built upon: the traditional two-by-four block. In the first two chapters, we will study vector spaces and linear transformations. These are the building blocks of everything we will study in this workshop, the two-by-four block of linear algebra. Just remember: Everything is awesome, everything is cool when you work as a team. 1.1. Fields. Definition 1.1.1. A field F is a set with two binary operations +, · : F × F → F and distinct elements 0 6= 1 satisfying: For all a, b, c ∈ F, (1) Associativity of +: a + (b + c) = (a + b) + c. (2) Commutativity of +: a + b = b + a. (3) Existence of 0: a + 0 = a. (4) Additive inverses: There exists (−a) ∈ F such that a + (−a) = 0. (5) Associativity of ·: a · (b · c) = (a · b) · c. (6) Commutativity of ·: a · b = b · a. (7) Existence of 1: a · 1 = a. (8) Multiplicative inverses of nonzero elements: If a 6= 0, there exists a−1 ∈ F such that a · a−1 = 1. (9) The Distributive Law: a(b + c) = ab + ac. More abstractly, • (F, +, 0) forms an abelian group, i.e., + satisfies associativity and commutativitiy, and each element has an inverse; • (F \ {0}, ·, 1) forms an abelian group; and • the distributive law holds. We typically omit · when writing multiplication. √ Example 1.1.2. Examples of fields include R, C, Q, and Q[i] := {a + b −1 : a, b ∈ Q}. Here, R denotes the reals, C denotes the complex numbers, and Q denotes the rationals.

8

DEREK JUNG

1.2. Vector spaces. A wonderful property about some sets is that we have a notion of addition and scalar multiplication of elements. For example, we can talk the convergence of a sequence of functions to a limiting function since we can talk about the difference of two functions. Sets for which we have nicely behaving binary operations of addition and scalar multiplication are called vector spaces. Definition 1.2.1. A vector space over a field F is a nonempty set of vectors V with an element 0 ∈ V and operations + : V × V → V and scalar multiplication of vectors satisfying: for all x, y, z ∈ V and α, β ∈ F, (1) Associativity of +: (x + y) + z = x + (y + z). (2) Commutativity of +: x + y = y + x. (3) Existence of 0: x + 0 = x. (4) Existence of additive inverses: There exists −x ∈ V such that x + (−x) = 0. (5) Associativity of scalar multiplication: α(βx) = (αβ)x; (6) Commutativity of scalar multiplication: αx = xα; (7) Multiplication by the unit scalar: 1x = x; (8) The distributive laws: α(x + y) = αx + αy (α + β)x = αx + βx. Note conditions (1)-(4) states that (V, +, 0) forms an abelian group. Random Thought 1.2.2. Kid complaining to parent... Kid: I hate adding! Parent: Hey, have a better additude! Remark 1.2.3. Technically, {0} forms a vector space over any field F. (Check the conditions.) However, throughout these notes, all vector spaces are implicitly assumed to be nonzero, i.e., V 6= {0}. Example 1.2.4. The most important example of a vector space is Fn , where Fn := {(a1 , . . . , an ) : ai ∈ F}. Fn is endowed with the natural vector addition and scalar multiplication.

SWILA NOTES

9

Example 1.2.5. The space of polynomials with F-valued coefficients F[x] forms a vector space over F with the operations: (an xn + · · · + a0 ) + (bn xn + · · · + b0 ) := (an + bn )xn + · · · + (a0 + b0 ) c(an xn + · · · + a0 ) = (can )xn + · · · + ca0 . A useful tool when doing computations with multiple polynomials is assuming they all have the same number of terms by defining some higher coefficients to be zero, if necessary. Example 1.2.6. We say that a function f : R → R is smooth if its k th derivative f (k) (x) exists for all k ∈ N and x ∈ R. The space of real-valued smooth functions C ∞ (R) forms a vector space over R with addition and scalar multiplication defined by: for f, g ∈ C ∞ (R), α ∈ R, (f + g)(x) := f (x) + g(x) (αf )(x) := αf (x). Similarly, the space of real-valued continuous functions C(R) forms a vector space over R with similarly defined addition and scalar multiplication. Example 1.2.7. Given two vector spaces V and W , we can form the product V × W := {(v, w) : v ∈ V

and w ∈ W }.

This becomes a vector space when endowed with the operations of coordinate-wise addition and scalar multiplication: (v1 , w1 ) + (v2 , w2 ) := (v1 + v2 , w1 + w2 ) α(v, w) := (αv, αw). Note that this is a slight abuse of notation as we are using + for addition in three different spaces, and similarly with scalar multiplication. This abuse of notation will be typical throughout these notes to save ink, and it is expected that the reader uses the context to sift through the abuse. Vector spaces share a number of intutitve properties involving their additive and multiplicative identity elements: Proposition 1.2.8. For all x ∈ V , (1) 0x = 0. (2) α0 = 0. (3) (−1)x = −x. (4) If αx = 0, then α = 0 or x = 0. Proof. We only prove (4). (1)-(3) are left as exercises to the reader. If α 6= 0, then α has a multiplicative inverse α−1 . We have 0 = α−1 (αx) = 1x = x.  As mentioned before, a fundamental property of vector spaces is that we can add scalar multiples of vectors together. Remark 1.2.9. A vector space in which we can also multiply vectors is called an algebra.

10

DEREK JUNG

Definition 1.2.10. A linear combination of vectors in a vector space V is an element x ∈ V of the form x = α 1 x1 + α 2 x2 + · · · + α n xn , for some finite collection of vectors x1 , . . . , xn ∈ V and scalars α1 , . . . , αn ∈ F. Remark 1.2.11. Note that we can only consider finite sums of vectors in V . In general vector vector spaces, we have no concept of limits, and hence, infinite series. Definition 1.2.12. Fix a subset S ⊆ V . We define span(S) to be the collection of all linear combinations of vectors of S. This forms a subspace of V (closed under vector addition and scalar multiplication), and we say S spans V if span(S) = V . 1.3. Linear independence. Definition 1.3.1. A collection of vectors {vα }α is linearly independent if: a1 vα1 + · · · + an vαn = 0 for ai ∈ F

=⇒

ai = 0 for all i.

A collection of vectors {vα }α is linearly dependent if it is not linearly independent. It isn’t hard to prove the following results. Proposition 1.3.2. (Characterization of linear dependence) A collection of vectors {x1 , . . . , xn } ⊂ V is linearly dependent if and only if either x1 = 0, or we can find a smallest k ≥ 2 such that xk is a linear combination of x1 , . . . , xk−1 . Corollary 1.3.3. If V = span{x1 , . . . , xn }, then there exists a subset {xi1 , . . . , xik } that forms a basis for V . Proposition 1.3.4. (Steinitz Replacement) Let y1 , . . . , ym ∈ V be linearly independent and V = span{x1 , . . . , xn }. Then m ≤ n and V has a basis of the form y1 , . . . , ym , xi1 , . . . , xil , where l ≤ n − m. In particular, we may complete any linearly independent subset of a finitedimensional vector space to a basis. Here, we say that a vector space is finite-dimensional if it has a finite basis. 1.4. Bases and coordinates. Definition 1.4.1. Let V be a vector space over a field F. An F-basis for V is a linearly independent collection of vectors {vα }α∈A that spans V . We will say {vα }α∈A is a finite basis if A is finite. We simply write basis when the underlying field of scalars is clear from context. Equivalently, each vector in V can be uniquely written as a linear combination of vectors in {vα }α∈A , i.e., for each v ∈ V , there exist unique vectors vα1 , . . . , vαn and scalars a1 , . . . , an ∈ F (depending on v) such that v = a1 vα1 + · · · + an vαn . Remark 1.4.2. The zero vector space is typically defined to have dimension 0. Remark 1.4.3. In this workshop, we will typically only work with vector spaces which have finite bases.

SWILA NOTES

11

Example 1.4.4. Define the standard basis vectors e1 , e2 , . . . , en in Fn by e1 := (1, 0, 0, . . . , 0) e2 := (0, 1, 0, . . . , 0) .. . en := (0, . . . , 0, 1). In general ei = (a1 , . . . , an ), where aj = 0 if i 6= j and ai = 1. It is easy to see that {ei }1≤i≤n forms a finite F-basis for Fn . We will define S := {e1 , . . . , en }. Example 1.4.5. The monomials {xn }n∈N ∪ {1} form an F-basis for F[x]. However, these monomials do not form a basis for the vector space of formal power series F[[x]] := {

∞ X

an xn : an ∈ F}.

n=0

(Why not?) Example 1.4.6. For each 1 ≤ i ≤ l and 1 ≤ j ≤ m, define Eij to the (l × m)-matrix for which the (i, j)th entry is 1 and every other entry is 0. The vector space Ml×m (F) of (l × m)-matrices has {Eij }1≤i≤l 1≤j≤m as a basis. Remark 1.4.7. One can show using Zorn’s Lemma that every nonzero vector space has a basis. However, no nonzero vector space over R, Q, or C has a unique basis. (Why?) Remark 1.4.8. Suppose {v1 , . . . , vn } is a basis for a vector space V . If a1 v1 + · · · + an vn = b1 v1 + · · · + bn vn , we see ai = bi for each i. This means we have a correspondence V ↔ Fn via the identification a1 v1 + · · · + an vn ←→ (a1 , . . . , an ). Definition 1.4.9. Suppose V is a vector space over F with an ordered basis B = (v1 , . . . , vn ). Given x = a1 v1 + · · · + an vn ∈ V , we define the coordinates of x with respect to B to be [x]B := (a1 , . . . , an ). Remark 1.4.10. An ordered basis is one in which we ascribe a specific order to the elements of the basis. Note the basis in Definition 1.4.9 needs to be ordered to ensure the coordinates defined with respect to it are well-defined. We will typically assume our bases are ordered. Example 1.4.11. Let S be the standard basis for R2 and let B = {(2, 0), (1, 2)}. Note B is a basis for R2 . As 3e1 + 2e2 = (3, 2) = (2, 0) + (1, 2), we have [(3, 2)]S = (3, 2) [(3, 2)]B = (1, 1).

12

DEREK JUNG

Example 1.4.12. Let P3 (F) be the collection of polynomials of degree at most 3 with coefficients in F. Note that P3 (F) forms a vector space over F, but not an algebra. (Why?) We define the bases B := {1, x, x2 , x3 } and C := {1, 1 + x, 1 + x + x2 , 1 + x + x2 + x3 } for P3 (F). As x2 + x3 = (1 + x + x2 + x3 ) − (1 + x), we have [x2 + x3 ]B = (0, 0, 1, 1) [x2 + x3 ]C = (0, −1, 0, 1).

SWILA NOTES

13

2. Linear transformations: More than meets the I 2.1. Linear transformations. I love to go to math conferences. Get to meet new people, receive a nice per diem, and travel to a new place. One thing about traveling though, I’m rarely familiar with the town. So, I have to use a map to help navigate the area to avoid being lost. While some people may just see intersecting lines on maps that make up the roads, I see directions to help keep me on the street and narrow. What’s the moral of this aside? Linear maps enable us to explore the spaces in which we study. And that’s what this chapter is about. Throughout these notes, if we mention two vector spaces V and W , we will assume both vector spaces are over a common field F. For a linear transformation T : V → W , we write 0 6= T or T 6= 0 to mean that there exists x ∈ V such that T (x) 6= 0. Definition 2.1.1. Let V , W be vector spaces over a field F. A map T : V → W is said to be a linear transformation if for all x, y ∈ V and α ∈ F, T (αx) = αT x T (x + y) = T x + T y. Equivalently, for all x, y ∈ V and α ∈ F, T (αx + y) = αT x + T y. Definition 2.1.2. A linear transformation T : V → V is called a linear operator. A linear transformation T : V → F is called a linear functional. Example 2.1.3. The identity operator idV : V → V given by v 7→ v, and the zero transformation 0 : V → W given by v 7→ 0W are the typical examples of linear transformations. Example 2.1.4. Define l1 (R) to be the vector space of R-valued sequences that are absolutely summable, i.e., X |ai | < ∞, ai ∈ R}. l1 (R) := {(ai )i∈N : i∈N

The linear transformation S : l1 (R) → l1 (R) induced by ei 7→ ei+1 , i ∈ N, is an example of a linear transformation that is not invertible. One can similarly define a noninvertible linear transformation S˜ : Σc (R) → Σc (R), where Σc (R) := {(ai )i∈N : ai ∈ R, ∃N so that ai = 0 for all i > N } is the vector space of real sequences with compact support. The interesting thing about these transformations is that they are norm-preserving when equipping l1 (R), Σc (R) with the l1 , sup norm, respectively. R Example 2.1.5. Define L2 (R) to be the set of all functions f : R → R satisfying R |f (x)|2 dx < ∞. L2 (R) forms an R-vector space with the natural operations of function addition and scalar R multiplication. Given f ∈ L2 (R), the map Tf : L2 (R) → R defined by g 7→ R f (x) · g(x) dx defines a linear functional. (Note one needs to check the latter integral is finite for each g ∈ L2 (R).)

14

DEREK JUNG

2.2. Linear transformations T : Fm → Fl as matrices. Lemma 2.2.1. Any linear map T : Fm → F is of the form T = ( x1

···

xm ),

where xi = L(ei ) ∈ F. Hence, T (a1 e1 + · · · + am em ) = a1 x1 + · · · + am xm . Proof. This follows from the linearity of T .



Remark 2.2.2. (Matrix multiplication as a linear transformation) Fix a matrix A ∈ Ml×m (F). Then T : Fm → Fl defined T (x) = Ax defines a linear transformation. Observe for each i = 1, . . . , m,     a1i a11 a12 · · · a1m  a2i   a21 a22 · · · a2m      (e ) = T (ei ) =  .   ..  . .. .. . . i    .. . .  . . al1

al2 · · ·

alm

ali

(We always apply matrices on the left to column vectors. I typically will not differentiate between row vectors and column vectors:   b1  b2    (b1 , b2 . . . , bn ) ←→  ..   .  bn throughout these notes.) Remark 2.2.2 leads us to the following proposition: Proposition 2.2.3. We have a correspondence between Ml×m (F)

←→

{linear transformations T : Fm → Fl }

given by A

←→

(x 7→ Ax).

Proof. We noted in Remark 2.2.2 that a matrix in Ml×m (F) naturally defines a linear transformation T : Fm → Fl . On the other hand, fix a linear transformation T : Fm → Fl . Define the matrix   | | | A =  T (e1 ) T (e2 ) · · · T (em )  | | | So, for each i = 1, . . . , m, the ith column of A is the vector T (ei ) ∈ Fl . Given (a1 , . . . , am ) ∈ Fm ,   a1   A  ...  = a1 T (e1 ) + · · · + am T (em ) = T (a1 e1 + · · · + am em ). am Thus, T (x) = Ax for all x ∈ Fm .