Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
MEHRAN UNIVERSITY OF ENGINEERING AND TECHNOLOGY, JAMSHORO. FIRST SEMESTER FIRST YEAR (M.Phil.) 15BATCH PROGRAM: APPLIED MATHEMATICS Subject: Scientific Computing LECTURE 13 Conducted on: 04/05/2015 Monday Timings: 6pm to 7.30pm
“I do not believe in taking the right decision, I take a decision and make it right.” Muhammad Ali Jinnah SYSTEM OF LINEAR ALGEBRAIC EQUATIONS (Eigenvalues and Eigenvectors)
Objectives: o o o o o o o o o o o
Introduction to Eigenvalues and associated Eigenvectors Importance of Eigenvalue Problems in Scientific Areas Properties of Eigenvalues Computation of Eigenvalues using analytical approach Numerically Dominant and Sub-dominant Eigenvalues Spectral radius, Spectrum and Eigenspace Computation of Eigenvalues using numerical approach (Power’s Method) Convergence of the Power’s Method Rayleigh Quotient Method Applied Problems Discussion on Research Papers written on the Topic
Introduction and Importance of Eigenvalues Consider the following vector equation:
1
Ax x
Here x 0 (trivial solutions) is an uninteresting solution, therefore, we seek only x 0 (nontrivial solutions). A x x 0 A I x 0
2
The above innocent looking equation 1 has numerous applications. It comes up all the time in Engineering, Physics, Geometry, Numerics, Theoretical Mathematics, Biology, Environmental Science, Urban Planning, Economics, Psychology and other areas. For example, in the subject of Vibrational Analysis of Mechanical Systems, eigenvalues and eigenvectors describe the angular frequency and mode of vibration of the system respectively, while in mechanics they represent principal stresses and the principal axes of stress in bodies subjected to external forces. Eigenvalues play an important role in the stability analysis of dynamical systems. 1
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Equation (2) is a homogeneous system. Therefore, for non-trivial solutions, it is required that A I 0 3
This is called Characteristic Polynomial and its roots are called Eigenvalues. Eigenvalue is a German word, its English term is Characteristic Value but seldom used. The set of all eigenvalues of A is called Spectrum of A. The spectrum consists of at least one eigenvalue and at most n numerically different eigenvalues. Eigenvalues of a system can be real, complex, distinct and real, real and repeated or any of the combinations. Eigenvector: - Eigenvalues are the numbers for which Ax x has non-trivial solutions. For each eigenvalue, there is a non-trivial solution of the system. This solution is called Eigenvector. It should be noted that for a single eigenvalue, there are infinite eigenvectors. Eigenspace: - If w and x are eigenvectors of a matrix A corresponding to the same eigenvalue of A, so are
w x (provided w x ) and k x for any k 0 . Hence the eigenvectors corresponding to one and the same eigenvalue of A, together with 0 , form a vector space, called the Eigenspace of A corresponding to that . In particular, an eigenvector is determined only up to a constant factor. Hence, we can normalize x , that is multiply it by a scalar to get a unit vector. For instance, 1 x1 , and x1 12 22 5 2 Normalized eigenvector: 1 x1 5 . 2 5 Example. Determine eigenvalues for the system: 2 x y x
3 x 4 y y Solution. 2 1 x x 3 4 y y Ax x 2 1 1 0 2 A I 3 4 0 1 3 A I 0
2
1
3
4
1 4
0
2 trace A det A 0 2 6 5 0 1, 5
2
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Example. Find eigenvalues and eigenvectors for: 2 0 1 A 1 1 1 3 2 2
Solution. 0 1 2 1 0 0 1 A I 1 1 1 0 1 0 1 3 2 2 0 0 1 3 1 A I 0 1 3
2
0
1
1
2
2
0
3 22 2 0 2,1,1 For 2 : 0 x 1 2 x 1 1 1 y 2 y 3 2 2 z z 0 x x 1 2 1 1 1 y 2 y 0 3 2 2 z z
1 2 0 2 0 0 x 1 1 1 0 2 0 y 0 3 2 2 0 0 2 z 3 2 0 x 0 1 1 1 y 0 3 2 0 z 0
Solving this homogeneous system, we get 3 5 x t, y t, z t 2 2 1 X 1 t 3 2 5 2
For 1:
3
2 1 2
1 2 0
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
1 X 2 t 1 1
For 1: 1 X 3 t 0 1
Example. Determine spectrum and Eigenspace for the following matrix: 2 2 3 A 2 1 6 1 2 0
Solution.
3 2 21 45 0 1 5, 2 3 3 Spectrum 5,3
For 1 5 : 7 2 3 A I A 5I 2 4 6 1 2 5
Reduced row echelon form is: 1 0 1 0 1 2 x1 x3 0 ; x is a free parameter 3 x2 2 x3 0 0 0 0
Let x3 k , then
x1 k and x2 2k Thus, k 1 X 1 2k 2 k k 1
For 2 3 3 :2 3 1 A I A 3I 2 4 6 1 2 3
Reduced row echelon form is:
4
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
1 2 3 0 0 0 x 2 x 3x 0; x and x are free parameters 1 2 3 2 3 0 0 0
Let x2 k1 and x3 k2 , then
x1 2k1 3k2 Thus, 2k1 3k 2 2k1 3k 2 2 3 k 0 1 k 0 k X k1 1 1 2 0 k 2 0 1 k2
We have obtained two linearly independent eigenvectors of A corresponding to 3 : 2 3 X 2 1 , X 3 0 0 1
Hence, the Eigenspace for 5 is: 1 3 E5 A x R : x k 2 , k R 1
Hence, the Eigenspace for 3 is: 2 3 E3 A x R 3 : x k1 1 k 2 0, k1 , k 2 R 0 1
Note that dim E5 A 1 and dim E3 A 2 . Properties of Eigenvalues 1. The sum of the eigenvalues of A is: i trace A aii n
n
i 1
i 1
2. The product of the eigenvalues of A is: i det A n
i 1
3. The eigenvalues of A1 , provided it exists, are: 4. 5. 6. 7. 8.
T
1
,
1
,
1
1 2 3
, ,
1
n
Eigenvalues of A = Eigenvalues of A If k is a positive integer then the eigenvalues of Ak are: 1k , k2 , , kn Eigenvalues of a real symmetric matrix are always real but the converse is not true. Eigenvalues of a skew-symmetric matrix are pure imaginary numbers or zero. The Eigenvalues of an orthogonal matrix A are real or complex conjugate in pairs and have absolute value 1. 5
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Verify eighth property on the following matrix? 2 / 3 1/ 3 2 / 3 2 / 3 2 / 3 1 / 3 1 / 3 2 / 3 2 / 3
Physical Applications The mass-spring system in the following figure1 is a simple way to get insight into how eigenvalues seek entrance in our physical world. To simplify the analysis, assume that each mass has no external or damping forces acting on it. In addition, assume that each spring has the same length l and the same spring constant k. Finally, assume that the displacement of each spring is measured relative to its own local coordinate system with an origin at the spring’s equilibrium position. Under these assumptions, Newton’s second law can be employed to develop a force balance for each mass: ma F
d 2 x1 m1 2 kx1 k x2 x1 dt
m2
d 2 x2 kx2 k x2 x1 dt 2
Fig1. A two mass-three spring system with frictionless rollers vibrating between two fixed walls. The position of three masses can be referenced to local coordinates with origins at their respective equilibrium positions (a). As in (b), positioning the masses away from equilibrium creates forces in the springs that on release lead to oscillations of the masses.
6
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Here, displacement of mass i way from its equilibrium position is shown by xi . From Vibration Theory, solution to above system is:
xi X i sin t 4
X i the amplitude of the oscillation of mass i (m) the angular frequency of the oscillation (radians/time):
2 Tp
T p the period (time/cycle).
Inverse of the period is called the ordinary frequency f (cycles/time). If time is in seconds, the unit of f is the cycles/s, which is referred to as Hertz (Hz). Differentiating (4) twice and substituting into above system of Ordinary Differential equations, we get
2k k 2 X 1 X2 0 m1 m1
2k k X 1 2 X 2 0 m2 m2
Comparison of above equations with (2) reveals that it has been converted into an eigenvalue problem, where the eigenvalue is the square of the frequency. If we are given m1 m2 40kg , k 200N / m then above set of equations become
10 X 5 X 0 10 5 5 X 10 X 0 2
1
2
2
1
2
Let 2 :-
X 10 5 X 1 1 5 10 X 2 X2 7
5 X 1 2 X1 X 10 X 2 2
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
2 20 75 0 15,5 Thus, the two eigenvalues are 2 15 and 2 5 / s 2 and corresponding eigenvectors are: For 2 15 : 5 X1 5 X 2 0 5 X1 5 X 2 0
X1 X 2
For 2 5 :5 X1 5 X 2 0 5 X1 5 X 2 0
X1 X 2
The example provides us valuable information regarding the behavior of the system shown in figure 1. First, it tells us that the system has two primary modes of oscillation with angular frequencies of 15 3.873 and 5 2.36 radians/second, respectively. Periods = 1.62 and 2.81 seconds, respectively Ordinary frequency = 0.6164 and 0.3559 Hz, respectively.
Second, if the system is vibrating in the first mode, the first eigenvector tells us that the amplitude of the second mass will be equal but of opposite sign to the amplitude of the first. In the second mode, the eigenvector specifies that the two masses have equal amplitudes at all times. Dominant and Subdominant Eigenvalue: - Let A be an n n matrix and 1 , 2 ,, n be its n distinct eigenvalues so that
1 2 3 n ; then 1 is said to be dominant eigenvalue of A and the remaining eigenvalues 1 , 2 , 3 ,, n are called subdominant eigenvalues of A. Moreover, the eigenvectors corresponding to 1 are called dominant eigenvectors of A. 8
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Power’s Iteration Method It is an iterative procedure to find numerically dominant eigenvalue of a matrix A. The power iteration is a very simple algorithm. It does not compute a matrix decomposition, and hence it can be used when A is a very large sparse matrix. However, it will find only one eigenvalue (the one with the greatest absolute value) and it may converge only slowly. The algorithm is also known as the Von Mises iteration. The method fails when a matrix has repeated dominant eigenvalues. Moreover, the method can be used to find smallest eigenvalue if applied to inverse of A, provided that it exists. Although the power iteration method approximates only one eigenvalue of a matrix, it remains useful for certain computational problems. For instance, Google uses it to calculate the PageRank of documents in their search engine, and Twitter uses it to show users recommendations of who to follow. For matrices that are well-conditioned and as sparse as the Web matrix, the power iteration method can be more efficient than other methods of finding the dominant eigenvector. There are other better methods to find eigenvalues including Jacobi’s method, Householder’s method, LR method, QR method and many more. Iterative Process yk 1 Axk
yk 1 ; k 0,1,2, xk 1 max yk 1
1 1 It is convention to let x0 be an initial guess for the dominant eigenvector. 1 Example. Use five iterations for Power’s Iteration Method to determine dominant eigenvalue and its associated eigenvector for the following matrix:
9
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
10 7 8 A 7 6 6 8 6 19
Solution. Let 1 x0 1 1 10 7 8 1 25 7.575757575757576e- 01 y1 Ax0 7 6 6 1 19 33 5.757575757575758e- 01 dominant x1 8 6 19 1 33 1 10 7 8 7.575757575757576e- 01 1.960606060606061e+ 01 y2 Ax1 7 6 6 5.757575757575758e- 01 1.475757575757576e+ 01 8 6 19 2.851515151515152e+ 01 1 6.875664187035068e- 015.175345377258235e- 01 y2 2.851515151515152e+ 01 5.175345377258235e- 01 dominant x2 9.999999999999999e- 01 10 7 8 6.875664187035068e- 015.175345377258235e- 01 1.849840595111583e+ 01 1.391817215727949e+ 01 y3 Ax2 7 6 6 5.175345377258235e- 01 8 6 19 2.760573857598299e+ 01 9.999999999999999e- 01 6.700927743773338e- 01 y3 2.760573857598299e+ 01 5.041767717596336e- 01 dominant x3 1
10 7 8 6.700927743773338e- 01 1.823016514609077e+ 01 y4 Ax3 7 6 6 5.041767717596336e- 01 1.371571005119914e+ 01 8 6 19 2.738580282557647e+ 01 1 6.656794128768445e- 01 y4 2.738580282557647e+ 01 5.008328636029470e- 01 dominant x4 1.000000000000000e+ 00
10 7 8 6.656794128768445e- 01 1.816262417398908e+ 01 y5 Ax4 7 6 6 5.008328636029470e- 01 1.366475307175560e+ 01 8 6 19 1.000000000000000e+ 00 2.733043248463244e+ 01
10
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
6.645567787557586e- 01 y5 2.733043248463244e+ 014.999830529370186e- 01 dominant x5 1
Output from a C++ source code is shown below for the above problem where absolute relative error is computed using the following formula:
new old new
The iterations have been stopped after attaining the absolute relative error of 10 7 . Look at following graph to capture linear behavior of the error:
11
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Example. Determine smallest eigenvalue of the given matrix A. Further, find SPECTRUM of A if its dominant eigenvalue is 2.7093: 2 1 1 A 0 2 1 5 2 3
Solution. Let 8 1 3 B A 5 1 2 . 10 1 4 1
We will find dominant eigenvalue of B using Power’s method and take its reciprocal which is in fact smallest eigenvalue of the given matrix A. Iterations are shown in the following screenshot:
Thus, approximate smallest eigenvalue of A is
1 0.1939 . 5.1563
2.7093 0.1939 3 1 3 1.9032 SPECTRUM = 2.7093, 0.1939, 1.9032 Convergence of the Power’s Method If A is an n n diagonalizable matrix with a dominant eigenvalue, then there exists a non-zero vector x0 such that the sequence of vectors given by Ax0 , A2 x0 , A3 x0 , , Ak x0 ,
approaches a multiple of the dominant eigenvector of A. 12
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
If the eigenvalues of A are ordered so that
1 2 3 n , then the power’s method will converge quickly if
2 is small and it will converge slowly if 1 2 is close to 1. 1 Example. How many iterations are required for the power’s method to converge to the dominant eigenvalue with error tolerance of 107 for the following matrix?
4 5 A 6 5 Solution. Eigenvalues for the matrix are: 10 and -1. Look at the following screenshot for number of iterations (8):
Example. How many iterations are required for the power’s method to converge to the dominant eigenvalue with error tolerance of 102 for the following matrix? 4 10 A 5 7
Solution. Eigenvalues for the matrix are: 10 and -9. 13
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Look at the following screenshot for number of iterations (36):
The method attains such a small error of 10-2 in 36 iterations whereas in previous example, the error of 10-7 was attained only in 8 iterations For the first matrix:
2 1 0.1 1 1 10
For the second matrix:
2 9 0.9 1 1 10
Rayleigh Quotient Method English Physicist John William Rayleigh (1842 – 1919) It is used to determine an eigenvalue from an eigenvector. If X is an eigenvector of a matrix A, then its corresponding eigenvalue is given by
Ax x This quotient is called the Rayleigh Quotient. x x
Proof: Since x is an eigenvector of A, we know that
Ax x We can write
14
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Ax x x x x x x x x x x x In cases for which the power method generates a good approximation of a dominant eigenvector; the Rayleigh Quotient provides a correspondingly good approximation of the dominant eigenvalue. Example. Find dominant eigenvector for: 2 12 A 1 5
Solution. 2 12 1 10 2.5 x1 Ax0 4 1 5 1 4 1 2 12 10 28 2.8 x2 Ax1 10 1 5 4 10 1 2 12 28 64 2.9091 x3 Ax2 22 1 5 10 22 1 2 12 64 136 2.9565 x4 Ax3 46 1 5 22 46 1 2 12 136 280 2.9787 x5 Ax4 94 1 5 46 94 1 2 12 280 568 2.9895 x6 Ax5 190 1 5 94 190 1
Approximations are approaching to scalar multiple of 3 1
Dominant
Eigenvecto r
Example. Determinant dominant eigenvalue for the above matrix using Rayleigh Quotient method? Solution. After 6th iteration, we have 568 2.9895 x6 190 190 1 2 12 2.9895 6.0210 Ax 1 5 1 2.0105 15
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
6.0210 2.9895 Ax x 6.0210 2.9895 2.0105 1 20 .0103 2.0105 1 2.9895 2.9895 x x 9.9371 1 1
Ax x 20.0103 2.0137 x x 9.9371 Approximate dominant eigenvalue is -2.0137. Example. Engineers and scientists use mass-spring models to gain insight into the dynamics of structures under the influence of disturbances such as earthquakes. Following figure shows such a model for three story building. Each floor mass is represented by mi , and each floor stiffness is represented by ki for i 1,2,3.
For this case, the analysis is limited to horizontal motion of the structure as it is subjected to horizontal base motion due to earthquakes. Using the same approach as in previous application problem, dynamic force balances can be can be developed for this system as k1 k 2 k n2 X 1 2 X 2 0 m1 m1
k k k k2 X 1 2 3 n2 X 2 3 X 3 0 m2 m2 m2
k k3 X 2 3 n2 X 3 0 m3 m3
Where X i represent horizontal floor translation (m), and n is the natural, or resonant, frequency (radians/s). The resonant frequency can be expressed in Hertz (cycles/s) by dividing it by 2 radians/cycle.
16
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
Determine the eigenvalues and eigenvectors for this system. Graphically represent the modes of vibration for the structure by displaying the amplitudes versus height for each of the eigenvectors. Normalize the amplitudes so that the translation of the third floor is one. Solution. Substituting all the values of parameters into the force balances, we get
450 X 240 X 420 X 2 n
1
200 X 2 0
1
2 n
2
180 X 3 0
225 X 2 225 n2 X 3 0
Iterations for finding dominant eigenvalue are:
Using MATLAB:
17
Prepared by Asif Ali Shaikh/Sania Qureshi Assistant Professors Department of BSRS Mehran, UET, Jamshoro
Note: Detailed discussion of following topics is being carried out in the classes. It is highly recommended that you may follow textbooks suggested by your mentors. These handouts can never ever replace textbooks. Material included here has been taken from textbooks and internet.
The eigenvalues are: 698.5982
339.4779
56.9239
2.9324
1.2008
Resonant frequencies in Hertz are: 4.2066
The corresponding eigenvectors are normalized in such a way that their third component is 1: 1.6934 2.1049 , 1
0.9207 0.5088 , 1
0.3801 0.7470 1
The three primary modes of oscillation of the three-story building
18
