Eigenvectors and Eigenvalues

Eigenvectors and eigenvalues are fundamental notions of linear algebra. They describe the "invariant directions" of a linear operator and underlie the diagonalization of matrices, with applications in quantum mechanics, machine learning, the theory of oscillations, and elsewhere.

Definitions

Let $φ$ be a linear operator with matrix $A$ in some basis (that is, $φ (\overline{x})$ in coordinates is $A \overline{x}$ ).

Definition. A nonzero vector $\overline{x} \neq = \overline{0}$ is called an eigenvector of the operator $φ$ (of the matrix $A$ ) if there exists a number $λ$ such that

A \overline{x} = λ \overline{x} .

The number $λ$ is then called the eigenvalue corresponding to the vector $\overline{x}$ . The field of scalars is $R$ or $C$ .

Geometric Interpretation

An eigenvector points in a direction that the operator does not change, only scales by a factor of $λ$ . If $λ = 1$ , the vector is preserved; if $λ = - 1$ , its direction reverses; $∣ λ ∣ > 1$ means stretching, $∣ λ ∣ < 1$ means shrinking.

The Characteristic Equation

Rewrite the definition by moving everything to one side:

A \overline{x} = λ \overline{x} ⟺ (A - λ E) \overline{x} = \overline{0} .

This is a homogeneous $n \times n$ system in the coordinates of $\overline{x}$ . A nonzero solution (an eigenvector is nonzero by definition) exists if and only if the determinant of the system's matrix is zero:

det (A - λ E) = 0.

Definition. The polynomial $f (λ) = det (A - λ E)$ of degree $n$ is called the characteristic polynomial, and the equation $det (A - λ E) = 0$ is called the characteristic equation. Its roots are the eigenvalues.

Theorem. The characteristic polynomial does not depend on the choice of basis (similar matrices $A$ and $C^{- 1} A C$ have the same characteristic polynomial). So the eigenvalues are a characteristic of the operator itself, not of its matrix in a particular basis.

Algorithm

Set up the characteristic equation $det (A - λ E) = 0$ and find its roots $λ_{1}, \dots, λ_{k}$ — the eigenvalues.
For each $λ_{i}$ , solve the homogeneous system $(A - λ_{i} E) \overline{x} = \overline{0}$ — its nonzero solutions give the eigenvectors. The set of all solutions (together with zero) forms the eigenspace $ker (A - λ_{i} E)$ corresponding to $λ_{i}$ .

Example

A = (4123) .

1. Eigenvalues.

det (A - λ E) = 4 - λ 1 2 3 - λ = (4 - λ) (3 - λ) - 2 = λ^{2} - 7 λ + 10 = 0 \Rightarrow λ_{1} = 5, λ_{2} = 2.

(Check: the sum of the roots $5 + 2 = 7 = tr A$ , and the product $5 \cdot 2 = 10 = det A$ .)

2. Eigenvectors. For $λ_{1} = 5$ : $(A - 5 E) \overline{x} = (- 1 1 2 - 2) \overline{x} = 0 \Rightarrow x_{1} = 2 x_{2}$ , which gives $\overline{v}_{1} = (21)$ .

For $λ_{2} = 2$ : $(A - 2 E) \overline{x} = (2121) \overline{x} = 0 \Rightarrow x_{1} = - x_{2}$ , which gives $\overline{v}_{2} = (1 - 1)$ .

Diagonalization

If, in an $n$ -dimensional space, an operator has a basis consisting of eigenvectors, then in this basis its matrix is diagonal, with the eigenvalues on the diagonal. In matrix language: assembling a matrix $C$ from the eigenvectors as columns gives

C^{- 1} A C = D = diag (λ_{1}, \dots, λ_{n}), A = C D C^{- 1} .

For the example above, $C = (21 1 - 1)$ gives $C^{- 1} A C = (5002)$ (cf. the formula $B = C^{- 1} A C$ ).

Properties

A matrix $A \in R^{n \times n}$ has at most $n$ distinct eigenvalues (roots of the characteristic polynomial of degree $n$ ).
Eigenvectors corresponding to distinct eigenvalues are linearly independent. If there are exactly $n$ distinct eigenvalues, the operator is diagonalizable.
$det A = λ_{1} λ_{2} \dots λ_{n}$ and $tr A = λ_{1} + λ_{2} + \dots + λ_{n}$ (counted with multiplicity).
For a symmetric matrix, all eigenvalues are real, and eigenvectors corresponding to distinct eigenvalues are orthogonal.