Quadratic Forms

A quadratic form in variables $x_{1}, x_{2}, \dots, x_{n}$ is a homogeneous polynomial of the second degree — a function containing no first-degree terms and no constant term:

f (x_{1}, \dots, x_{n}) = i = 1 \sum n a_{ii} x_{i}^{2} + 2 i < j \sum a_{ij} x_{i} x_{j} = a_{11} x_{1}^{2} + \dots + a_{nn} x_{n}^{2} + 2 a_{12} x_{1} x_{2} + 2 a_{13} x_{1} x_{3} + \dots

The Matrix of a Quadratic Form

The form is associated with a symmetric matrix $A = (a_{ij})$ , $a_{ij} = a_{j i}$ : the diagonal holds the coefficients of the squares, and the coefficient of $x_{i} x_{j}$ is split evenly between the symmetric positions $a_{ij}$ and $a_{j i}$ (which is why the form carries a factor of $2$ ). Then

f (X) = X^{T} A X, X = (x_{1}, \dots, x_{n})^{T} .

For example, the form $f = 5 x_{1}^{2} + 6 x_{1} x_{2} + 2 x_{2}^{2} + 2 x_{1} x_{3} - x_{2} x_{3} + x_{3}^{2}$ corresponds to the matrix

A = 531 32 - \frac{1}{2} 1 - \frac{1}{2} 1

(the coefficient $6$ of $x_{1} x_{2}$ gives $a_{12} = a_{21} = 3$ ; the coefficient $- 1$ of $x_{2} x_{3}$ gives $a_{23} = a_{32} = - \frac{1}{2}$ ).

Definiteness

A quadratic form (and its matrix) is called:

positive definite if $f (X) > 0$ for all $X \neq = 0$ ;
negative definite if $f (X) < 0$ for all $X \neq = 0$ ;
positive (negative) semidefinite if $f (X) \geq 0$ (respectively $f (X) \leq 0$ ) for all $X$ ;
indefinite if $f$ takes both positive and negative values.

Sylvester's Criterion

Introduce the leading (principal) minors of the matrix $A$ :

D_{1} = a_{11}, D_{2} = a_{11} a_{21} a_{12} a_{22}, \dots, D_{n} = det A .

Theorem (Sylvester's criterion).

The form is positive definite $⟺$ all leading minors are positive:

D_{1} > 0, D_{2} > 0, \dots, D_{n} > 0.

The form is negative definite $⟺$ the signs of the leading minors alternate, starting with a minus:

D_{1} < 0, D_{2} > 0, D_{3} < 0, D_{4} > 0, \dots ((- 1)^{k} D_{k} > 0) .

Example (Positive Definite)

f (x_{1}, x_{2}, x_{3}) = 2 x_{1}^{2} + 2 x_{2}^{2} + 3 x_{3}^{2} + 2 x_{1} x_{2}, A = 210120003 .

Leading minors:

D_{1} = 2 > 0, D_{2} = 2112 = 3 > 0, D_{3} = det A = 3 \cdot 3 = 9 > 0.

All positive $\Rightarrow$ the form is positive definite. (Indeed, $f = (x_{1} + x_{2})^{2} + x_{1}^{2} + x_{2}^{2} + 3 x_{3}^{2} \geq 0$ , with equality only at zero.)

Example (Indefinite)

f (x, y) = x^{2} + 4 x y + y^{2}, A = (1221) .

Here $D_{1} = 1 > 0$ , but $D_{2} = 1 - 4 = - 3 < 0$ , so the form is neither positive nor negative definite — it is indefinite. This is also clear directly: $f (1, 1) = 6 > 0$ , while $f (1, - 1) = - 2 < 0$ .

Relation to Eigenvalues

Any quadratic form can be reduced, by an orthogonal change of coordinates (in a basis of eigenvectors of the symmetric matrix $A$ ), to canonical form with no cross terms:

f = λ_{1} y_{1}^{2} + λ_{2} y_{2}^{2} + \dots + λ_{n} y_{n}^{2},

where $λ_{1}, \dots, λ_{n}$ are the eigenvalues of the matrix $A$ (all real, since $A$ is symmetric). This gives a second definiteness criterion:

the form is positive definite $⟺$ all $λ_{i} > 0$ ;
negative definite $⟺$ all $λ_{i} < 0$ ;
indefinite $⟺$ there are eigenvalues of both signs.

Applications

Quadratic forms arise everywhere: examining the sign of the second differential (the Hessian matrix) when finding extrema of functions of several variables; classifying curves and surfaces of the second order; potential energy and the energy of small oscillations in physics; covariance matrices in statistics.