Eigenvalues & Eigenvectors

For a square $n\times n$ matrix $A$, a scalar $\lambda$ is an eigenvalue of $A$, where:

$$Av=\lambda v$$

This can be rewritten as a homogenous equation in an unknown vector $v$:

$$(A-\lambda I_n)v=o_n$$

This equation has infinitely many non-trivial solutions for $v$, where:

$$\det (A-\lambda I_n)=0$$

This is the characteristic equation of $A$, and the eigenvalues $\lambda$ are scalars that satisfy this. Since the characteristic equation is an $n$-th degree polynomial, an $n\times n$ matrix will have $n$ eigenvalues ${\lambda }_i$ for $i=1,\mathrm{2...},n$.

Corresponding to each eigenvalue ${\lambda }_i$, eigenvectors $v^{(i)}$ are non-trivial solutions of:

$$(A-{\lambda }_i{I}_n)v^{(i)}=o_n$$

Example

Eigenvalues and vectors of:

$$A=\left[\begin{array}{cc}0& 1\\ -2& -3\end{array}\right]$$

The characteristic equation and it's solutions:

$$\det (A-\lambda I_n)=0$$

$$\begin{gathered} \left|\begin{array}{cc}-\lambda & 1\\ -2& -3-\lambda \end{array}\right|={\lambda }^2+3\lambda +2=0 \\ \lambda =-1,-2 \end{gathered}$$

Eigenvector for $\lambda =-1$:

$$(A-\lambda I_n)v=o_n$$

$$\left(\left[\begin{array}{cc}0& 1\\ -2& -3\end{array}\right]-\left[\begin{array}{cc}-1& 0\\ 0& -1\end{array}\right]\right)\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=0$$

$$\begin{gathered} x_1+x_2=0 \\ -2x_1-2x_2=0 \end{gathered}$$

$$x_1=-x_{2}x=\left[\begin{array}{c}-\alpha \\ \alpha \end{array}\right]$$

Eigenvector for $\lambda =-2$:

$$(A-\lambda I_n)v=o_n$$

$$\left(\left[\begin{array}{cc}0& 1\\ -2& -3\end{array}\right]-\left[\begin{array}{cc}-2& 0\\ 0& -2\end{array}\right]\right)\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=0$$

$$\begin{gathered} 2x_1+x_2=0 \\ -2x_1-x_2=0 \end{gathered}$$

$$-2x_1=x_{2}x=\left[\begin{array}{c}\alpha \\ -2\alpha \end{array}\right]$$

Spectral Decomposition

An $n$x$n$ matrix $A$ has $n$ eigenvectors ${\lambda }_1...{\lambda }_n$ and $n$ associated eigenvectors $v^{(1)}...v^{(1)}$.

$V$ is an $n$x$n$ matrix of column eigenvectors, and $\Lambda$ is an $n$x$n$ diagonal matrix of eigenvalues

$$V=\left[\begin{array}{cccc}{v}^{(1)}& v^{(2)}& ...& v^{(n)}\end{array}\right]$$

$$\Lambda =\left[\begin{array}{cccc}{\lambda }_1& 0& \dots & 0\\ 0& {\lambda }_2& \dots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & {\lambda }_n\end{array}\right]$$

$AV=V\Lambda$ for all $n\times n$ matrices

In general, eigenvectors of $A$ are linearly independent and so $V^{-1}$ exists. The spectral decomposition of a matrix $A$ can then be written:

$$A=V\Lambda V^{-1}$$

This allows for diagonalisation of a matrix in terms of its eigenvectors, and for breaking down a multi-dimensional problem into a set of single dimensional problems.

• This is only possible if all eigenvectors are linearly independent.
  • If any are repeated then this is not the case

If $A$ is a symmetric matrix, then the eigenvectors are mutually orthogonal, ie $v^{(i)}\cdot v^{(j)}=0$ for all $i\ne j$. If these eigenvectors are orthonormalised (of unit length), then the matrix of eigenvectors $V$ is an orthogal matrix, meaning its transpose is equal to it's inverse. Hence, the spectral resolution of a symmetric matrix $A$ is:

$$A=V\Lambda V^{-1}=V\Lambda V^{\prime }$$

Example

Find the spectral resolution of, and hence diagonalise:

$$A=\left[\begin{array}{cc}1& 2\\ 2& 1\end{array}\right]$$

The eigenvalues of $A$ are ${\lambda }_1=3$ and ${\lambda }_2=-1$. These can then be used to compute the corresponding eigenvectors:

$$v^{(1)}=\left[\begin{array}{c}\alpha \\ \alpha \end{array}\right]v^{(2)}=\left[\begin{array}{c}\alpha \\ -\alpha \end{array}\right]$$

Using $\alpha =1$:

$$\Lambda =\left[\begin{array}{cc}{\lambda }_1& 0\\ 0& {\lambda }_2\end{array}\right]=\left[\begin{array}{cc}3& 0\\ 0& -1\end{array}\right]$$

$$V=\left[\begin{array}{cc}{v}_1^{(1)}& v_1^{(2)}\\ v_2^{(1)}& v_2^{(2)}\end{array}\right]=\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]$$

$$V^{-1}=\left[\begin{array}{cc}0.5& 0.5\\ 0.5& -0.5\end{array}\right]$$

The spectral resolution of $A$ is given by:

$$A=V\Lambda V^{-1}=\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}3& 0\\ 0& -1\end{array}\right]\left[\begin{array}{cc}0.5& 0.5\\ 0.5& -0.5\end{array}\right]$$

$A$ can then be diagonalised by $V^{-1}AV$:

$$V^{-1}AV=\left[\begin{array}{cc}0.5& 0.5\\ 0.5& -0.5\end{array}\right]\left[\begin{array}{cc}3& 0\\ 0& -1\end{array}\right]\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]=\left[\begin{array}{cc}3& 0\\ 0& -1\end{array}\right]$$