Last modified: May 20, 2018

This article is written in: 🇺🇸

Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors are foundational concepts in linear algebra, with extensive applications across various domains such as physics, computer graphics, and machine learning. These concepts are instrumental in decomposing complex matrix transformations, thereby simplifying numerical computations.

Definitions

An eigenvector of a square matrix $A$ is a non-zero vector $v$ that, when multiplied by $A$, results in a scaled version of $v$. The scalar factor is the eigenvalue corresponding to that eigenvector. In mathematical terms, this relationship is described as:

$$ A v = \lambda v $$

where:

• $A$ is a square matrix,
• $v$ is an eigenvector of $A$,
• $\lambda$ is the corresponding eigenvalue.

Procedure for Finding Eigenvalues and Eigenvectors

1. Eigenvalues: Eigenvalues are calculated by solving the characteristic equation, formulated as $det(A - \lambda I) = 0$. Here, $I$ is the identity matrix of the same dimension as $A$, and $det(\cdot)$ denotes the determinant. The roots of this equation yield the eigenvalues.

2. Eigenvectors: Upon finding each eigenvalue, its corresponding eigenvectors are obtained by substituting the eigenvalue into the equation $(A - \lambda I)v = 0$, followed by computing the null space.

Example

Consider a 2x2 matrix:

$$A = \begin{bmatrix} 4 & 1 \ 2 & 3 \ \end{bmatrix}$$

1. Solve the characteristic equation $det(A - \lambda I) = 0$, which gives $\lambda^2 - 7\lambda + 10 = 0$. The roots of this equation are $\lambda_1 = 2$ and $\lambda_2 = 5$, representing the eigenvalues.

2. To find the corresponding eigenvectors:

   • For $\lambda_1 = 2$, solve the equation $(A - 2I)v = 0$, which yields the eigenvector $v_1 = [1, -2]$.
   • For $\lambda_2 = 5$, solve the equation $(A - 5I)v = 0$, which yields the eigenvector $v_2 = [1, 1]$.

Applications

• Eigenvalues and eigenvectors are used in PCA for dimensionality reduction, enabling data to be represented in fewer dimensions without significant loss of information.
• They are employed in solving systems of linear differential equations, providing a simplified approach to understanding dynamic systems.
• In quantum mechanics, eigenvalues and eigenvectors of operators correspond to physically measurable quantities and their associated states, respectively.

Limitations

• Eigenvalues and eigenvectors do not exist for all matrices. Specifically, non-square matrices lack these properties.
• For large matrices, the computation of eigenvalues and eigenvectors can be computationally demanding, potentially slowing down analyses or algorithm implementations.