Last modified: July 08, 2019
This article is written in: 🇺🇸
Linear Algebra forms the backbone of many machine learning algorithms, including linear regression. Understanding matrices and vectors is fundamental in this context.
A vector can be represented as:
$$ y = \begin{bmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{m} \end{bmatrix} $$
where $x_{1}, x_{2}, ..., x_{m}$ are the elements of the vector.
Understanding matrix manipulation is essential in linear algebra and machine learning for organizing and processing data. Here are some fundamental operations:
Element-wise addition of two matrices of the same dimension.
Important: The matrices must have the same number of rows and columns.
Multiplying every element of a matrix by a scalar value.
Multiplying a matrix with a vector.
Important: The number of columns in the matrix must equal the number of elements in the vector.
Combining two matrices.
Procedure:
Important: The number of columns in the first matrix must match the number of rows in the second matrix.
Matrix multiplication, a crucial operation in linear algebra, has specific properties that distinguish it from scalar multiplication.
I. For real numbers, multiplication is commutative.
$$3 \cdot 5 == 5 \cdot 3$$
II. The commutative property does not hold for matrix multiplication.
$$A \times B \neq B \times A$$
I. Associative property holds for multiplication of real numbers.
$$3 \cdot (5 \cdot 2) == (3 \cdot 5) \cdot 2$$
II. Matrix multiplication is associative.
$$A \times (B \times C) == (A \times B) \times C$$
I. In scalar multiplication, 1 is the identity element.
$$z \cdot 1 = z$$
II. For matrices, the identity matrix $I$ serves as the identity element.
$$ I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} $$
When multiplied by any compatible matrix $A$, it results in $A$ itself.
$$A \times I = A$$
I. For non-zero real numbers, the multiplicative inverse holds.
$$x \cdot \frac{1}{x} = 1$$
II. Only square matrices can have an inverse. Not every square matrix has an inverse, analogous to how 0 has no multiplicative inverse in real numbers. Finding matrix inverses involves specific numerical methods.
The transpose of a matrix $A$ of size $m \times n$ is another matrix $B$ of size $n \times m$, where the elements are flipped over its diagonal.
$B_{(j,i)} = A_{(i,j)}$.
Suppose there are multiple hypotheses to predict house prices based on different factors. With a dataset of house sizes, these hypotheses can be applied simultaneously using matrix operations.
For four houses and three different hypotheses:
This approach demonstrates how matrix multiplication can streamline the application of multiple hypotheses to a dataset, enhancing efficiency and scalability.
These notes are based on the free video lectures offered by Stanford University, led by Professor Andrew Ng. These lectures are part of the renowned Machine Learning course available on Coursera. For more information and to access the full course, visit the Coursera course page.