Basic Matrix Definitions

Basic definitions.

Table of contents
  1. Matrix
    1. Elementary Row Operations
      1. Row Equivalence
    2. Basic Matrix Math Operations
    3. Matrix Multiplication
  2. Square Matrix
    1. Identity Matrix
    2. Elementary Matrix
    3. Power of a Matrix
  3. Transpose of a Matrix
    1. Symmetric Matrix
    2. Skew-symmetric Matrix
    3. Properties of Transpose

Matrix

Following is a $\mathbf{m \times n}$ matrix $A$, with $m$ rows and $n$ columns:

$$ A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} $$

  • $a_ij$ is the $(i, j)$-th entry or element of $A$.

  • Often denoted in a short-hand notation:

    \[A = [a_{ij}]_{m \times n}\]

Elementary Row Operations

  • Row exchange: swap two rows of a matrix.
  • Row scaling: multiply a row by a nonzero scalar.
  • Row addition: add a scalar multiple of one row to another row.

Row Equivalence

Matrices $A$ and $B$ are row equivalent if there is a sequence of elementary row operations that transforms $A$ into $B$.

We denote row equivalence by:

$$ A \sim B $$

Basic Matrix Math Operations

  • Addition and subtraction of matrices are defined only for matrices of the same size.
  • Scalar multiplication is defined for any matrix and any scalar.
    • Transpose of a scalar is itself: $\lambda^\top = \lambda$

Basic matrix calculations generally satisfy all the usual rules of algebra: commutative, associative, distributive, etc.

Matrix Multiplication

Do not confuse matrix multiplication with Hadamard product (element-wise product).

Matrix multiplication $A \times B$ is defined only if the number of columns of $A$ is equal to the number of rows of $B$: $A \in \mathbb{R}^{m \times n}$ and $B \in \mathbb{R}^{n \times p}$.

If the resulting matrix $C = A \times B$ is defined, then $C \in \mathbb{R}^{m \times p}$.

The $(i, j)$-th element of $C$ is the dot product of the $i$-th row of $A$ and the $j$-th column of $B$:

$$ c_{ij} = \sum_{k=1}^n a_{ik} b_{kj} $$

  • Associative: $(A B) C = A (B C)$
  • Distributive: $A (B + C) = A B + A C$ and $(A + B) C = A C + B C$
  • Not commutative: $A B \neq B A$
  • Not cancellative: $A B = A C$ does not imply $B = C$
  • $A B = 0$ does not imply $A = 0$ or $B = 0$
Expand $(A+B)^2$, etc. to check your understanding

Given that $A$ and $B$ are square matrices of the same size, confirm that

\[\begin{gather*} (A+B)^2 = A^2 + AB + BA + B^2\\ (A-B)^2 = A^2 - AB - BA + B^2\\ (A+B)(A-B) = A^2 - AB + BA - B^2 \end{gather*}\]

Square Matrix

A square matrix is a matrix with the same number of rows and columns.

A $\mathbf{n \times n}$ matrix is called an $n$-dimensional square matrix.

Identity Matrix

An identity matrix (unit matrix) is a diagonal matrix whose principal diagonal elements are all one.

The identity matrix is often denoted by $I$ or $I_n$.

\[I = \begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{bmatrix}\]

Elementary Matrix

An elementary matrix is a square matrix that can be obtained from the identity matrix by a single elementary row operation (row exchange, linear combination of rows).

From $I_3$, we could obtain the following few examples of elementary matrices:

\[I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \quad E_1 = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix} \quad E_2 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -3 & 0 & 1 \end{bmatrix} \quad E_3 = \begin{bmatrix} 2 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}\]

Multiplying a matrix by an elementary matrix is equivalent to performing the corresponding elementary row operation.

Elementary matrices are invertible. Which makes sense, because elementary row operations are invertible.
The inverse of an elementary matrix is also an elementary matrix which undoes the elementary row operation.

Power of a Matrix

For a square matrix $A$, $A^k$ is defined as the power of matrix $A$.

It has the following properties:

  • $A^0 = I$
  • $(A^k)^l = A^{kl}$
  • $A^k A^l = A^{k+l}$

Transpose of a Matrix

The transpose of a matrix $A$ is denoted by $A^\top$:

\[A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} \quad A^\top = \begin{bmatrix} a_{11} & a_{21} & \cdots & a_{m1} \\ a_{12} & a_{22} & \cdots & a_{m2} \\ \vdots & \vdots & \ddots & \vdots \\ a_{1n} & a_{2n} & \cdots & a_{mn} \end{bmatrix}\]

Symmetric Matrix

A matrix $A$ is symmetric if $A = A^\top$.

  • Only square matrices can be symmetric.
  • If $A$ is invertible, then $A^\top$ is also invertible: $(A^{-1})^\top = (A^\top)^{-1}$
  • $A^\top A$ is always symmetric: $(A^\top A)^\top = A^\top A$
    • Same for $A A^\top$: $(A A^\top)^\top = A A^\top$

Skew-symmetric Matrix

A matrix $A$ is skew-symmetric if $A^\top = -A$.

Properties of Transpose

For any matrices $A$ and $B$ and any scalar $c$:

  • $(A^\top)^\top = A$
  • $(A + B)^\top = A^\top + B^\top$
  • $(A B)^\top = B^\top A^\top$
  • $(c A)^\top = c A^\top$