Diagonal of a Matrix / Trace / Triangular Matrix
Table of contents
Principal diagonal
The principal diagonal of a square matrix is the diagonal from the upper left to the lower right.
$$ \{a_{ij} \mid i = j\} $$
Also called the main diagonal.
Each element $a_{ii}$ is called a principal diagonal element.
Trace
A trace of a square matrix $\boldsymbol{A}$ is the sum of its principal diagonal elements:
$$ \tr(\boldsymbol{A}) = \sum_{i=1}^n a_{ii} $$
For square matrices $\boldsymbol{A}$ and $\boldsymbol{B}$:
- $\tr(\boldsymbol{A} + \boldsymbol{B}) = \tr(\boldsymbol{A}) + \tr(\boldsymbol{B})$
$\tr(\boldsymbol{A} \boldsymbol{B}) = \tr(\boldsymbol{B} \boldsymbol{A})$
This property also holds for non-square matrices.
- $\tr(\lambda \boldsymbol{A}) = \lambda \tr(\boldsymbol{A})$
- $\tr(\boldsymbol{I}_n) = n$
Invariant under Cyclic Permutation
Coming from the above properties, for matrices $\boldsymbol{A}, \boldsymbol{B}, \boldsymbol{C}$, whose product is defined but are not necessarily square matrices,
$$ \tr(\boldsymbol{ABC}) = \tr(\boldsymbol{BCA}) = \tr(\boldsymbol{CAB}) $$
Diagonal Matrix
A diagonal matrix is a square matrix whose non-principal diagonal elements are all zero.
\[A = \begin{bmatrix} a_{11} & 0 & \cdots & 0 \\ 0 & a_{22} & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & a_{nn} \end{bmatrix}\]Often denoted by:
$$ A = diag(a_{11}, a_{22}, \cdots, a_{nn}) $$
Properties of a Diagonal Matrix
For diagonal matrices $A = diag(a_{11}\cdots, a_{nn})$ and $B = diag(b_{11}\cdots, b_{nn})$:
- $A + B = diag(a_{11} + b_{11}, \cdots, a_{nn} + b_{nn})$
- $A B = diag(a_{11} b_{11}, \cdots, a_{nn} b_{nn})$
Triangular Matrix
For both upper and lower triangular matrices, if the principal diagonal elements are all 1, then the matrix is called a unit (upper/lower) triangular matrix.
Upper Triangular Matrix
An upper triangular matrix is a square matrix whose non-principal diagonal elements below the principal diagonal are all zero.
Upper triangular matrix, often denoted by $U_n$ is:
$$ U_n = [u_{ij}]_{n \times n} \quad \text{where} \quad \forall i > j,\; u_{ij} = 0 $$
Example:
\[\begin{bmatrix} 1 & 2 & 3 \\ 0 & 4 & 5 \\ 0 & 0 & 6 \end{bmatrix}\]Lower Triangular Matrix
An lower triangular matrix is a square matrix whose non-principal diagonal elements above the principal diagonal are all zero.
Lower triangular matrix, often denoted by $L_n$ is:
$$ L_n = [l_{ij}]_{n \times n} \quad \text{where} \quad \forall i < j,\; l_{ij} = 0 $$
Example:
\[\begin{bmatrix} 1 & 0 & 0 \\ 2 & 3 & 0 \\ 4 & 5 & 6 \end{bmatrix}\]Rectangular Diagonal Matrix
Sometimes you might see people say rectangular diagonal matrix which refers to a non-square matrix that constains a diagonal matrix as a submatrix, and the rest of the elements are all zero.
Suppose $\boldsymbol{A} \in \mathbb{R}^{m \times n}$.
If $m > n$:
\[\boldsymbol{A} = \begin{bmatrix} a_{11} & 0 & \cdots & 0 \\ 0 & a_{22} & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & a_{nn} \\ 0 & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0 \end{bmatrix}\]If $m < n$:
\[\boldsymbol{A} = \begin{bmatrix} a_{11} & 0 & \cdots & 0 & 0 & \cdots & 0 \\ 0 & a_{22} & \cdots & 0 & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & a_{mm} & 0 & \cdots & 0 \end{bmatrix}\]You’ll see this in the context of singular value decomposition.
$a_{ii}$ can be zero.