Symmetric, Positive Definite Matrix

Related topics:

• Bi-Linear Mapping
• General Inner Product

Table of contents

1. Relation to Inner Product
   1. Symmetric, Positive Semi-Definite Matrix
2. Nullspace of Symmetric, Positive Definite Matrix
3. Diagonal Elements of Symmetric, Positive Definite Matrix

---

Relation to Inner Product

Any general inner product can be represented as a matrix multiplication.

The matrix that uniquely defines the inner product is called the symmetric, positive definite matrix.

Let $V$ be an $n$-dimensional inner product space with inner product $⟨\cdot ,\cdot ⟩:V\times V\to \mathbb{R}$.

Let $B=({\mathbf{b}}_1,\dots ,{\mathbf{b}}_n)$ be an ordered basis of $V$.

Any vector $\mathbf{x},\mathbf{y}\in V$ can be represented as a linear combination of the basis vectors:

$$\mathbf{x}=\sum _{i=1}^n{\lambda }_i{\mathbf{b}}_i\mathbf{y}=\sum _{j=1}^n{\psi }_j{\mathbf{b}}_j$$

for some ${\lambda }_i,{\psi }_j\in \mathbb{R}$.

Also, let $\hat{\mathbf{x}}=[{\lambda }_1\dots {\lambda }_n{]}^T$ and $\hat{\mathbf{y}}=[{\psi }_1\dots {\psi }_n{]}^T$, which are the coordinate vectors of $\mathbf{x}$ and $\mathbf{y}$ with respect to the ordered basis $B$.

Often also denoted $[\mathbf{x}{]}_B$ and $[\mathbf{y}{]}_B$. See coordinate vectors with respect to orderded bases.

The inner product $⟨\mathbf{x},\mathbf{y}⟩$ have three properties:

• Bilinear
• Symmetric
• Positive definite

First using bilinearity:

$$⟨\mathbf{x},\mathbf{y}⟩=⟨\sum _{i=1}^n{\lambda }_i{\mathbf{b}}_i,\sum _{j=1}^n{\psi }_j{\mathbf{b}}_j⟩=\sum _{i=1}^n\sum _{j=1}^n{\lambda }_i⟨{\mathbf{b}}_i,{\mathbf{b}}_j⟩{\psi }_j={\hat{\mathbf{x}}}^T\mathbf{A}\hat{\mathbf{y}}$$

$\hat{\mathbf{x}},\hat{\mathbf{y}}$ are just coordinates, so $\mathbf{A}$ is the matrix that uniquely defines the inner product operation.

Because an inner product is symmetric, $\mathbf{A}$ is also a symmetric matrix.

Now using positive definiteness:

$$\begin{array}{}\text{(Positive Definite)}& \mathrm{\forall }\mathbf{x}\in V\setminus \{\mathbf{0}\},{\mathbf{x}}^T\mathbf{A}\mathbf{x}>0\end{array}$$

Such $\mathbf{A}$ is called the symmetric, positive definite matrix.

Hence, $⟨\mathbf{x},\mathbf{y}⟩$ is a valid inner product if and only if there exists a symmetric, positive definite matrix $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ such that

$$⟨\mathbf{x},\mathbf{y}⟩={\hat{\mathbf{x}}}^T\mathbf{A}\hat{\mathbf{y}}$$

Again, remember that $\hat{\mathbf{x}}$ and $\hat{\mathbf{y}}$ are coordinate vectors of each respect to the ordered basis $B$.

Symmetric, Positive Semi-Definite Matrix

In equation $\text{(Positive Definite)}$, if the inequality is loosened to $\ge$, then $\mathbf{A}$ is called the symmetric, positive semi-definite matrix.

---

Nullspace of Symmetric, Positive Definite Matrix

By definition, we know that ${\mathbf{x}}^T\mathbf{A}\mathbf{x}>0$ for all $\mathbf{x}\ne \mathbf{0}$. So trivially, $\mathbf{A}\mathbf{x}=\mathbf{0}$ only when $\mathbf{x}=\mathbf{0}$.

Therefore, the kernel of $\mathbf{A}$ only contains the zero vector.

In other words, the columns of $\mathbf{A}$ are linearly independent. Also, $\mathbf{A}$ is invertible.

---

Diagonal Elements of Symmetric, Positive Definite Matrix

Diagonal elements of $\mathbf{A}$ are positive, because $a_{ii}={\mathbf{e}}_i^T\mathbf{A}{\mathbf{e}}_i>0$.