# Normal Distribution#

Part of a series: Stochastic fundamentals.

Follow reading here

## Definition:#

A random variable \(X\) is normally distributed with mean \(\mu\) and variance \(\sigma\) if its PDF is given by

We write \(X \sim N\).

## Properties:#

\(E(X) = \mu\)

The higher moments are given by \(E[(X-\mu)^2]=\begin{cases} 0 & \text{if p is odd}\\ \sigma^p(p-1)!! & \text{if p is even} \end{cases}\)

Here \((p-1)!! = (p-1)(p-3)(p-5)...1\) is the double factorial

The CDF is given by \(F_X(x) = \Phi \left( \frac{x-\mu}{\sigma}\right) = \frac{1}{2}\left[1 + \text{erf}\left(\frac{x-\mu}{\sqrt{2}\sigma}\right) \right]\) with the special functions \(\Phi (x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^x e^{-t^2/2} \ dt\)

\(\text{erf} (x) = \frac{1}{\sqrt{ \pi}} \int_{-x}^{+x} e^{-t^2} \ dt\)

The 68-95-99.7 rule (3-\(\sigma\)-rule)

How probable is it that \(X\) assumes a value which differs from \(\mu\) less than \(n \cdot \sigma\)?

\[P(|x-\mu| \leq n\sigma ) = F_X(\mu+n \sigma) - F_X(\mu-n \sigma) :\]\[n=1 \quad : \quad 0.683...\]\[n=2 \quad : \quad 0.954...\]\[n=3 \quad : \quad 0.997...\]

Why are normally distributed random variables so important in maths and physics?

Many variables are really normally distributed.

Example: Position of a quantum harmonic oscillator in its ground state

Adding up many iid random variables (with finite variance), the sum tends to a normally distributed random variable.

This is called the central limit theorem, which we prove explicitly later.

Thus: Many uncorrelated stochastic influences

\(\Rightarrow\) Cumulated stochastic influence is normal.

Of all distributions with fixed expectation values \(E(X)=\mu\) and fixed variance \(V(X)=\sigma^2\), the normal distribution \(N(\mu,\sigma^2)\) is the one with maximum entropy \(H(X)= \int_{-\infty}^{+\infty} f_X(x) \log \left(f_X(x)\right) dx \ .\) Thus, if we have only \(\mu\) and \(\sigma^2\) and otherwise have maximum uncertainty about \(X\), we should assume that it’s normal.

Convenience

Large deviations are exponentially rare

All integrals \(\int dx f_X(x) g(x)\) converge

All moments exist and are analytically known

We can also extend this definition to the multivariate case.

So consider a set of \(k\) random variables \(X_j: \Omega\rightarrow\Re^k\) for all \(j=1...k\). Defining component-wise addition and scalar multiplication, we can call the set of random variables a random vector $\vec{x} =

**Def.**
A random vector \(\vec{X}\) is (multivariate) normally distributed with
mean \(\vec{\mu} \in \Re ^{k\times k}\) and covariance matrix
\(S\in \Re ^{k\times k}\) if it has the PDF
\(f_{\vec{X}}(\vec{x})=\frac{1}{\sqrt{2\pi}^k\sqrt{\det(S)}} e^{-\frac{1}{2}(\vec{x}-\vec{\mu})^TS^{-1}(\vec{x}-\vec{\mu})}\)
We write \(\vec{X} \sim N(\vec{\mu},S)\)

**Lemma**
A random vector \(\vec{X}:\Omega\rightarrow \Re ^k\) is normally
distributed if and only if any linear combination
\(\Sigma ^k_{j=1} a_j \vec{X_j}\) is (univariate) normally distributed.