Normal Distribution

Normal Distribution#

Part of a series: Stochastic fundamentals.

Follow reading here


A random variable \(X\) is normally distributed with mean \(\mu\) and variance \(\sigma\) if its PDF is given by

\[ f_{X}(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \]

We write \(X \sim N\).

Normal distribution

Fig. 12 Normal distributions with different values for the mean \(\mu\) and standard deviation \(\sigma\).#


  1. \(E(X) = \mu\)

  2. The higher moments are given by \(E[(X-\mu)^2]=\begin{cases} 0 & \text{if p is odd}\\ \sigma^p(p-1)!! & \text{if p is even} \end{cases}\)

    Here \((p-1)!! = (p-1)(p-3)(p-5)...1\) is the double factorial

  3. The CDF is given by \(F_X(x) = \Phi \left( \frac{x-\mu}{\sigma}\right) = \frac{1}{2}\left[1 + \text{erf}\left(\frac{x-\mu}{\sqrt{2}\sigma}\right) \right]\) with the special functions \(\Phi (x) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^x e^{-t^2/2} \ dt\)

    \(\text{erf} (x) = \frac{1}{\sqrt{ \pi}} \int_{-x}^{+x} e^{-t^2} \ dt\)

  4. The 68-95-99.7 rule (3-\(\sigma\)-rule)

    How probable is it that \(X\) assumes a value which differs from \(\mu\) less than \(n \cdot \sigma\)?

    \[P(|x-\mu| \leq n\sigma ) = F_X(\mu+n \sigma) - F_X(\mu-n \sigma) :\]
    \[n=1 \quad : \quad 0.683...\]
    \[n=2 \quad : \quad 0.954...\]
    \[n=3 \quad : \quad 0.997...\]

Why are normally distributed random variables so important in maths and physics?

  1. Many variables are really normally distributed.

    Example: Position of a quantum harmonic oscillator in its ground state

  2. Adding up many iid random variables (with finite variance), the sum tends to a normally distributed random variable.

    This is called the central limit theorem, which we prove explicitly later.

    Thus: Many uncorrelated stochastic influences

    \(\Rightarrow\) Cumulated stochastic influence is normal.

  3. Of all distributions with fixed expectation values \(E(X)=\mu\) and fixed variance \(V(X)=\sigma^2\), the normal distribution \(N(\mu,\sigma^2)\) is the one with maximum entropy \(H(X)= \int_{-\infty}^{+\infty} f_X(x) \log \left(f_X(x)\right) dx \ .\) Thus, if we have only \(\mu\) and \(\sigma^2\) and otherwise have maximum uncertainty about \(X\), we should assume that it’s normal.

  4. Convenience

    • Large deviations are exponentially rare

    • All integrals \(\int dx f_X(x) g(x)\) converge

    • All moments exist and are analytically known

We can also extend this definition to the multivariate case.

So consider a set of \(k\) random variables \(X_j: \Omega\rightarrow\Re^k\) for all \(j=1...k\). Defining component-wise addition and scalar multiplication, we can call the set of random variables a random vector $\vec{x} =

(1)#\[\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix}\]

Def. A random vector \(\vec{X}\) is (multivariate) normally distributed with mean \(\vec{\mu} \in \Re ^{k\times k}\) and covariance matrix \(S\in \Re ^{k\times k}\) if it has the PDF \(f_{\vec{X}}(\vec{x})=\frac{1}{\sqrt{2\pi}^k\sqrt{\det(S)}} e^{-\frac{1}{2}(\vec{x}-\vec{\mu})^TS^{-1}(\vec{x}-\vec{\mu})}\) We write \(\vec{X} \sim N(\vec{\mu},S)\)

Lemma A random vector \(\vec{X}:\Omega\rightarrow \Re ^k\) is normally distributed if and only if any linear combination \(\Sigma ^k_{j=1} a_j \vec{X_j}\) is (univariate) normally distributed.


Philipp Böttcher, Dirk Witthaut