Moments, Variance, and Correlation

Flip Coin Example

$x_{i}$	$p_{i}$
0	$1 - p$
1	$p$

First moment: $⟨ x ⟩ = \sum p_{i} x_{i} = (1 - p) \cdot 0 + p \cdot 1 = p$
Second moment: $⟨ x^{2} ⟩ = \sum p_{i} x_{i}^{2} = (1 - p) \cdot 0^{2} + p \cdot 1^{2} = p$
$n$ -th moment: still $p$

$Var X = ⟨ x^{2} ⟩ - ⟨ x ⟩^{2} = p - p^{2} = p (1 - p)$

The variance $p (1 - p)$ is a downward parabola in $p \in [0, 1]$ , maximized at $p = 1/2$ .

General continuous form:

$⟨ x^{n} ⟩ = \int p (x) x^{n} d x$

For $p (x) = 1$ on $[0, 1]$ :

$⟨ x^{n} ⟩ = \int p (x) x^{n} d x$
$⟨ x ⟩ = \int_{0}^{1} 1 \cdot x d x = \frac{x ^{2}}{2}_{0}^{1} = \frac{1}{2}$
$⟨ x^{2} ⟩ = \int_{0}^{1} 1 \cdot x^{2} d x = \frac{x ^{3}}{3}_{0}^{1} = \frac{1}{3}$
$Var = \frac{1}{3} - (\frac{1}{2})^{2} = \frac{1}{3} - \frac{1}{4} = \frac{1}{12}$

$σ^{2} \equiv Var (X) \equiv E [(X - μ)^{2}] \equiv E [X^{2} - 2 μ X + μ^{2}]$

$\equiv E [X^{2}] - 2 μ μ E [X] + μ^{2}$

$\equiv E [X^{2}] - μ^{2} \equiv E [X^{2}] - (E [X])^{2}$

Higher moments characterize properties of a distribution.

$σ^{2} \equiv E [(X - μ)^{2}] \equiv E [X^{2}] - E [X]^{2} \equiv \int (x - μ)^{2} p (x) d x$

Dimensionless-normalized cumulant:

$S \equiv \frac{E [( X - μ ) ^{3} ]}{σ ^{3}} = E [(\frac{X - μ}{σ})^{3}]$

Zero for Gaussian, bounded below by $- 1$ :

$k \equiv \frac{E [( X - μ ) ^{4} ]}{σ ^{4}} - 3$

$Cov (X, Y) \equiv E [(X - μ_{X}) (Y - μ_{Y})] = E [X Y] - μ_{X} μ_{Y}$

$ρ (X, Y) = Corr (X, Y) \equiv \frac{Cov ( X , Y )}{Var ( X ) Var ( Y )} = E [(\frac{X - μ _{X}}{σ _{X}}) (\frac{Y - μ _{Y}}{σ _{Y}})]$

Dividing covariance by standard deviations makes correlation a pure number:

$- 1 \leq ρ (X, Y) \leq 1$