Fundamental Statistics Theory Notes (4)


Let $x_1,x_2…,x_n$ have $E(x_i)=\mu, Var(x_i)=\sigma^2$ and are independent with each other.

  1. $E(\bar {x}) = \mu$

  2. $Var(\bar x) = Var(\frac {x_1+x_2+…x_n}{n})\overset{\text{indep}}{=} \frac {1}{n^2}\sum_{i=1}^{n}Var(x_i)=\frac{1}{n^2}\cdot n\sigma^2=\frac{\sigma^2}{n}$

  3. $S^2 = \frac{1}{n-1}\sum_{i=1}^n(x_i - \bar x)^2 = \frac{1}{n-1}\sum_{i=1}^n(x_{i}^2-2x_i\bar x+\bar x^2)$
    $= \frac{1}{n-1}[\sum_{i=1}^nx_i^2-\sum_{i=1}^{n}2x_i\bar x+\sum_{i=1}^n\bar x^2]$
    $=\frac{1}{n-1}[\sum_{i=1}^n x_i^2-2\bar x\sum_{i=1}^nx_i + n\bar x^2]$
    $=\frac{1}{n-1}[\sum_{i=1}^n x_i^2 - n\bar x^2]$, where $\sum_{i=1}^{n}x_i = n\bar x$

  4. $E(S^2) = \frac{1}{n-1}[\sum_{i=1}^{n}E(x_i^2)-nE(\bar x)^2]$
    $= \frac{1}{n-1}[n(\sigma^2+\mu^2)-n(\frac{\sigma^2}{n}+\mu^2)]$
    where $E(x_i^2) = Var(x_i)+E(x_i)^2,\ E(\bar x^2) = Var(\bar x) + E(\bar x)^2$

Random samples from a Normal Population
Let $x_1, x_2…x_n \overset {\text{i.i.d}}{\sim} N(\mu,\sigma^2)$, then:
a. $\frac{x_i-\mu}{\sigma}\sim N(0,1)$
b. $\bar x \sim N(\mu,\frac{\sigma^2}{n})$
c. $\frac{n-1}{\sigma^2}S^2 \sim \chi_{n-1}^2$
d. $\bar x \ and\ S^2\ are\ independent $

Proof of a and b:

Proof of c:
$\sum_{i=1}^{n}(x_i - \mu)^2 = \sum_{i=1}^{n}(x_i - \bar x +\bar x - \mu)^2$
$= \sum_{i=1}^{n}(x_i - \bar x)^2 + 2\sum_{i=1}^{n}(x_i-\bar x)(\bar x - \mu) + \sum_{i=1}^{n}(\bar x - \mu)^2$
$= \sum_{i=1}^{n}(x_i - \bar x)^2 + n(\bar x - \mu)^2$
where $\sum_{i=1}^{n}(x_i-\bar x)(\bar x - \mu) \Rightarrow (\bar x - \mu)\sum_{i=1}^{n}(x_i-\bar x) \Rightarrow (\bar x - \mu)\cdot 0$
beacsue $\sum_{i=1}^{n}(x_i-\bar x)\ is\ equal\ to\ \sum_{i=1}^{n}x_i-n\bar x\ which\ is\ zero. $
And we know $\sum_{i=1}^{n}(x_i - \bar x)^2 = (n-1) S^2$, so we just divide them by $\sigma^2$ to get:
$\sum_{i=1}^{n}(\frac{(x_i - \mu)}{\sigma})^2 = \frac{(n-1)S^2}{\sigma^2}+(\frac{\bar x -\mu}{\sigma \sqrt{n}})^2$ and we should notice $U_i = \frac{x_i - \mu}{\sigma} \sim N(0,1)$ and $\frac{\bar x - \mu}{\sigma/\sqrt{n}}\sim N(0,1)$
$\Rightarrow U_i^2 \sim \chi_1^2,\ \frac{\bar x - \mu}{\sigma/\sqrt{n}} \sim \chi_1^2\ and\ \sum_{i=1}^{n}U_i^2 \sim \chi_n^2$.
So, $S^2$ and $\bar x$ are independent, $\frac{(n-1)S^2}{\sigma^2}$ and $(\frac{\bar x -\mu}{\sigma/\sqrt{n}})^2$ are independent.

Note: $A = B + C,\ where A\sim \chi_n^2, C\sim \chi_1^2 \Rightarrow B \sim \chi_{n-1}^2$

If you like my article, please feel free to donate!