Fundamental Statistics Theory Notes (5)

t-distribution & F-distribution


Let $x_1,x_2…x_3 \overset{\text{i.i.d}}{\sim} N(\mu, \sigma^2)$, $\frac{n-1}{\sigma^2}S^2 \sim \chi_{n-1}^2,\ \frac{\bar x - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$.
To construct CI for $\mu$, we usually don’t now $\mu,\ \sigma$, so we substitute S for $\sigma$(intuitively make sense since $E(S^2) = \sigma^2$)
To get $\frac{\bar x - \mu}{S/\sqrt{n}}$, what is the distribution?
We can express $\frac{\bar x - \mu}{S/\sqrt{n}}$ as $\frac{({\bar x - \mu})/({\sigma/\sqrt{n}})}{({S/ \sqrt{n}})/({\sigma/\sqrt{n}})}$
$\Rightarrow \frac{({\bar x - \mu})/({\sigma/\sqrt{n}})}{\sqrt{S^2/ \sigma^2}} \sim N(0,1)$, it’s very like $\chi^2$.
Given $\frac{\bar x - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$,
$\Rightarrow \frac{\bar x - \mu}{S/\sqrt{n}} \sim \frac{U}{\sqrt{V/{n-1}}}$,
$where\ U\sim N(0,1),\ V\sim \chi_{n-1}^2\ and\ they\ are\ independent\ since\ \bar x, S^2\ are\ independent.$

Definition: A r.v. T has students t-distribution with p d.f. denoted as $T \sim t_p$, if $T \sim \frac{U}{\sqrt{V/P}}$ where $U \sim N(0,1),\ V \sim \chi_p^2$ are independent.

PDF of $t_p$:
Let $x \sim N(0,1),\ y\sim \chi_p^2,$ define $U = \frac{x}{\sqrt{y/P}}, V = y$, inverses are $\ x = U\sqrt{\frac{V}{P}}, y=V$, where $x \in \mathbb{R}, y > 0$.
$\Rightarrow U\in \mathbb{R}, V > 0$,
$J = det\begin{bmatrix}
\sqrt{\frac{v}{P}} & … \\
0 & 1
\end{bmatrix} = \sqrt{\frac{V}{P}}$

$f_{U,V}(U,V) = f_{x,y}(U\sqrt{\frac{V}{P}}, V)|\sqrt{\frac{V}{P}}|$
$= \frac{1}{\sqrt{2\pi}}e^{-(\mu\sqrt{\frac{V}{P}})^2/2}\cdot\frac{1}{\Gamma(\frac{P}{2})2^{P/2}}V^{\frac{P}{2}-1}e^{-\frac{V}{2}}\cdot \sqrt{\frac{V}{P}}$
since $x,\ y$ are independent, the marginal PDF of V is:
$f_V(V) = \frac{1}{\sqrt{2\pi}}\frac{1}{\Gamma(\frac{P}{2})2^{P/2}}\frac{1}{\sqrt{P}}\int_0^{\infty}e^{-\frac{U^2V}{2P}-\frac{V}{2}}V^{\frac{P}{2}-1}\cdot V^{\frac{1}{2}}dV$
$= \frac{1}{\sqrt{2\pi}}\frac{1}{\Gamma(\frac{P}{2})2^{P/2}}\frac{1}{\sqrt{P}}\int_0^{\infty}e^{-V(\frac{1}{2}+\frac{\mu^2}{2P})}V^{\frac{P+1}{2}-1}$, the $e^{-V(\frac{1}{2}+\frac{U^2}{2P})}V^{\frac{P+1}{2}-1}$ is exactly Gamma density.

Gamma PDF with $\alpha,\ \beta$:
$\int_0^{\infty}\frac{1}{\Gamma(\alpha)\beta^\alpha}x^{\alpha-1}e^{-x/\beta} = 1$
$\Rightarrow \int_0^{\infty}x^{\alpha-1}e^{-x/\beta} = {\Gamma(\alpha)\beta^\alpha}$
so $f_V(V) = \frac{1}{\sqrt{2\pi}}\frac{1}{\Gamma(\frac{P}{2})2^{P/2}}\frac{1}{\sqrt{P}}\cdot {\Gamma(\frac{P+1}{2})(\frac{1}{\frac{1}{2}+\frac{U^2}{2p}})^{\frac{P+1}{2}}}$, where $\alpha = \frac{P+1}{2},\ \beta = (\frac{1}{\frac{1}{2}+\frac{U^2}{2p}})$.


Let $x_1,x_2…x_n \overset{\text{i.i.d}}{\sim} N(\mu_{x}, \sigma_{x}^2)$, and $y_1,y_2…y_n \overset{\text{i.i.d}}{\sim} N(\mu_{y}, \sigma_{y}^2)$ and they are independent. Now we want to compute the variances $\frac{\sigma_{x}^2}{\sigma_{y}^2}$, using data the approximation is $\frac{S_{x}^2}{S_{y}^2}$.

Define A r.v. has an F distribution with p and q d.f. if it ca be represented by $\frac{U/p}{V/q}$, where $U \sim \chi_p^2,\ V \sim \chi_q^2$, and they are independent(Notation: $F_{p,q}$).

What is the distribution of $\frac{S_x^2/\sigma_x^2}{S_y^2/\sigma_y^2}$, recall $\frac{(n-1)S^2}{\sigma^2}\sim \chi_{n-1}^2$
$\Rightarrow \frac{S_x^2/\sigma_x^2}{S_y^2/\sigma_y^2} = (\frac{(n-1)S_x^2}{\sigma_x^2}\cdot \frac{1}{n-1})/(\frac{(m-1)S_y^2}{\sigma_y^2}\cdot \frac{1}{m-1}) = \frac{U/(n-1)}{V/(m-1)} \sim F_{n-1,m-1}$, where $U \sim \chi_{n-1}^2, V \sim \chi_{m-1}^2$, they are independent as they originates from independent distribution.

Ex1. $x_1,x_2…x_4 \overset{\text{i.i.d}}{\sim} N(0,1)$, what is the distribution of $\frac{3x_1^2}{x_2^2+x_3^2+x_4^2}$?
Suppose $U = x_1^2 = \chi_1^2,\ V = x_2^2+x_3^2+x_4^2 = \chi_3^2$, they are independent.
$\Rightarrow \frac{3U}{V} = \frac{U/1}{V/3} = \frac{\chi_1^2/1}{\chi_3^2/3} = F_{1,3}$

Ex2. $x_1, x_2 \overset{\text{i.i.d}}{\sim} N(0,2)$, what is the distribution of $\frac{1}{2}(x_1^2+x_2^2)$?
As $\frac{x_1}{\sqrt{2}}\sim N(0,1),\ \frac{x_2}{\sqrt{2}}\sim N(0,1),\ \frac{1}{2}(2U^2+2V^2)=U^2+V^2 = \chi_1^2+\chi_1^2 = \chi_2^2$,
where $U = \frac{x_1}{\sqrt{2}},\ V = \frac{x_2}{\sqrt{2}}$.

If you like my article, please feel free to donate!