Fundamental Statistics Theory Notes (1)




Suppose y is a continuous random variable, which has a PDF (Probability distribution function) $f(y)$ and a CDF (Cumulative distribution function) $F(y)$ shown as below




For CDF, $F(y)$ is equal to $P(Y<=y) = \int_{-\infty}^{y}f(t)dt$. If we differtiate the $F(y)$, we will get $f(y)$, the PDF, like $f(y) = \frac{d}{dy}F(y)$.

Suppose $z\sim N(0,1)$, $f(z) = \frac{1}{\sqrt{2\pi}}e^{\frac{-z^2}{2}}$, $z \epsilon R$.
Define $Y = |z|$, find PDF/CDF of Y.
First Step: Identify the range
For any y greater than 0, $P(Y\leq y) = P(|z|\leq y) = P(-y\leq z\leq y) = \int_{-y}^{y}\frac{1}{\sqrt{2\pi}}e^{\frac{-z^2}{2}}dz$
Define $\Phi(z) = $ CDF of $N(0,1) = \int_{-\infty}^z\frac{1}{\sqrt{2\pi}}e^{\frac{-z^2}{2}}dz \Rightarrow \Phi(y) - \Phi(-y)$, which is the CDF of y for any y $\geq0$.

Second Step: Get derivative of CDF
PDF of Y is $ f(y) = \frac{d}{dy} \left( \Phi(y) - \Phi(-y) \right) $
= $ \Phi’(y) + \Phi’(-y) $
= $\frac{1}{\sqrt{2\pi}} e^{\frac{(-y)^2}{2}}$ + $\frac{1}{\sqrt{2\pi}} e^{-{\frac{(-y)^2}{2}}} $
= $\frac{2}{\sqrt{2\pi}} e^{-\frac{y^2}{2}} $ for any y $\geq 0$


For Y contains with PDF $f(y)$
$E(Y) = \int_{-\infty}^{+\infty}yf(y)dy$ which is “expected value” or “mean”.
$E(g(y)) = \int_{-\infty}^{+\infty}g(y)f(y)dy$
Varince: $Var(Y) = E(Y^2) - E(Y]^2$

  • Expectations are linear: $E(a + bx + cy) = a + bE(x) + cE(y)$
  • When x, y are independent: $E[g(x)h(y)] = E[g(x)]E[h(y)]$
  • $M_y(t) = E[e^{ty}]$ is the MGF of Y. The MGF uniquely determines the destribution, i.e. if $M_x(t) = M_y(t)$ then $x$~$y$.

$x\sim Expo(\beta)$, which has MGF $M_x(t) = \frac{1}{1-\beta t}$. Whta is the distribution of $Y=2x$?

The MGF of Y is $M_y(t) = E[e^{ty}] = E[e^{t\cdot2x}] = E[e^{(2t)x}] = M_x(2t) = \frac{1}{1-2\beta t}$, which is the MGF of $Expo(2\beta)$.

Since MGF determines the distribution, $Y\sim Expo[2\beta]$.

If you like my article, please feel free to donate!