Fundamental Statistics Theory Notes (3)

Statistics is the function of everything.

Variance & Covariance

  1. $E(a+by+cz)=a+bE(y)+cE(z)$
  2. $Var(a+by+cz)=b^2Var(y)+c^2Var(z)+2bc\cdot Cov(y,z)$
  3. $Var(\sum_{i=1}^{n}a_i y_i) = \sum_{i=1}^{n}a_i^2Var(y_i)+2\sum_{i<j} a_i a_jCov(y_i,y_j)$
  4. $Cov(aU+bV,cX+dY)=acCov(U,X)+adCov(U,Y)+bcCov(V,X)+bdCov(V,Y)$
  5. $Cov(\sum_{i=1}^{n}a_ix_i,\sum_{i=1}^{n}b_iy_i)=\sum_{i=1}^{m}\sum_{j=1}^{n}a_ib_jCov(x_i,y_i)$

    Random Sample

    Define the r.v’s $y_1,y_2…y_n$ are a random sample of size n, if they are all independent and have the same distribution (PMF/PDF), i.e. they are “independent and identically distributed” (iid).

Example 1:
Suppose $y_1, y_2…y_n\overset{\text{iid}}{\sim}Expo(\beta)$, the joint density $f(y_1,y_2…y_n)\overset{\text{indep}}{=}f_{y_1}(y_1)f_{y_2}(y_2)…f_{y_n}(y_n)$
$=\frac{1}{\beta ^n}e^{-\frac{\sum_{i=1}^{n}y_i}{\beta}}, y_i>0\ for\ i=1,2…n$

Define a statistic is a function of the random sample $y_1…y_n$, so a statistic is also a r.v with a distribution, means, variances, etc.

Distribution of statistics are called “sampling distributions.”

Example 2:
Sample mean $\bar Y = \frac{y_1+y_2+…y_n}{n}$, sample variances $S^2 = \frac{1}{n-1}\sum_{i=1}^{n}(y_i-\bar y)^2$, sample standard deviation (SD) is $S = \sqrt{S^2}$.

Properties: let $x_1,x_2…x_n$ be r.v’s with $E(x_i) = \mu, Var(x_i)=\sigma ^2$

  1. $E(\bar X)=E(\frac{x_1+x_2+…x_n}{n})=\frac{1}{n}[E(x_1)+…+E(x_n)]=\frac{1}{n}\cdot n\mu = \mu$
If you like my article, please feel free to donate!