Fundamental Statistics Learning Note(18)

Classical hypothesis testing workflow

I am going to summarize the workflow of hypothesis testing including LR test, $\alpha$ level and p-value.

  1. Scientific context determines $H_O, H_A$
  2. Determine test statistics $W$ (LR test is commonly used)
  3. Scientist set an acceptable type I error rate ($\alpha$-level)
  4. We use that to set rejection region $R$ to achieve $\alpha=\max\limits_{\theta\in\Omega_0}p(W\in R|\theta)$
  5. power analysis: does the test have a high probability to reject $H_O$ when $H_A$ is true? i.e. Calculate $pow(\theta)$ for $\theta\in\Omega_O$
  6. Calculate p-value once we have data (smallest level $\alpha$ at which $H_O$ would be rejected)

E.X. $X\sim Expo(\beta), H_O:\beta\leq 1\text{ vs }H_A:\beta >1$, test statistics $W=X$.
(a). Determine rejection region $R$ for a level $\alpha=0.05$ test. Since $E(W)=\beta$, reject $H_O$.

$$\begin{equation}\begin{split}
\alpha = 0.05 & = \max \limits_{\beta\leq 1} p(W\geq C) \\
& = \max\limits_{\beta\leq 1}\int_c^\infty \frac{1}{\beta}e^{-X/\beta}dx \\
& =\max \limits_{\beta\leq 1}e^{-C/\beta}=e^{-C}
\end{split}\end{equation}$$

Note: $C$ denotes how large is $W$ for rejecting $H_O$?

So $C=-log(0.05)=2.995 \Rightarrow R=\lbrace W\geq 2.995\rbrace$, i.e. to achieve 0.05 level test, reject $H_O$ if $W\geq 2.995$

(b). For MLE in (a), plot power function.

$pow(\beta)=p(W\geq 2.995 |\beta)=e^{\frac{-2.995}{\beta}} $

e.g. if $\beta=10$, we will correctly reject $H_O$ with probability $e^{\frac{-2.995}{10}}\approx0.74$

(c). We observe the value $X=5$. Calculate p-value.
$\text{p-value}=\max\limits_{\beta\leq 1}p(W\geq 5)=\max \limits_{\beta \leq 1}e^{-5/\beta}=e^{-5}\approx 0.0067$
We would reject $H_O$ at 0.05 level, and also at any level $\alpha\geq 0.0067$

If you like my article, please feel free to donate!