Fundamental Statistics Learning Note(16)

LR-test continued

Continued:
If LR test statistics $\lambda$ has an unknown distribution, we use large-sample approximate LR test.
Reject $H_O$ if $-2log\lambda \geq (1-\alpha)$ quantile of $\chi_p^2$, where $p$ is difference in # free parameters(between $\Omega_0\ and\ \Omega_0 \cup \Omega_1$)

Ex. $x_1,\dots,x_n \overset{\text{iid}}\sim N(\mu_x,\sigma_x^2), y_1,\dots,y_n \overset{\text{iid}}\sim N(\mu_y,\sigma_y^2)$, what is the value of p for the approximate LR test?
$$\begin{matrix}
H_O:\mu_x=\mu_y=0, \sigma_x^2=\sigma_y^2=1\text{ vs } H_A:otherwise && p = 4-0 \\
H_O:\mu_x=\mu_y=0 \text{ vs } H_A:otherwise && p = 4-2 \\
H_O:\mu_x=\mu_y \text{ vs } H_A:\mu_x\neq\mu_y && p = 4-3
\end{matrix}$$

Ex. $x_1,\dots,x_n \overset{\text{iid}}\sim Pois(\theta)$. Derive approximate LR test for $H_O:\theta=\theta_0$ vs $H_A:\theta \neq \theta_0$(when n is large)

$$\begin{equation}\begin{split}
L(\theta|x_1,\dots,x_n)& = \prod \limits_{i=1}^n e^{-\theta}\frac{\theta^{x_i}}{x_i!} \\
& = e^{-n\theta}\frac{\theta^{\sum x_i}}{\prod x_i!}
\end{split}\end{equation}$$
Then LR test $\lambda$ is:
$$\begin{equation}\begin{split}
\lambda&= \frac{\max\limits_{\theta=\theta_0}L(\theta|x_1,\dots,x_n)}{\underbrace{\max\limits_{\theta>0}L(\theta|x_1,\dots,x_n)}_{\text{MLE is }\hat \theta = \bar x}} \\
&=\frac{e^{-n\theta_0}\theta_0^{\sum x_i}/\prod x_i!}{e^{-n\bar x}\bar x^{\sum x_i}/\prod x_i!} \\
& = \underbrace{e^{-n\theta_0+n\bar x}(\frac{\theta_0}{\bar x})^{n\bar x}}_{\text{is actually a value given data}}
\end{split}\end{equation}$$
Though we can’t easily tell the distribution of $\lambda$, so approximate $-2log\lambda \overset{\text{approximate}}\sim \chi_1^2$.
Then approximate LR test: reject $H_O$ if $-2log\lambda \geq (1-\alpha)\text{ quantile of }\chi_1^2$

p-values

Defnition: Let $W$ be a test statistics for $H_O: \theta \in \Omega_0$.
Data: $ \widetilde{X} = X_1,\dots,X_n$ and observed value $\tilde{x} = x_1,\dots,x_n$.
If $W$ is larger when $H_O$ is false: p-value=$\max\limits_{\theta\in\Omega_0}P(W(\widetilde{X})\geq W(\tilde{x})|\theta)$
If $W$ is smaller when $H_O$ is false: p-value=$\max\limits_{\theta\in\Omega_0}P(W(\widetilde{X})\leq W(\tilde{x})|\theta)$
Note: $\widetilde{X}$ is random variable and $\tilde{x}$ is observed value.

E.X. $x_1,\dots,x_n \overset{\text{iid}}\sim Bern(p). H_O: p\leq0.4\text{ vs }H_A: p\geq 0.4$.
$W=\sum \limits_{i=1}^4 X_i$ is test statistics.
Suppose data: $x_1=x_2=x_3=1, x_4=0$, what is the p-value?
$E(W)=4p$, $W$ is larger when $H_O$ is false.
$$\Rightarrow W\sim Bin(4,p)$$
$$\begin{equation}\begin{split}
\text{p-value}&=\max\limits_{p\leq 0.4}[p(W=3)+P(W=4)] \\
&=\max\limits_{p\leq 0.4}[{4 \choose 3}p^3(1-p)^1+{4\choose4}p^4(1-p)^0] \\
&=4(0.4)^3(0.6)+0.4^4=0.1792
\end{split}\end{equation}$$
Note: when $p=0.4$, this makes data most likely in the parameter space ($p\leq 0.4$).

If you like my article, please feel free to donate!