Fundatmental Statistics Learning Note(End)

Some Exercises

Let $x_1,\dots,x_n \overset{\text{iid}}\sim N(\mu_x,1)$, $y_1,\dots,y_n \overset{\text{iid}}\sim N(\mu_y,1)$, and both are independent.
Derive LR test for $H_O:\mu_x=\mu_y \text{ vs } H_A:\mu_x\neq\mu_y$. <

$$\begin{equation}\begin{split}
L(\mu_x,\mu_y|x_1,\dots,x_n,y_1,\dots,y_n) & = f(x_1,\dots,x_n,y_1,\dots,y_n|\mu_x,\mu_y) \
& \overset{\text{indep}}=\prod_{i=1}^nf(x_i|\mu)\prod_{j=1}^m f(y_j|\mu_y) \
& = \prod_{i=1}^{n}\frac{1}{\sqrt{2\pi}}e^{-\frac{(x_i-\mu_x)^2}{2}}\cdot \prod_{i=1}^{n}\frac{1}{\sqrt{2\pi}}e^{-\frac{(y_i-\mu_y)^2}{2}}
\end{split}\end{equation}$$
$\Rightarrow (\frac{1}{2\pi})^{\frac{m+n}{2}}e^{-\frac{\sum(x_i-\mu_x)^2}{2}}e^{-\frac{\sum(y_i - \mu_y)^2}{2}}$

$x_1,\dots,x_n,y_1,\dots,y_n \overset{\text{iid}}\sim N(\bar \mu, 1)$. all $x,y$ are iid with same unknown mean, so maximized by $\hat \mu = \frac{\sum x_i + \sum y_i}{n+m}$ (Overall average)

$$\begin{equation}\begin{split}
LR & = \frac{\max\limits_{\mu_x=\mu_y}L(\mu_x,\mu_y|\mathbf{x},\mathbf{y})}{\max\limits_{\mu_x,\mu_y}L(\mu_x,\mu_y|\mathbf{x},\mathbf{y})} \
& = \frac{(\frac{1}{2\pi})^{\frac{m+n}{2}}e^{-\frac{\sum(x_i-\hat x)^2}{2}}e^{-\frac{\sum(y_i - \hat y)^2}{2}}}{(\frac{1}{2\pi})^{\frac{m+n}{2}}e^{-\frac{\sum(x_i-\bar x)^2}{2}}e^{-\frac{\sum(y_i - \bar y)^2}{2}}}
\end{split}\end{equation}$$
where $\hat \mu_x = \bar x, \hat \mu_y = \bar y$.

To get approx LR test, $-2logLR \overset{\text{approx}}\sim \chi_{2-1}^2$, reject $H_O$ if $-2logLR > (1-\alpha)$ quantile of $\chi_1^2$

Let $x$ has PDF $f(x|\theta)=\frac{2x}{\theta^2}, 10$
Set prior $\pi(\theta) \propto \frac{1}{\theta^2}$, derive posterior PDF pf $\theta|x$
$\pi(\theta|x)\propto \underbrace{\frac{2x}{\theta^2}I(\theta>x)}_{\text{likelihood}}\cdot \underbrace{\frac{1}{\theta^2}}_{\text{prior}} = \frac{2x}{\theta^2}I(\theta>x)$

$\Rightarrow \int_x^{\infty}\frac{2x}{\theta^2}d\theta = -\frac{2x}{\theta^2}|_x^\infty = \frac{2}{3x^2}$
Hence, the valid PDF is $\pi(\theta|x) = \frac{3x^2}{2}\cdot \frac{2x}{\theta^2}I(\theta>x)\overset{\text{set}}=1$

If you like my article, please feel free to donate!