Fundamental Statistics Learning Note(22)

Rcall posterior is a compromise between prior and likelihood
i.e. Bayesian analysis provides framework to update posterior based on data/evidence (e.x. college football rankings)

Bayesian Hypothesis Testing

Consider $H_O: \theta \in \Omega_0 \text{ vs } \theta \in \Omega_1$
Since $\theta$ is a r.v under Bayesian inference:

  • the prior probability that $H_O$ is true is $P(\theta\in \Omega_O)$, calculated using $\pi(\theta)$.
  • the posteriro probability that $H_O$ is true is $P(\theta\in \Omega_O|x_1,\dots,x_n)$, calculated using $\pi(\theta|x_1,\dots,x_n)$.

Definition: The Bayes factor in favour of $H_O$ is determined by $$\underbrace{\frac{P(\theta\in\Omega_0|x_1,\dots,x_n)}{P(\theta\in\Omega_1|x_1,\dots,x_n)}}_{\text{posterior odds that }H_O\text{ is true}}=\underbrace{\frac{P(\theta\in \Omega_0)}{P(\theta\in\Omega_1)}}_{\text{prior odds that }H_O\text{ is true}}\times \text{Bayes Factor}$$

Intepretation:
$$\begin{matrix}
\text{B.F }\geq 1 & \text{data support }H_O \\
\text{B.F }< 1 & \text{data support }H_A
\end{matrix}$$
Ex. $x_1,\dots,x_n \overset{\text{iid}}\sim Bern(p)$. $H_O: p\leq 0.5\text{ vs }H_A: p>0.5$. Suppose prior on $p$ is $p\sim Beta(1,2)$.
(a) prior probability $H_O$ is true?
$$\begin{equation}\begin{split}
P(p\leq 0.5)=\int_0^{0.5}\pi(p)dp &=\int_0^{0.5}\frac{\Gamma(3)}{\Gamma(1)\Gamma(2)}p^{1-1}(1-p)^{2-1}dp \\
&=\int_0^{0.5}2(1-p)dp \\
&=2p-p^2|_0^{0.5} \\
&=1-0.5^2 \\
&=0.75
\end{split}\end{equation}$$

(b) Observe 5 successes out of 5, what is the posterior probability $H_O$ is true?
Given $\pi(\theta|x_1,\dots,x_n) \sim Beta(1+5,2+0)$ from last chapter.
$$\begin{equation}\begin{split}
P(p\leq 0.5) &= \int_0^{0.5}\frac{\Gamma(8)}{\Gamma(6)\Gamma(2)}p^{6-1}(1-p)^{2-1}dp \\
&=\int_0^{0.5}42p^5(1-p)^1dp \\
&=42[\frac{p^6}{6}-\frac{p^7}{7}]|_0^{0.5} \\
&=0.0625
\end{split}\end{equation}$$

(c) prior odds $H_O$ is true?
$\frac{0.75}{1-0.75}=3$

(d) posterior odds $H_O$ is true?
$\frac{0.0625}{1-0.0625}=0.0667$

(e) Bayes factor
$0.0667=3\times B.F$
$\Rightarrow B.F = 0.0222 \ll 1$
Data strongly supports $H_A: p>0.5$

If you like my article, please feel free to donate!