Self-reassurance that p-value properties don’t depend on regularity assumptions on the test statistic.
Let \((\Omega, \mathcal E, \text{Pr})\) be a probability space, where \(\Omega\) is the space of random outcomes, \(\mathcal E\) the \(\sigma\)-algebra of measurable events, and \(P\) the probability measure.
Given a random variable \(T\colon \Omega \to \mathbb R\), define \(p_T\colon \Omega \to \left[0,1\right]\) as:
\[ p_T(\omega) = \text{Pr}(\{\omega'\in \Omega\,\vert\, T(\omega')\geq T(\omega)\}) \]
Theorem. \(p_T\) is measurable and \(\text{Pr}(p_T\leq \alpha) \leq \alpha\) for all \(\alpha \in \left[0,1\right]\). Equality holds if and only if there exists a sequence \(\{\omega_n\}_{n\in \mathbb N}\) such that \(p_T(\omega_n) \leq \alpha\), and \(p_T(\omega _n)\to \alpha\) as \(n \to \infty\).
Proof. Let \(\alpha\in\left[0,1\right]\), and denote: \[ E_T(\omega) = \{\omega'\in \Omega\,\vert\, T(\omega')\geq T(\omega)\}, \] so that \(p_T(\omega) = \text{Pr}(E_T(\omega))\).
Assume first that there exists \(\omega_\alpha \in p_T^{-1}(\alpha)\), that is to say \(\text{Pr}(E_T(\omega)) = \alpha\). We can show that:
\[ N_T(\omega_\alpha) = \{\omega \vert p_T(\omega) \leq \alpha\} \backslash E_T(\omega_\alpha) \] is a measurable, zero probability set, which proves the thesis for this particular case. As a matter of fact, for any \(\omega \in \Omega\), if \(p_T(\omega)\leq \alpha\) and \(T(\omega) <T(\omega_\alpha)\), then we must have:
\[ \text{Pr}(\{\omega'\in \Omega\,\vert\, T(\omega_\alpha)>T(\omega')\geq T(\omega)\} ) = p_T(\omega) - \alpha=0. \] If \(t_* = \inf_{p_T(\omega)\leq \alpha}T(\omega)\) and \(\{a _n\}_{n \in \mathbb N}\) is a sequence in \(\Omega\) such that \(T(a _n)\to t_*\) as \(n\to \infty\), then:
\[ N_T(\omega _\alpha) \subseteq \cup _n \{\omega'\in \Omega\,\vert\, T(\omega_\alpha)>T(\omega')\geq T(a_n)\}, \] the right hand side being a probability zero set.
If \(p_T^{-1}(\alpha)\) is empty, let \(\alpha^* = \sup _{p_T(\omega)\leq \alpha}p(\omega)\), and let \(\{b _n\}_{n\in \mathbb N}\) be a sequence in \(\Omega\) such that \(p_T(b_n)\to \alpha^*\) as \(n\to \infty\). Then:
\[ \{\omega \vert p_T(\omega) \leq \alpha\}= \{\omega \vert p_T(\omega) \leq \alpha^*\}= \cup _n \{\omega \vert p_T(\omega) \leq p_T(b_n)\}, \] so that, from the particular case proved earlier, we have:
\[ \text{Pr}(p_T \leq \alpha) = \lim _{n \to \infty} \text{Pr}(p_T \leq p_T(b_n)) \leq \lim _{n \to \infty} p_T(b_n) = \alpha ^* \leq \alpha, \] as was to be proved.
If you see mistakes or want to suggest changes, please create an issue on the source repository.
Text and figures are licensed under Creative Commons Attribution CC BY-SA 4.0. Source code is available at https://github.com/vgherard/vgherard.github.io/, unless otherwise noted. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
For attribution, please cite this work as
Gherardi (2023, June 7). vgherard: p-values and measure theory. Retrieved from https://vgherard.github.io/posts/2023-06-07-p-values-and-measure-theory/
BibTeX citation
@misc{gherardi2023p-values,
author = {Gherardi, Valerio},
title = {vgherard: p-values and measure theory},
url = {https://vgherard.github.io/posts/2023-06-07-p-values-and-measure-theory/},
year = {2023}
}