EXACT P-VALUES FOR DISCRETE MODELS OBTAINED BY ESTIMATION AND MAXIMIZATION

Abstract

In constructing exact tests from discrete data, one must deal with the possible dependence of the P-value on nuisance parameter(s) ψ as well as the discreteness of the sample space. A classical but heavy-handed approach is to maximize over ψ. We prove what has previously been understood informally, namely that maximization produces the unique and smallest possible P-value subject to the ordering induced by the underlying test statistic and test validity. On the other hand, allowing for the worst case will be more attractive when the P-value is less dependent on ψ. We investigate the extent to which estimating ψ under the null reduces this dependence. An approach somewhere between full maximi..