Comparing Exact, Mid‑p, and Wilson Score Tests for a Single Binomial Proportion

Abstract

Recent methodological studies have continued to refine and scrutinize exact binomial inference, providing comparisons between exact tests, approximate methods, and mid‑p corrections. This ongoing work reflects that even after many decades, the simple binomial problem continues to yield valuable theoretical insights. A consistent finding across studies is the inadequacy of methods based on the normal approximation. In particular, the exact binomial test, while exact in principle, is known to be highly conservative, with its actual significance level often substantially lower than the nominal level (actual significance leve  l), implying a loss of statistical power and thus a reduced ability to detect a true effect when it exists. This study compares four methods for inference on a single binomial proportion: the standard exact binomial test (using the nominal significance level ), the exact binomial test (using the actual significance level), the mid-p test, and the widely recommended Wilson score test. Using Monte Carlo simulation with  replications, we compare these tests in terms of empirical Type I error rate and statistical power across a comprehensive grid of sample sizes (  to ) and nine null success probabilities ( ). The evidence strongly indicates that the mid-p test outperforms the standard exact binomial test and the Wilson score test for the analysis of a single binomial proportion, particularly in small-to-moderate sample sizes. It achieves a Type I error rate close to the nominal level while consistently providing higher power. The use of an actual significance threshold, although a valuable theoretical exercise, offers no practical advantage over the simpler and more effective mid-p procedure. We therefore strongly recommend the adoption of the mid-p test in practice.