SMU SOE Seminar Series (August 25, 2026): Empirical Bayes for Compound Adaptive Experiments

TOPIC:

ABSTRACT

We investigate Empirical Bayes methods in the context of compound adaptive experiments, where the arm distribution in each experiment follows a normal distribution with an unknown mean that we seek to estimate. There are two main EB strategies: $g$-modeling, which estimates the prior by maximizing the marginal likelihood, and $f$-modeling, which derives posterior means directly from the empirical distribution of the observations. We show that $g$-modeling continues to be a valid EB procedure even when it incorrectly assumes that data are collected exogenously; its validity does not depend on the particular sampling algorithm or on whether sample sizes are endogenous. In practice, one can apply standard $g$-modeling techniques by acting as though the data were exogenously sampled. Strikingly, we prove that risk guarantees established for $g$-modeling under i.i.d. sampling can be extended to data generated adaptively, without requiring any prior knowledge of the sampling rule, even when it differs across experiments. By contrast, $f$-modeling yields biased estimators. We corroborate the robustness of $g$-modeling through simulations with widely used adaptive algorithms and demonstrate its applicability using a real-world dataset consisting of multiple sequential experiments.