Abstract

Classical methods for multiple testing try to avoid false positive conclusions at all cost. Modern methods based on the False Discovery Rate (FDR), popular in omics data analysis, prefer merely to limit the proportion of false positives among the findings. This is a sensible strategy, as it gives sufficient control over the potential flood of unreliable findings, while simultaneously avoiding too many false negative results. What many users do not realize, however, is that these FDR -based methods allow much less flexibility in the way the results of these methods can be used. For example, simple acts of selection among results or aggregation (e.g. from probe level to gene level) may dramatically increase the proportion of false positives. In this talk I propose an alternative way of controlling or estimating the proportion of false positives that allows greater flexibility for the user in later post-processing of the results, e.g. using biological knowledge or bioinformatics tools, while retaining statistically guaranteed properties. This method is based on a combination of closed testing and the Simes inequality. Special attention is given to the computational challenge of performing closed testing in situations with hundreds or thousands of hypotheses.