Skip to content

Assuming independence in every multiple-test screen

Correlated error events can make the independent-test family probability an inappropriate calculation.

On this page

Locate the assumption in the multiplication

The chance that all tests avoid errors equals a product of marginal no-error probabilities only under the stated independence assumption. Reusing the same data or applying related specifications may create dependence. The number of tests alone does not settle it.

Compare perfectly linked error events

Consider ten true-null tests whose error events are identical: either all reject incorrectly or none do. If that single event has probability 0.05, the family chance of at least one false rejection is 0.05. The independent model would give about 0.4012631. Both scenarios have expected false-rejection count 0.5.

Use a bound when only marginal sizes are known

The union bound gives family risk at most the sum of individual error probabilities. For ten tests each bounded by 0.05, that bound is 0.5. It is an upper bound, not a claim that the actual probability equals 50%.

Report assumptions with any family-risk number

Required labels
Reported quantityNecessary context
Exact independent family probabilityTrue nulls, event sizes and independence
Expected false-rejection countMarginal false-rejection probabilities
Union upper boundValid marginal bounds and defined family

Further references