Skip to content

Family-wise error and false discovery rate

Family-wise error probability and false discovery rate summarize different features of repeated research outcomes.

On this page

Define counts before probabilities

Let V be the number of false rejections and R the total number of rejections. Family-wise error rate is P(V ≥ 1). False discovery proportion is V/R when R is positive and zero when R is zero. False discovery rate is the expectation of that proportion.

Compare two fictional outcomes

Outcomes with a false rejection
OutcomeVRAny false rejection?False discovery proportion
One false result out of one rejection11Yes1
One false result out of twenty rejections120Yes0.05

Keep a guarantee separate from one outcome

A procedure-level bound on an expectation is not a certificate for the fraction in one realized report. Likewise, a bound on the probability of any error does not reveal which individual result is false. Both targets need a specified testing procedure and its assumptions.

Inspect the all-true-null special case

If every tested null is true, any rejection is false. Then V = R and the false discovery proportion is one whenever R > 0, otherwise zero. In that special case, its expectation equals the probability of at least one rejection. The two targets differ more generally when genuine effects are possible.

Further references