Skip to content

False-rejection counts and family error risk

Ten independent true-null tests at exact 5% size have expected count 0.5 and about 40.1263% probability of at least one error.

On this page

State a complete synthetic experiment

Imagine ten true null hypotheses and ten independent error events, each with rejection probability 0.05. This experiment is designed for arithmetic clarity; ten signals tested on the same return history need not have independent errors.

Calculate the expected number

Write the count as the sum of ten indicators, one per false rejection. Each indicator has expectation 0.05, so the total expectation is 10 × 0.05 = 0.5. An expectation of half an error is an average over repeated experiments, not a possible half-rejection in one experiment.

Calculate at least one error

The no-error probability is 0.95^10 ≈ 0.5987369. Its complement is approximately 0.4012631. The outcome V = 0 has positive probability, while other experiments can contain more than one error.

Different questions
QuestionResult
How many false rejections on average?0.5
Chance of at least one false rejection?About 40.1263%
Chance of no false rejections?About 59.8737%

Keep the stated population of nulls visible

When some null hypotheses are false, total rejections include potential true discoveries and false rejections. Counting every rejection as an error then becomes incorrect. Record which mathematical scenario is being discussed before applying the all-true-null illustration.

Further references