CAIA: Statistical inference

Explanations, applications and practice for statistical inference.

• Bonferroni thresholds and the family error bound

  Divide a family error budget across valid individual tests.

  Formulas

• Assuming independence in every multiple-test screen

  Inspect the product step rather than treating 1 − (1 − alpha)^m as universal.

  Common mistakes

• Family-wise error and false discovery rate

  Identify whether a method controls any error or an expected fraction among rejections.

  Concepts

• False-rejection counts and family error risk

  Read two different summaries of a fictional ten-test experiment.

  Worked examples

• Halving every two-sided p-value

  Check the statistic sign and the planned alternative before using a tail shortcut.

  Common mistakes

• Mean confidence intervals and prediction intervals

  Distinguish estimating the population mean from predicting a new observation.

  Comparisons

• Reconciling a mean interval with a two-sided test

  Use the same model and uncertainty inputs in both calculations.

  Worked examples

• A mean test with known population standard deviation

  Reconstruct the standard error, standardized statistic and two-sided decision.

  Worked examples

• Error probability across independent tests

  Use the complement calculation with its independence and true-null assumptions.

  Formulas

• P-values and the null model

  Read the conditioning statement before interpreting a small probability.

  Concepts

• A one-sided test with an opposite-sign statistic

  Use the planned tail when the estimate points away from the alternative.

  Worked examples

• Treating a p-value as the probability of the null

  Repair the reversal from a model-conditional tail area to a hypothesis probability.

  Common mistakes

• Practice: tails, uncertainty and error budgets

  Check test direction, independent-mean precision and family corrections with explained answers.

  Practice

• Sample size and mean-estimation precision

  Calculate the sample-size change needed to halve standard error under a fixed-variance model.

  Worked examples
• 

  Statistical test research log

  Download a structured CSV for hypotheses, selection, uncertainty and family treatment.

  Templates

• Sample standard deviation versus standard error

  Explain which variation each statistic describes before comparing magnitudes.

  Concepts

• Standard error of an independent mean

  Distinguish uncertainty of a mean from variability of individual observations.

  Formulas

• Statistical significance and effect size

  Read magnitude and uncertainty before assigning practical meaning to a threshold crossing.

  Concepts

• Normal-test p-values by direction

  Calculate upper, lower and two-sided tails for a standard-normal statistic.

  Formulas

• Designing and reporting a hypothesis test

  Connect the research question, null model, test direction and reported result.

  Lessons