On this page
A benchmark can misstate a manager's relative performance if its fund universe was selected after the result was known or if failed funds disappeared from the historical sample. Selection asks who entered the comparison; survivorship asks who remained in it. For a CAIA® Level II candidate, the practical response is to reconstruct the benchmark's membership and timing before deciding whether an apparent excess return means anything.
Define the comparison before judging the manager
An alternative fund's return does not become informative merely because a provider prints an index beside it. The benchmark should represent the opportunity set that was relevant to the investor, measured over a comparable period and using comparable return conventions. A peer group of strategies with different leverage, liquidity or valuation practices answers a different question. CAIA Association's 2027 curriculum overview includes benchmarking and performance attribution within Level II risk and risk management, which signals that benchmark choice is part of the analysis rather than a decorative final step.
CAIA Association has also published a cautionary discussion of private equity peer universes: coverage and provider methods vary, and a fund can look different across comparison sets. The same basic warning applies when a candidate sees an alternative investment index with an authoritative-sounding name. Ask what population was eligible, which funds were actually observed, whether the sample was investable at the comparison date and whether the membership was fixed in advance. Those questions test the meaning of the benchmark, not just its arithmetic.
Selection bias begins at the entry door
Selection bias occurs when the method used to form a sample changes its characteristics relative to the population the analyst wants to describe. In voluntary performance databases, the choice to report can be related to the performance being measured. A manager may decide to join a database after a strong period, while a different manager may never report. Even if every reported number is accurate, the reported set can be unrepresentative. The analyst cannot infer the missing population's average simply by averaging the observed funds.
There is a second, more direct form in benchmark choice. An analyst may compare a fund with several plausible peer groups and publish only the one that makes the fund look best. The fund's return has not changed; the denominator of the comparison has. This is benchmark selection after seeing the outcome. It differs from a predefined strategy benchmark that is documented before the evaluation period. The diagnostic question is therefore temporal: when was the comparison rule chosen, and what alternatives were rejected?
Survivorship bias acts at the exit door
Survivorship bias enters when funds that close, merge or stop reporting vanish from historical calculations. If departures are related to weak performance, the surviving sample's history can look stronger than the history of the starting population. But the direction and size cannot be asserted for every dataset without examining its rules and records. A closed fund might stop reporting for more than one reason. The safe statement is that excluding it changes the population being measured and may systematically distort the result.
This is a different issue from current investability. An investor evaluating a new commitment today may care about the funds currently accessible. An analyst estimating what a past opportunity set delivered needs to retain the funds that existed at the past starting date, including those that later failed. Neither universe is universally correct. The question determines which one belongs in the denominator. A candidate should identify the decision date before claiming that an index is biased or unbiased.
An illustrative membership audit
Consider a hypothetical one-year starting group of four funds with net returns of plus 12%, plus 8%, minus 4% and minus 16%. Their equal-weighted arithmetic average is zero. Suppose the last fund closes and an index vendor later displays only the three survivors. Their equal-weighted average is about plus 5.33%. The difference of about 5.33 percentage points is created by membership, not by improved performance of any surviving fund. These are invented teaching figures, not observed fund returns or a market estimate of survivorship bias.
Suppose a separate manager returned plus 7% over the same period. Against the complete starting group, its arithmetic advantage is seven percentage points. Against the survivor-only display, it is about 1.67 points. The manager is identical in both statements. The apparent size of its edge changes because the benchmark changed. This calculation is intentionally simple; real comparisons would also need to specify net versus gross fees, cash-flow treatment, currency and whether equal weighting resembles an investor's actual exposure.
Now reverse the question. If the investor could have selected only among the three funds that remained open at the decision date, the full historical starting group might not represent that particular current choice set. You still cannot erase the failed fund from a historical claim about the starting universe. Instead, label the two analyses differently. The first is an ex ante historical cohort. The second is a current availability screen. A precise label resolves a great deal of confusion that a single headline return conceals.
Backfilled returns create a timing problem
A newly reporting fund may supply performance from before it joined a database. If that earlier history is inserted into old index values, an index that appears to have existed at a past date can contain information that was unavailable then. This is often called backfill or instant history. The remedy for a real-time comparison is to record the first date the fund was observable to the database and separate pre-entry history from later reported returns. Doing so is a timing check, not an assertion that every backfilled return is false.
Backfill, selection and survival can interact, but they are not interchangeable labels. A fund that never reports illustrates entry selection. A fund that leaves the dataset illustrates an exit or survival rule. Historical returns added after the fund enters illustrate backfill. The same fund may be involved in more than one mechanism, which makes a simple sum of three supposed bias estimates unreliable. First identify exactly which observations are missing or added, then calculate the change caused by each dataset rule.
Test whether the benchmark answers the decision
For a manager evaluation, write the intended comparison in one sentence before looking at rankings. Specify strategy, geography if relevant, period, return convention and the investor's available opportunity set. Then inspect the provider's inclusion and deletion rules. Does it retain dead funds? Are performance histories revised after initial publication? Can a manager choose whether to submit returns? Is the peer set stable across the whole period? A benchmark with clear limitations may still be useful, but the conclusion must reflect what it actually measures.
CAIA Association's discussion of private equity relative comparisons explains why a peer set may be incomplete and why comparisons across providers can diverge. Its article on alternative asset allocation likewise notes that the investable universe may be hard to know and a representative benchmark may be unavailable. These observations support caution, but they do not tell a candidate to discard every comparison. They tell the candidate to state the benchmark's scope.
Avoid the easy but unsupported conclusion
It is tempting to say that all missing funds performed badly, all survivor indices are inflated, or that any benchmark adjustment can recover the true population return. None follows automatically from the label alone. The unobserved funds are unobserved. A careful analyst can show a sensitivity range, compare alternative provider universes and report the membership assumptions, but should not claim a precise correction without the missing data. That restraint is especially useful when a short exam case offers only a description of an index construction rather than a complete return history.
The decision sequence is simple: establish the target population, identify who could enter and leave the dataset, record when information became available, and only then compare performance. The CAIA programme page locates this work in the larger course of study, while the Omni methodology page explains the preparation approach. The answer to the reader's question is a method, not a universal adjustment factor: fix the comparison's membership and timing before interpreting the manager's apparent edge.
Sources
- Explore the Curriculum | CAIA Association · caia.org · Retrieved
- Relative Comparisons in Private Equity: A Cautionary Tale | CAIA Association · caia.org · Retrieved
- Strategic Asset Allocation: Practical Considerations for Alternative Investments | CAIA Association · caia.org · Retrieved



