CAIA® operational controls: what evidence proves they work?

On this page

To evaluate a manager’s operational controls for CAIA® Level II, trace a real transaction through the people and systems that authorise, record, reconcile and resolve it. Ask for evidence that each check ran, that exceptions were escalated, and that someone independent reviewed the resolution. A policy or impressive performance history alone cannot establish that the control works.

Start with the risk the control is meant to stop

An operational review asks whether a fund can produce reliable records, safeguard assets and continue serving investors when ordinary processes fail. This is a different reader question from judging a manager’s investment skill. CAIA Association places operational due diligence within its Level II manager-selection curriculum, alongside analysis of terms, business activities and investment process. Its curriculum overview is an anchor for the scope of the topic. A strong return series does not show who can change a payment instruction or how a disputed position enters the fund’s valuation.

For a candidate, the useful unit of analysis is a control objective, an action that should achieve it, and evidence that the action occurred. Consider a cash payment: the objective is to prevent an unauthorised transfer; the action might require a separate approver; the evidence could be a dated approval record and a bank confirmation. If the manager merely says payments are reviewed, the candidate has learned the intended process but has not established that it operated. This distinction carries through every part of the infrastructure review.

Trace the trade rather than count job titles

Begin with an order and follow its lifecycle: decision, execution, booking, settlement, external confirmation, portfolio record and net asset value. At each point ask who can create a record, who can change it, and who is expected to challenge it. Separation of duties matters because a person able to create and approve the same entry can hide an error as easily as they can correct one. Distinct names on an organisation chart may still report through a common person who can override both roles.

Suppose a trader submits an order, an operations analyst books it and a controller checks the broker confirmation. The apparent split is useful, but it is incomplete if the analyst can also edit the controller’s exception queue. The candidate should look for access rights, approval history and a sample of resolved exceptions. The manager may have good reasons to grant emergency access; then the question is whether use of that access is logged, independently reviewed and limited. An absolute ban on role overlap is too crude for a small firm; an unexplained ability to self-approve is the actual weakness.

This is why a walkthrough is more informative than a policy document alone. Ask staff to demonstrate one transaction from source document to final ledger entry. Compare their description with the written procedure. Check whether the time stamps make sense and whether the reviewer had access to independent information. If a manager cannot reproduce the chain for a sampled trade, the due-diligence conclusion should say exactly which link is unverified rather than call the entire operation either good or bad.

Make reconciliation an evidence question

A reconciliation compares an internal record with a separate source, such as a custodian or broker statement. Its purpose is not to make two files match by force; it is to identify a difference and determine why it exists. Evidence should reveal the date of comparison, the source used, the person who performed the work, the unresolved items and the path to closure. A checklist with every box ticked but no record of exceptions can be less persuasive than a log that shows a problem was found, investigated and corrected.

Frequency has to fit the activity and exposure. Fast moving cash and positions often warrant more frequent checks than a less active process, but the right interval depends on how quickly an error could cause harm and whether another control catches it. A candidate should avoid treating a frequency label as a universal pass mark. Independence also has degrees: a reviewer outside the investment team is helpful, yet one who can edit the source ledger may still be marking their own work. The decisive question is whether the comparison can expose a discrepancy that the person creating the record could not silently erase.

Break management is the part candidates often skip. Ask how long open items remain unresolved, which types trigger escalation, and whether aged breaks appear in management reporting. A small discrepancy repeatedly rolled forward can indicate a systematic data problem. A large one closed quickly may still warrant a root-cause review if it arose from an unauthorised change. The analyst should connect severity, duration and cause rather than judge only the count of open items.

An illustrative control walkthrough

Imagine an invented credit fund with a round-number portfolio of £100 million. On a sampled day, its internal cash ledger shows £10 million while an independent bank statement shows £9.9 million. The difference is £100,000. The operations team labels it as an unsettled fee payment; the controller checks the payment instruction and bank settlement date, and a separate reviewer signs the closure after the bank debit appears. All amounts and roles here are illustrative teaching inputs, not observed fund data.

The arithmetic is simple, but the judgment is not. First confirm that the two balances refer to the same account and cut-off time. Then establish whether the £100,000 difference is a timing item or an erroneous entry. Next ask who could authorise the payment and whether that person could alter the ledger or approve its own reconciliation. Finally inspect the closure evidence: a bank debit and independent sign-off are stronger than a note saying “resolved”. If the payment never settled, the fund’s cash and possibly its valuation remain misstated. An exam answer should identify the evidence needed before accepting the manager’s explanation.

Now change only one fact: the controller who books fees can delete the exception and mark it closed without a second review. Even if the cash balances ultimately agree, the control design is weaker because the same person can cause and conceal the break. The candidate should propose a specific remediation, such as restricting closure rights or requiring independent review of changes, then ask for a later sample showing the new control in use. The manager’s promise to fix it is a plan, not proof of completed remediation.

Systems and continuity need proof of use

Technology should fit the volume and complexity of the strategy. A spreadsheet may be adequate for a limited task with version control and independent checks; it is a poor substitute for controlled position records when many people can overwrite formulas without trace. Ask how trade, cash and valuation records move between systems, which fields are entered manually, and what happens when an interface fails. An automated reconciliation can still be wrong if both sides are fed by the same defective source.

Cybersecurity and business continuity are also operational controls. Access lists, removal of departed users, multi-factor authentication, backup restoration records and incident exercises offer different pieces of evidence. A written continuity plan tells the reviewer what should happen; a dated test and its follow-up show whether the plan survived contact with reality. The candidate should separate the existence of a plan from the ability to resume critical tasks, including payment approval and investor reporting, during a disruption.

CAIA Association’s discussion of manager selection warns that operational weaknesses in areas such as compliance and valuation can undermine an otherwise attractive manager assessment. That supports treating control findings as substantive, while avoiding a mechanical rule that every weakness has the same consequence. Severity depends on the process at risk, compensating checks, evidence of past failures and the manager’s response. A missing control over asset movement is more consequential than a cosmetic procedure update.

Turn findings into a defensible conclusion

A sound recommendation states the objective, the evidence observed, the gap, the possible consequence and the action needed. “Reconciliation is weak” is too vague. “The person posting cash entries can close the matching exception without independent sign-off, so a false entry could remain undetected; restrict closure rights and sample subsequent approvals” is testable. It lets a reader distinguish an unresolved concern from a documented remediation. Do not infer effectiveness from confidence in the investment team or from the absence of a known loss.

When revising, practise giving a short judgement for each control layer: people and authority, transaction records, reconciliation, technology access, incident response and continuity. Then ask what observation would overturn your first impression. That question prevents a candidate from accepting a polished questionnaire at face value. The CAIA programme page gives the programme context, while the Omni methodology page explains the broader study approach. For this reading, the examinable habit is evidence-led evaluation: identify exactly what has been tested and what remains an assertion.