Make AI Show Its Evidence Before It Shapes a Family Decision

AI can help a family office find information, draft a first pass and flag records that deserve attention. It cannot make a payment authorized, turn an old valuation into a current one, or decide what a trust document permits. The useful question is not simply where can we use AI? It is: which task can the office test against known records, under the right permissions, before anyone acts on the answer?

Start with a narrow, reversible workflow such as searching approved documents or preparing an exception list. Give a named employee responsibility for the output. Require the tool to point to the exact source and version. Keep decisions, official books, client communications and transactions on their existing approval routes until a measured pilot justifies a specific change.

01Choose the work before choosing the model

“AI for family offices” covers very different activities. Northern Trust discusses investment, leadership and operational uses, while PwC US describes research, diligence, cash-flow analysis and reporting support. Those are possible uses, not evidence that one tool should receive every family document or perform every function.

Before choosing, note how often the task occurs, the current time to finish it, whether the source set is complete, the cost of a wrong answer, who can review it and how staff will do it if the tool fails. A rare task with messy records may be a worse first pilot than a frequent, checkable one.

Sort a proposed task by what its output can cause:

The first two rows can still carry material risk. A search result that omits an amendment can mislead a decision-maker. A report draft can silently combine figures from different dates. Choose a pilot because its boundaries and errors are observable, not because its label sounds “low risk.”

Four candidate pilots call for different evidence. Document search needs a controlled signed-file inventory and a counsel or operations reviewer; test a missing amendment before relying on a clause summary. Statement exception flagging needs dated statements and authoritative entity records; a controller tests whether the tool spots a known mismatch without posting a correction. Report drafting starts from reconciled figures; the report owner checks dates, labels and omissions before release. Cash-flow support needs actual entity cash, known commitments and settlement dates; the treasurer tests whether a forecast keeps paying entities separate. A wrong document answer can misdirect legal preparation; a wrong cash forecast can cause a missed obligation. Start where the records, reviewer and manual baseline are ready, not where a vendor demo looks most polished.

Keep the adjacent investment job separate. Our AI in investment management guide tests whether a proposed signal survives data, model and trading scrutiny. Here the office is testing operational answers and document handling, including who may see a record and what a reviewer can prove from it.

02Set the source, permission and approval boundaries

Before uploading documents, identify the exact records an AI system may use. An office may have signed documents in counsel's files, current versions in a document system, email attachments, drafts on personal drives and scanned amendments that search cannot read. “Search all files” is not a reliable scope until someone has reconciled that inventory.

For each pilot, record five boundaries:

• Source: the approved repository, included document types, versions and owner of the inventory.
• People: which users can ask, view retrieved passages, edit the corpus and approve the answer.
• Use: the question the tool may answer and what it may not decide or trigger.
• Record: the prompt, source references, output, reviewer correction and final decision trail the office will retain.
• Fallback: the manual path if a document is absent, the model is unavailable or the result cannot be verified.

An access control must apply to retrieval as well as the final answer. A user who cannot see a counsel-restricted file should not receive its text through AI search. A tool should say its answer is limited to the files the user is authorized to search; it should not pretend to have checked the whole office archive.

Morgan Lewis's family-office AI analysis emphasizes approved tools, assigned ownership, secure infrastructure and human oversight. NIST's voluntary AI Risk Management Framework offers a way to govern, map, measure and manage AI risks. Neither source makes one configuration automatically safe for every family. Counsel, privacy and security owners should review actual documents, users, services and contracts.

03Test one answer against the complete approved pilot set

Consider a fictional office with twelve signed private-fund side letters and two amendments. A family representative asks: “Which of these approved side letters and amendments contain transfer notice or consent language, and where?” The bounded task is document retrieval and issue spotting. It cannot establish every transfer obligation: fund agreements, subscription documents and other governing records may contain additional terms. Counsel must review the full applicable document set before advising on a transfer.

The office first creates a 14-file inventory for an authorized counsel reviewer, with each fund, document type, signing date, version and access owner. One amendment is a scan the initial index missed. Before repair, counsel's expected 14-file set has only 13 indexed files. For this fictional pilot, the office sets a stop rule before testing: a missing signed amendment or unauthorized disclosure blocks expansion. A repaired answer alone cannot release the pilot; a new held-out sample and comparable total-effort measure must also pass.

The other amendment is included in the 14-file counsel inventory but restricted from an operations user's view. The operations answer must be limited to that user's permitted subset. Counsel confirms which version governs. The system must show the document and clause behind each proposed answer, not merely a confident sentence.

This case separates three tests that are often collapsed into “accuracy.” Retrieval asks whether the right current files were available. Interpretation asks whether the answer reflects the cited text and its conditions. Authorization asks whether that user may see the text and rely on the answer for this task. A model can pass one and fail another.

The acceptance record should list each fund, file and version in this bounded search, relevant passage, missing or conflicting item, reviewer, resolution and next action. Keep the results distinct:

• Unknown—source missing: an expected file is unreadable or unindexed.
• Restricted—refer to authorized reviewer: a counsel-only test status when a role lacks access; do not reveal the file or status to that user.
• No clause found in reviewed scope: the permitted source set is complete, but the search found no matching text there.
• Interpretation pending: the cited text's legal meaning is contested.

None is equivalent to “no notice required.” Counsel uses all applicable governing documents for that legal conclusion.

This fictional closeout sheet makes the outcome and untested work visible together. It is an example of how to record a pilot, not evidence that a product or family office achieved these results.

The 25-minute scan repair may be partly one-time. A retest should compare equivalent questions over an agreed evaluation window, including reviewer time and ongoing corpus maintenance, and should account for setup or repair cost on the same basis. A faster later run would not offset a material missed condition or access leak.

04Make the pilot measurable before access expands

Select a sample that contains ordinary and difficult cases: clean PDFs, scans, amendments, conflicting drafts, permission-restricted files and questions with no answer in the available records. Have an authorized reviewer establish the expected source set and answers before looking at the AI result. Otherwise the tool's fluency can become the answer key.

For each case, record:

There is no universal pass percentage in this table. A trivial formatting error and a missed consent condition have different consequences. The office should set its tolerance, sample and decision owner before the test; for restricted-data leakage, a single incident warrants containment rather than averaging it into an accuracy score. Record failed examples alongside successes so a later team can see what the pilot did not establish.

For permission testing, an authorized reviewer should also try direct source links, snippets, exports, conversation history and a revoked user's cached results after access changes. The user-facing reply must not expose restricted titles, counts or content. Keep permission expansion blocked until those paths have been checked and logged; the fictional single-query pass above does not cover them.

Measure cost on the same basis. Include document cleanup and indexing, security review, vendor fees, reviewer time, training and ongoing exception handling. If retrieval is faster but staff spend longer checking unreliable answers, the pilot has not demonstrated a useful time saving. A narrower search workflow or better document inventory may be the real improvement.

05Examine the vendor contract and the tool's reach

A product demonstration can show a polished answer while leaving the office's most important questions unanswered. Ask each finalist to show the same permissioned corpus and error cases, then obtain written terms for the actual deployment:

• Which files, prompts, outputs and logs leave the office's environment? Where are they stored, for how long, and who can access them?
• May the vendor or its subprocessors use customer data to train or improve models? What do the contract and configuration actually say?
• Can permissions mirror the office's source system, including revocation and counsel-only material? What happens to cached results after access changes?
• Does every material answer retain a source link, version and activity log? Can reviewers export the record and correct the indexed corpus?
• What actions can the tool take in connected systems? Can read, write and approval permissions be separated and disabled?
• What are the incident, retention, deletion, export and exit terms? Who provides a working fallback when a service fails?

Do not assume every public AI tool trains on uploaded files, or that an “enterprise” label settles retention and access. Terms vary by vendor, plan, setting and agreement. The office must inspect the specific product and contract it intends to use. An outside provider's confidentiality promise also does not establish that the family's own document permissions are correct.

Keep the first pilot read-only where possible. If a later integration can draft a ledger entry or payment instruction, route it through the existing preparer, reviewer and signer permissions. The AI's proposed value and the official posted value should remain distinguishable in the audit trail. A retrieved answer is not a delegated signature.

06Decide what is ready, and what stays human

Close the pilot with a decision record, not a product opinion. State the approved use case and users, the corpus/version boundary, sample and material failures, measured reviewer effort, contract limits, information-security findings, human release owner and date for another review.

Expand only a task whose source coverage, access and answer checks have met the office's pre-set conditions. Repair and retest a task with a missing file, stale index or answer that omits a governing condition. Stop a task after unauthorized disclosure or an action outside approved permissions until the incident and control are resolved. Keep manual review for decisions whose evidence or authority the system cannot establish.

A family office can use AI to reduce searching and drafting work without asking it to become the family's lawyer, controller, investment committee or signer. The test is whether the office can show what the answer came from, who checked it, what remained uncertain and who made the decision.