In a recent post I argued that a board overseeing AI needs a short, regular report, and that directors do not need to be AI experts to read it. I also said they need basic grounding to know which questions that report should provoke, and that this grounding deserved its own discussion. This is that discussion.
The usual response to “the board should understand AI” is to schedule a training session. Often that session explains how technology works: models, training data, the vocabulary of the field. Directors can leave with more terms and no clearer sense of what to ask at the next meeting. The missing piece is rarely technical knowledge. It is a handful of ideas that change how a director reads a proposal or a report.
In my view, that handful is small. A director does not need to know how a model is built, but in the same way they do not need to know how a core system stores a transaction. What they need is to hold a few ideas firmly enough that the questions surface on their own, without someone prompting them.
Below are the four I would start with. They are not a curriculum, and they are not the complete list. They are the ones I think most change what gets asked in the room, and each comes with the question it should provoke.
An answer is not a fact
Most AI systems in use today produce estimates: a score, a classification, a drafted text, a recommendation. They are built from patterns in past data and return the most plausible output, not a verified one.
That distinction sounds academic until it meets a board pack. The risk score shown with two decimals looks precise. A summary written in clean, confident prose looks reliable. Neither appearance says anything about whether the output is right. With generative systems in particular, fluency and accuracy are separate qualities, and the system gives no signal when they diverge.
The grounding here is simple: every AI output carries an error rate, whether anyone has measured it. A proposal that cannot describe that rate is asking the board to accept an unknown.
The question it should provoke
How often is this wrong, how was that measured, and how would we know if it got worse?
Evidence has a date
Performance results describe how a system behaved on the data used to test it; at the time it was tested. They are photographs, not a guarantee. Customers change their behaviour, products change, markets move. A system that was accurate under last year’s conditions can become less accurate without any visible failure, because it keeps producing outputs with the same apparent confidence.
For a board, the risk is anchoring. Approval tends to rest on the evidence presented at that moment, and months later the same evidence can still be quoted as if it described the present, sometimes in the very material meant to supervise the system.
The question it should provoke
When was this last checked against current conditions, and what would trigger a check before the next scheduled one?
Explanation comes in degrees
“Can we explain it?” is often answered with a yes or a no. In practice, explanation sits on a range. Some systems can show which factors weighed most in each result. Others can show only what went in and what came out. With some, the honest answer is that the reasoning behind an individual output cannot be fully reconstructed.
None of these is disqualifying by itself. What matters is the match between the explanation available and the decision being made. An internal forecast used as one input among several may tolerate very little explanation. A decision that a customer, an auditor or a supervisor can ask about requires much more. The board does not need to judge the technique. It needs to judge the match.
The question it should provoke
For this decision, what could we tell the person affected, or a supervisor, about why the result came out as it did, and is that enough?
Accountability stays where it is
Much of the AI inside a regulated entity is likely to arrive from outside: embedded in software already licensed, supplied as a vendor model, or delivered by a service provider. That changes who builds and maintains the system. It does not change who answers for the outcome. A contract can allocate costs and obligations between the parties; it does not move the entity’s accountability to its customers or its supervisor.
If the only evidence the board has about a vendor’s system is the vendor’s own assurance, the board is supervising a promise rather than a system.
The question it should provoke
Who inside this organisation owns this outcome, and what do they see of how the system behaves?
Grounding is tested by questions, not attendance
How should a board build this grounding? My recommendation is to anchor it in the entity’s own uses rather than generic examples. Take two or three systems already in operation or under proposal and walk each of the four ideas through them. A general explanation of how AI works is easy to forget; seeing what these ideas reveal about a system the board has already approved is not.
The test of whether it worked is not attendance or a certificate. It is whether, at the next meeting, directors ask these questions without being prompted, and whether management starts arriving with the answers because it expects to be asked. That is also what makes the periodic report useful: the report shows what is happening, and the grounding is what tells a director which line deserves a follow-up.
A board does not need to understand how AI works. It needs to recognise what an AI answer cannot tell it on its own.
At Rhisco Group, our Risk, Models & AI Governance practice creates risk models, control artefacts and AI governance architecture to support positive outcomes when deploying AI and conducts independent reviews of risk models. If your board wants to oversee AI with the right questions rather than more vocabulary, let’s talk: www.rhisco.com/services
This article was co-created with the assistance of artificial intelligence under strict supervision, editing, and verification of our team.