The Governance Answers Come Before the Model
In most UK companies the blocker on an AI feature is not the engineering. It is the four questions somebody internal asks before it is allowed near a customer — and having no answer to them is what quietly stalls the project for a quarter.
Four Things Your Own Organisation Will Ask You
We have watched competent AI work sit unreleased for months because nobody could answer these in writing. So we answer them in week one, as deliverables, rather than as reassurance in a meeting.
Where the data physically goes
Which provider, which region, which endpoint. Whether a document leaves your tenancy at all and, if it does, which part of it leaves. We produce a written data flow rather than a verbal assurance, because that is the artefact an information governance lead can actually file.
Whether it trains somebody else’s model
On business and API tiers the answer is no, by contract. We set the account up that way, record the setting, keep the provider terms in your handover pack, and draw the distinction between that and a member of staff pasting the same document into a free consumer chatbot.
What UK GDPR permits here
Article 22 constrains decisions made solely by automation where the effect on a person is significant. That shapes the architecture: meaningful human authority over the outcome, a record of what the model was given, and an explanation a complaints handler can read aloud without flinching.
What you say to customers
The privacy notice wording, the on-screen label, and the visible route to a human being. Say it plainly and complaints fall. Leave it unsaid and you get to have the conversation later, at a worse moment, with somebody who feels misled.
None of this is legal advice. Your data protection officer or solicitor decides the position; our job is making sure the system you buy can support whatever they decide, which is considerably easier before it is built.
How an Answer Gets Assembled From Your Own Material
A model on its own answers from whatever it absorbed in training. A grounded system answers from your handbook, your policies and your case notes, and can show a reader exactly which paragraph it used.
Asked inside your product, carrying the signed-in account rather than an anonymous session.
The searchable set narrows to what this person could already open by hand. Filtering afterwards is not the same thing and does not survive an audit.
Short, relevant extracts. Sending entire documents is what makes a feature slow and expensive at the same time.
Instructed to work from the supplied extracts and to decline when they do not cover the question asked.
Every claim links to the paragraph behind it, so a colleague can check it in one click instead of arguing with a black box.
Held for an agreed retention period, scored against the evaluation set, and deletable when a subject access or erasure request lands.
Why the retrieval step is the part worth paying for
It keeps answers inside material you control, so when a policy changes in March you edit a document rather than commission a change request. It enforces permissions at the point of retrieval, which is the difference between a helpful assistant and an accidental disclosure of another client’s file. And it makes wrongness visible: a wrong citation is obvious in seconds.
It also sets a ceiling you should know about before you sign anything. If the answer is not written down anywhere in your organisation, no amount of prompt work will invent it. A fair proportion of grounded assistant projects turn out to be content projects in disguise, where somebody first has to write the policy nobody ever wrote. We would rather say that in week one than discover it in month three.
The index, the permission model and the logging all live in the backend, which is where most of the real engineering effort goes.
The backend work underneath thisWhere a Model Genuinely Earns Its Keep
Four shapes of work that come up repeatedly in UK businesses. They are not equally difficult, and the cheapest one is rarely the one people ask for first.
Assistants grounded in your own content
A letting agent whose staff answer the same forty tenancy questions every week. A professional services firm with a decade of precedent documents nobody can find. The value sits in the retrieval and the citations; the conversational part is the easy bit.
Getting structure out of documents
Supplier invoices, delivery notes, customs paperwork, timesheets photographed on site by a groundworker. The model reads awkward input and returns fields your system can store. Accuracy is measurable here, which makes it one of the easiest AI investments to justify to a finance director.
Classification and routing
Deciding which team an inbound enquiry belongs to, whether a complaint needs escalating, or which of eleven categories a case note falls into. Cheap per request, straightforward to evaluate, and often the quietest win on the list.
Drafting that a person finishes
A first draft of a survey report, a handover note, a reply to a routine enquiry. Framed as a starting point with a named author on the final version, this saves genuine hours. Framed as a finished product, it creates a different and more expensive problem.
What We Say Before You Commit a Budget
None of this is a reason to avoid AI. It is the set of things that decide whether what you build survives contact with real users and a real finance director.
Confident wrong answers are a property, not a defect
- A language model produces plausible text, and plausible is not the same thing as correct
- Grounding, citations and an instruction to decline reduce the rate substantially; nothing takes it to zero
- Anybody telling you hallucination is a solved problem is selling you something
- Design the screen so a wrong answer is cheap to spot and cheap to correct
Consequential output gets a person in front of it
- Anything touching money, tenancy, employment, care or a regulated outcome is reviewed before it lands
- Review has to be genuine: time to read it, authority to change it, and a record that they did
- A tick box everyone clicks through is worse than no review, because it manufactures false assurance
- We build the review queue, not just the model call
Cost per request becomes a line item at volume
- Billing is by the quantity of text processed, so a feature that reads a long document every time is expensive every time
- At a few hundred requests a month it is often under £50; at a few hundred thousand it warrants engineering attention
- We estimate the per-request cost during scoping and design retrieval to keep the input short
- Caching repeated questions is usually the largest single saving available
Latency decides what the interface can be
- A grounded answer takes seconds rather than milliseconds, and no amount of spinner design hides that
- Streaming the answer as it is written makes the wait tolerable; a blank screen does not
- Some jobs belong in a background queue with a notification rather than on a live screen
- We settle which of the two you are building before the design work starts
Evaluation matters more than the demonstration
- A demo proves the three questions the presenter chose in advance
- An evaluation set is real inputs paired with answers you have agreed are correct, run automatically on every change
- It is the only way to know whether last week’s improvement quietly broke something else
- We build it before the feature, and you keep it
Model behaviour moves underneath you
- Providers retire versions and change defaults on their timetable, not yours
- Pinning a version buys predictability until the day it is withdrawn, which is announced rather than negotiated
- The evaluation set is what turns a forced upgrade into an afternoon instead of a fortnight
- Put this in the maintenance budget on day one rather than discovering it in year two
Cases Where We Talk Clients Out of It
We would rather lose the AI line on a quote and keep the client. Every one of these is a conversation we have actually had.
Several of these point at a plain internal system or a reporting dashboard instead, and we will say so.
Nobody Has Been Granted an AI Exemption
We are software engineers rather than compliance advisers. What we can do is build so that your adviser has something they are able to approve.
Financial services
- FCA expectations around Consumer Duty apply to an automated answer exactly as they do to a member of staff
- Keep a reconstructable record of what the customer was told and on what basis
- A complaint reaching the Financial Ombudsman needs an explanation somebody can follow
Health and care
- NHS trusts and their suppliers work to the Data Security and Protection Toolkit, which shapes hosting, access and retention decisions
- CQC-registered providers need records that stand up during an inspection
- Clinical decision support is a regulated medical device conversation, and we will tell you when you have crossed that line
Public sector and education
- Councils, universities and colleges are bound by the public sector accessibility regulations, so an assistant meets WCAG 2.2 AA like everything else
- Procurement will want the data flow, the impact assessment and the supplier terms inside the tender response
- Framework agreements often fix retention and residency terms before design begins
One Job, Evidenced, Then the Next
Write the governance page first
Provider, region, retention, what is sent, who reviews the output. One side of A4 your information governance lead can sign off before engineering starts.
Agree what correct looks like
Thirty to a hundred real inputs with the answers your subject-matter expert agrees are right. That is the evaluation set, and it is the contract for whether the feature works.
Build the grounded pipeline
Ingestion, permission-aware retrieval, the model call, citations and logging. Wired into your existing sign-in so it only ever sees what that person could open themselves.
Release behind a flag and watch
A small internal group first, then a slice of users. We read the logs together every fortnight and the failures become the next sprint.
Asked Before Any of This Gets Signed Off
Where does our data actually go when the model processes it?
To a named provider, under a contract you can read, in a region you have agreed. We write that down before anything is built: which provider, which region, what is sent, how long it is retained and who at the provider can see it. Where UK or EU residency is a requirement we choose an endpoint that offers it, and where that is not available for the model you wanted, we tell you rather than hope nobody asks. That one document is what your information governance lead, your insurer and your largest customer all end up asking for, so it is cheaper to produce it properly at the start than to assemble it under pressure during a procurement round.
Will our documents be used to train a model we do not own?
On the business and API tiers of the major providers, no. Inputs and outputs are excluded from training by default, and that exclusion is a contractual term rather than a line on a marketing page. Consumer chat products are a different matter, which is the real reason a member of staff pasting a client file into a free tool is a governance incident whilst an API integration is not. We configure the account, record the setting, and put the provider terms into the handover pack so you can evidence the answer a year later without relying on anybody memory of a meeting.
Does UK GDPR stop us letting software decide something about a person?
Not outright, but Article 22 restricts decisions taken solely by automated means where they have a legal or similarly significant effect, which covers territory like credit, employment, tenancy and access to a service. The workable route is to keep a person in the decision with genuine authority to overturn it rather than a rubber stamp, and to be able to explain afterwards how the output was reached. We design for that in the first architecture session: the model proposes, the record shows what it was given, and a named person decides. Your data protection officer or solicitor settles the legal position; our job is making sure the software can support whatever they conclude.
Do we have to tell customers when an answer came from AI?
If the feature processes personal data then how it works belongs in your privacy notice, because transparency is one of the core UK GDPR principles and the ICO expects people to be able to understand what is being done with their information. Beyond the legal minimum, labelling is usually the commercially sensible choice anyway. An assistant that states plainly that it is automated and offers a visible route to a human generates far fewer complaints than one discovered to be automated after it gets something wrong. We build the label and the escalation path in from the beginning rather than retrofitting them after the first difficult email.
What goes into a data protection impact assessment for an AI feature?
A description of the processing and why you need it, the categories of personal data involved, your lawful basis, the flow of data to and from any provider, the risk to the people whose data it is, and the measures that reduce that risk. We supply the technical half: architecture and data flow diagrams, retention periods, access controls, what is logged, and where the human review step sits. You or your adviser own the assessment itself, because it is a judgement about your business and its appetite for risk rather than a document a supplier can sign on your behalf.
Describe the Job, Not the Technology
Tell us the task in plain words and who it affects. You get back whether a model is the right tool at all, what it costs to build in sterling excluding VAT, and what it will cost per request once it is live.