When an AI tool gets something wrong, the tool is not accountable. A person is. Before using AI in any process that matters, decide who reviews its output, who signs it off and who answers for it when it is wrong. If nobody can name that person, the process is not ready.
Most conversations about AI at work start with what it can do. Summarise a contract, draft a proposal, triage tickets, answer customer questions, write code. It can do a lot of that well. The conversation that usually comes later, often after an incident, is about who was responsible for checking it.
Why accountability does not move to the tool
A customer who receives a wrong answer does not care which tool produced it. A regulator does not accept “the model said so”. A contract signed on the basis of a bad summary is still signed. In every case, the organisation owns the result, and inside the organisation a specific person does.
AI changes how work is produced. It does not change who carries the risk.
What should you decide before using AI in a process?
I ask the same short set of questions whenever a team wants to put AI into real work:
- What does the model see, and where does that data go? Customer data, contracts and internal financials need clear rules.
- Who reviews the output, and do they have time? A review step that nobody has time for is not a control.
- What happens when it is wrong? How would you notice, and what is the cost of a mistake that nobody noticed?
- Who signs? One named role that owns the outcome, not “the team”.
- What does it cost to run? Licences, usage, integration and the time spent checking.
Where AI should not decide alone
Some decisions should always have a person in the loop:
- anything that commits the organisation commercially or contractually
- anything a customer will rely on without checking
- anything that changes access, money or personal data
- anything where a confident wrong answer is worse than no answer
In those areas AI can prepare, draft and suggest. A person decides.
Make the review real
A review step only works if it is designed as part of the process. That means the reviewer knows what to check, has the source material next to the output, and has the authority to reject it. It also means measuring how often the output is corrected. If nobody ever corrects anything, either the tool is perfect or nobody is looking.
The practical version
Use AI where it saves time and the cost of a mistake is low. Add a named reviewer where the cost is higher. Keep a person in charge where the decision is commercial, contractual or affects customers directly.
The question is never only “can AI do this?”. It is “who signs for it when it is wrong?”. Answer that first, and the rest of the rollout gets much simpler.