An AI purchase should solve a defined business problem with a manageable operating model. A polished demonstration, a recognised framework or a supplier's assurance statement cannot establish that the proposed workflow is suitable for your organisation.
This guide sets out a practical review of purpose, evidence, responsibilities and continuing cost. It supports a proportionate pilot and procurement decision, with specialist review where the circumstances require it.
Start with the task and affected people
Describe the problem, current process and proposed change. Identify who will use the system, whose information it will process and who could be affected by an incorrect result. Include the staff who must maintain sources, review outputs or resolve exceptions.
Define what success would mean against a measured starting point. Faster drafting is different from faster completion of an approved customer response. Consider ordinary software or process changes alongside AI, especially where the task has clear rules.
Assess readiness beyond the dataset
Review information quality, access, business ownership, operating capacity and relevant skills. A large document collection does not establish readiness if the team cannot identify which version is approved or who can resolve contradictions.
Check whether the data represents the intended use, including unusual formats and customer situations. Record collection gaps and restrictions. Representative examples and meaningful evaluation matter more than a general promise that more data will improve every outcome.
Map the complete workflow
Record the inputs, transformations, sources, tools, storage and outputs. Include connected services and the people who receive results. Follow the actual path through the application rather than assessing the model as an isolated component.
For personal information, identify the operations and parties involved, with responsibilities assessed for each relevant activity. Commercial labels do not determine every data protection role. Review the arrangement against current requirements and obtain qualified advice where needed.
Require evidence for the intended use
Ask which claims are supported by tests, what those tests measured and where their limits lie. An assurance report may cover one deployment, configuration or point in time. Check its relationship to your actual sources, users, permissions and actions.
Risk frameworks can organise questions about governance, context, measurement and management. They do not themselves certify a system or approve a purchase. Use the framework to expose decisions and evidence, with an owner for unresolved concerns.
Review the supplier and shared responsibilities
Identify who controls the platform, application, data preparation, permissions and day-to-day use. The split can change with the service and deployment. A general shared-responsibility diagram is not a substitute for the actual agreement and operating procedures.
Review service dependencies, security information, support arrangements, change notices and exit options. Ask how failures are investigated and which evidence the supplier can provide. Record contingency plans for an unavailable service or withdrawn capability.
Specify controls and decision ownership
Define supported tasks, prohibited actions, approval requirements and escalation routes. Give staff role-specific guidance rather than assuming that one introductory training session establishes competence for every activity. Keep operational permissions enforceable outside the model's instructions.
Agree who accepts the pilot, authorises wider use and can suspend it. Staff need a practical route for reporting unsupported claims, inappropriate access and unexpected actions. Include review capacity in the plan, not just the number of user licences.
Evaluate outcomes and continuing risk
Use reviewed cases covering expected tasks, missing evidence, unusual inputs and plausible failures. Compare against the current process, including staff checking time. Confidently incorrect answers and overreliance are reasons to test the whole workflow.
Continue monitoring after release. Inputs, sources, integrations and supplier services can change, so a successful pre-release test does not settle future performance. Record what triggers reassessment and who acts on the findings.
Budget for the full lifecycle
- Include preparation, integration, licences and usage charges.
- Allow for training, review, support and incident handling.
- Specify maintenance of approved sources and evaluations.
- Review contract changes and third-party dependencies.
- Plan data export, service transition and retirement.
- Record the decision, evidence, limitations and next review date.
Our AI evaluation guide helps define the evidence for a pilot, while the agent permissions guide covers enforceable action boundaries. Giraffe Digital's digital strategy service connects procurement with a practical delivery and operating plan.
