AI services can have separate settings for training, storage, logging and feature-specific retention. Turning off one use of data does not establish that the provider stores nothing, and an organisation's agreement may differ from a public default.
Before a pilot, map the actual workflow and verify its supported settings. Keep a dated record of the service, features and contractual arrangements you checked, then test the configuration you intend to use.
Identify the information the task needs
Describe the purpose and required inputs before connecting a whole mailbox or document collection. Select information relevant to the task and avoid unnecessary personal or confidential material. A short prompt can still contain a sensitive identifier or a complete customer record.
Review the complete path: uploads, retrieved passages, model requests, tool parameters, outputs, application logs and support records. Limiting the initial prompt does not remove information later supplied by an integration.
Separate training from retention
Whether data is used to train a model and whether it is retained are distinct questions. Review both for the actual account and product. Also establish what happens to files, stored conversations, batch jobs and other features that may keep state.
Some contractual arrangements or retention controls cover only specified operations. Do not assume that a zero-retention claim applies to every endpoint or automatically blocks unsupported features. Verify the scope, exclusions and enforcement with the applicable documentation and agreement.
Review logging and deletion deliberately
Define why each log is needed, who can access it and how long it should remain. A useful error reference may avoid recording the full customer message. The appropriate period depends on the purpose and obligations; there is no universal duration for every AI project.
Test the supported deletion process, including derived copies and connected systems. Record what can be deleted directly and what follows a separate retention schedule. Do not promise instant removal everywhere merely because a user-facing conversation disappears.
Distinguish pseudonymisation from anonymity
Replacing a name with a reference can reduce unnecessary exposure, but the reference may still identify someone when combined with another dataset. Consider who can access the lookup and whether the remaining details make the person recognisable.
Review samples before calling them anonymous. A job title, location and unusual event can reveal more than expected. Keep reference mappings protected and assess the real data flow rather than relying on a field label.
Check protection controls in their actual scope
Data loss prevention controls may depend on recognised information types, labels, licences and configuration. Test the channels and document types your staff use. A control that detects a familiar identifier may miss confidential pricing or an unlabelled strategy document.
Combine suitable controls with access boundaries, staff guidance and incident procedures. Record known gaps, so the team understands when review is still required. Technical detection is useful evidence, not proof that every confidential item is protected.
Explain the process and outcome appropriately
Decide what users need to understand about the system's purpose, information use and role in an outcome. Explanations should fit the audience and the actual workflow. A technical description of the model alone may not explain what happened to a customer's request.
Keep explanations accurate when the process changes. Review relevant privacy and other requirements with appropriate expertise. Avoid presenting a generic transparency statement as a complete assessment of every deployment.
Plan for platform and model changes
Record the supported model, interface and features. A compatible client library can hide differences in message formats, tool calls or unsupported fields. Test the behaviours the workflow relies on rather than assuming that a successful connection proves complete compatibility.
Monitor change and retirement notices. A platform's schedule may differ from that of a model provider or partner service. Allow time to evaluate a replacement against the same reviewed cases, check settings and update support procedures.
Measure adoption separately from value
- Check the reporting period, active-user definition and reporting delay.
- Separate assigned licences from actual activity.
- Verify which products or features a usage report includes.
- Measure completed business tasks and checking time.
- Review errors, exceptions and customer outcomes.
- Keep configuration and review records for each material change.
Our AI procurement review connects these settings with ownership and contracts. The evaluation guide provides a practical basis for comparing provider or model changes.
