AI evaluation should establish whether a system performs the intended task under the conditions the business expects. A fluent answer, a positive user rating and a completed transaction are different forms of evidence. Choose measures that match the decision you need to make.
This guide explains repeatable tests for answers, retrieval and actions. It helps a business compare changes without accepting a polished demonstration as proof of reliable operation.
Define success independently of the first output
Describe the required facts, boundaries and outcomes before scoring a response. Include what the system should do when information is unavailable or contradictory. A test built by copying the assistant's first answer can reward the repetition of an earlier mistake.
Use representative business tasks, including difficult and ambiguous cases. Record the expected result and why it is appropriate. Have a knowledgeable reviewer establish the reference rather than assuming that an AI-generated citation or explanation is authoritative.
Keep a repeatable case set
Preserve questions or conversations, approved sources, expected outcomes and scoring rules. Record the model, prompt, tools and relevant configuration for each run. Compare changes against the same reviewed cases, so the result does not depend on selecting a new set of favourable examples.
Maintain separate cases for independent assessment where appropriate. If examples repeatedly shape development decisions, explain that limitation. Do not describe an evaluation as independent if the same cases have already been used to tune or train the system without accounting for that overlap.
Separate quality dimensions
Fluency concerns expression, relevance concerns the question, completeness concerns required information and source support concerns the relationship to evidence. A well-written answer can omit an important condition. An answer supported by retrieved text can still rely on an obsolete source.
Define a practical rubric for each property that matters. Use concrete pass conditions for critical details and inspect failures individually. A generic average quality score can conceal an error in the one field needed to approve a booking.
Check factual claims outside the generated answer
AI outputs can contain incorrect claims, invented quotations or fabricated references. Inspect the actual source and verify important facts before using the answer as evidence. Asking the same system to confirm itself is not independent corroboration.
Permit uncertainty and define an appropriate no-answer route. These design choices can reduce unsupported assertions, but they do not eliminate errors. Test whether the system actually uses the route when evidence is missing.
Evaluate retrieval before judging generation
Check whether the relevant, current and authorised material was selected. Record the passage and its scope. A generated answer cannot reliably compensate for a collection that omits the required information or exposes the wrong customer's records.
Compare retrieval strategies on the same reviewed questions. If a model judges relevance, treat that score as additional evidence with its own limits. Inspect whether the selected material satisfies the business's approval and access requirements.
Verify actions through environment state
An agent saying that a reservation was saved is different from a record showing the reservation. Check the authoritative system, identifiers and final state. Include rejected actions, duplicate requests and operations that are still awaiting review.
Separate conversation success from task completion. A session classified as resolved by a platform may use a definition that does not establish the customer's actual outcome. Document the metric's triggering rules before using it as a business success measure.
Label classification errors clearly
For tasks such as enquiry routing, compare the known class with the predicted class. A confusion matrix can show correct classifications and the different error types, but both axes and the class order must be labelled. Unlabelled numbers can reverse the interpretation of false positives and false negatives.
Assess the consequence of each error type. Misrouting an urgent request may matter more than unnecessary staff review of a routine one. Set priorities around the workflow, with appropriate attention to coverage and the number of examples available.
Use feedback to improve the test set
- Connect feedback to the individual output or action.
- Separate satisfaction from independently checked accuracy.
- Preserve the relevant source and configuration.
- Review the expected result before adding a regression case.
- Retest meaningful failures after a change.
- Record remaining limitations and the release decision.
Our software release checklist connects evaluation with acceptance and operational readiness. Giraffe Digital's digital strategy service can help define the business outcomes, review process and evidence needed for a proportionate pilot.
