Insurance example
Extract agreed fields from policy documents, retain source references and flag ambiguous or missing information for specialist review.
Model & Data
Readiness Review
A sample of the evidence to agree for a focused pilot. This example uses insurance document extraction and retail product classification. No model has been trained or benchmarked for this sample.
01 / The decision to make
Does an adapted model improve a defined business task enough to justify its total cost and operating requirements?
Retail example
Classify products and extract agreed attributes from supplier descriptions, with exceptions routed to catalogue reviewers.
02 / Data readiness findings
- Sources & permissions
- Record origins, ownership and permitted training and evaluation uses.
- Quality & coverage
- Review labels, ambiguity, document or product families, language coverage and difficult cases.
- Sensitive information
- Identify data requiring minimisation and agree processing, access and retention boundaries.
- Readiness decision
- Record blockers, remediation actions, accountable owners and the conditions for proceeding.
03 / Evaluation design
Agree the baseline, acceptance criteria and a versioned dataset split before the experiment. Use training examples for adaptation, validation examples for model selection and held-out test examples for final evaluation.
- Check near-duplicates and related documents or products across splits.
- Separate by customer, document family, product family or time when appropriate.
- Document model versions, settings, test conditions and the limits of the comparison.
- Record uncertainty and performance by relevant segment, not only an overall average.
04 / Results to collect
| Measure | Baseline | Adapted model | Decision criterion |
|---|---|---|---|
| Task accuracy | Not measured | Not measured | Agreed field / label metric |
| Exception handling | Not measured | Not measured | Agreed escalation threshold |
| Response latency | Not measured | Not measured | Agreed target under load |
| Total operating cost | Not measured | Not measured | Agreed workload and cost assumptions |
All result cells are intentionally unmeasured. This is a sample report format, not evidence of performance or savings.
05 / Failure cases & limitations
Include representative errors, missed exceptions, unsupported inputs and the limits of the dataset. State where human review remains necessary and where evidence is insufficient.
06 / Recommendation & handover
Recommend proceed, revise or stop against the agreed criteria. Record unresolved risks, data preparation needs, deployment dependencies, licensing and ownership terms, and the proposed next scope.
Start with one task and a written scope.
Review depth, deliverables, fees and timing are agreed before work begins. Performance comparisons require a separately scoped experiment.
Scope a readiness review