Evaluations
Standard regression suites simulated against Bookly CX v2.4 — quality trends and root causes.
Composite score
93.9%
Weighted across 10 suites
Test cases
1,675
Simulated conversations per cycle
Suites regressing
3
Score below previous run
Auto-graded
98%
2% sampled for human review
Composite quality trend · last 12 runs
12 runs ago · 91.2%Hover to scrubLatest · 95.9%
Quality dimensions
Task completion95.9%
Policy adherence98.2%
Factual grounding93.4%
Tool selection95.4%
Tone & empathy94.7%
Containment87.3%
Test suites
| Suite | Scope | Cases | Score | Δ vs prev | Cadence | Last run | Status |
|---|---|---|---|---|---|---|---|
| Order status accuracy | Core | 240 | 97.1% | +0.7 | Every deploy | 2h ago | passing |
| Return eligibility reasoning | Core | 180 | 92.8% | -1.8 | Every deploy | 2h ago | watch |
| Refund policy compliance | Compliance | 150 | 99.3% | +0.2 | Nightly | 9h ago | passing |
| Tool-call correctness | Core | 320 | 95.4% | -0.3 | Every deploy | 2h ago | passing |
| Genre / recommendation guidance | Experimental | 90 | 84.2% | +4.7 | Weekly | 3d ago | watch |
| Escalation & handoff | Core | 110 | 96.0% | 0.0 | Nightly | 9h ago | passing |
| PII redaction & safety | Compliance | 200 | 99.8% | 0.0 | Nightly | 9h ago | passing |
| Voice transcription robustness | Channel | 130 | 88.6% | -2.3 | Weekly | 3d ago | failing |
| Multi-order disambiguation | Core | 95 | 91.5% | +1.3 | Weekly | 3d ago | watch |
| Tone & brand adherence | Quality | 160 | 94.7% | +0.6 | Nightly | 9h ago | passing |
Root-cause clusters across failing cases
Ambiguous RMA window on gifted orders
Return eligibility
Voice ASR mis-hears ISBN digit strings
Voice transcription
Stale inventory read before restock sync
Tool-call correctness
Over-apology loops on refund delays
Tone & brand
Missing genre profile for new customers
Recommendation guidance