Business Process Monitoring for AI-Driven Operations
Much of the AI that matters in a company isn't a chat window. It's the review step in claims processing, the classifier that routes support tickets, the extraction model in customer onboarding, the fraud check between order and fulfillment. When AI sits in the middle of a business process and misbehaves, the symptom is a stuck queue, a missed SLA, or an approval rate that drifts for a month before anyone links it to a prompt change.
Neither existing discipline covers that well. Process monitoring sees stages, queues and cycle times, and treats the AI step as a box that either finished or didn't. AI observability sees the model's runs in detail, with no idea they sit inside a claims workflow with a four-hour SLA. The failures that matter happen where the two meet.
The AI step can succeed and still hurt the process
- After a policy prompt change, a document extractor escalates more cases to human review. Every run succeeds; cycle time doubles.
- A ticket router keeps routing, a little worse than before, and other teams inherit a growing pile of misrouted tickets.
- Retries in a review agent make one stage four times more expensive per completed item, with no errors.
Nothing pages anyone. The process gets slower and costlier gradually, and gradual changes are the ones nobody notices until a quarterly review.
Catching them means watching process signals (stages, cycle times, completion and escalation rates, SLA clocks, outcomes such as approval rate) and AI signals (the traces, tool calls, eval scores and token spend of the runs inside each stage) together. That only works if they share keys.
flowchart LR
S1["claim<br/>submitted"] --> S2["<b>AI review step</b>"] --> S3["approved /<br/>escalated"]
S3 --> KPI["completion rate<br/>cycle time · SLA"]
S2 -. "same process ID" .- TRACE["trace · evals · cost<br/>of the run inside"]
KPI --> RCA["<b>cause</b><br/><i>the change behind<br/>the KPI drop</i>"]
TRACE --> RCA
style S2 stroke:#f4ae30,stroke-width:2px
style TRACE stroke:#4ea3ff,stroke-width:2px
style RCA stroke:#59ca6f,stroke-width:2pxInstrument the process in four steps
1. Name the process and define done
Use names the business already uses: claim review, refund approval, ticket triage, onboarding document check. Avoid implementation labels such as "AI analysis." Then define what a good outcome is: approved, resolved without escalation, completed within SLA.
2. Send one event per step
Each event carries the same process ID from start to finish:
Map the fields once in Field Mapping, for example process_id, step, status, account_tier, owner and sla_hours. Send a finished (or escalated, failed) event when the step ends.
3. Put the same ID on the AI run
Pass the process ID to the agent SDK as the session or user context, so the trace, eval scores and token cost of the run inside the step carry it too. This is the step that makes everything else work: without it you have two good datasets that can't be joined.
4. Measure the process the way you'd measure a service
| Process metric | AI signal to join | What it tells you |
|---|---|---|
| Stage cycle time | Model and tool latency, retries | Whether the AI step is slow or waiting on a dependency |
| Completion rate | Task-success eval, escalation | Whether the AI step produces more finished work |
| Escalation rate | Refusals, low-confidence output, policy checks | Whether people are absorbing AI failures |
| Cost per completion | Tokens, model route, retries | The workflow's unit economics |
| SLA risk | Queue age, stage duration, owner | Whether to act before a commitment is missed |
| Rework rate | Corrections, downstream rejections | Whether the next step accepts the AI's output |
Rework rate is the easiest to miss and often the most telling. An extractor that speeds up intake but adds manual corrections in review didn't improve the process; it moved the cost to another team.
What a connected finding looks like
With the layers joined, an investigation ends in one finding instead of four partial tickets:
Onboarding completion fell for enterprise accounts. The AI document check started escalating more often after Tuesday's policy prompt change. Cost per completed onboarding rose and two accounts are close to their SLA. The onboarding AI team owns the prompt.
The process owner knows what happened, the AI team knows what to revert, and everyone knows which accounts were affected.
Doing this with Anosys
Send step events over the REST API or OpenTelemetry, and pass the same process ID through get_user_context in the agent SDK. Chart the flow with a Funnel Plot or Sankey diagram to see where items drop out. A process unit (a saved query and transformation) can compute stage durations and backlogs, and a pipeline can run it on a schedule, score a sample of the AI step with an evaluation, and trigger an alert when items get stuck or escalations rise. The Business process monitoring guide has the setup steps.
Explore the Business Process Monitoring solution.
Free plan, no credit card
Monitor your AI-driven processes for free
Send the events your process already produces and see each case end to end: where it stalls, where the AI step got it wrong and what it cost.
