Web Analytics Made Easy - Statcounter
Skip to content

Alerts

Alerts turn telemetry, evals, cost, and policy signals into owner-routed action. Use them when teams need to know that a production AI workflow is drifting, failing, getting expensive, violating policy, or hurting a customer-facing outcome.

What Alerts Do

Anosys alerts can watch signals such as latency, token usage, cost, eval score, policy status, error rate, tool-call failures, workflow SLA, and custom business KPIs. Alerts can be threshold-based, anomaly-based, eval-based, policy-based, or derived from process units and pipelines.

When To Use Alerts

Use alerts for:

  • Silent quality regressions that do not show up as infrastructure errors.
  • Token or model-cost spikes by agent, workflow, user, team, or model.
  • Eval failures in CI or production.
  • Sensitive-data, policy, or governance violations.
  • Workflow bottlenecks, stuck process units, or SLA misses.
  • Ownership routing when incidents need the right engineering, product, support, or operations team.

Prerequisites

  1. Send the signal you want to alert on through OpenTelemetry, REST/HTTP events, JavaScript or pixel tracking, Claude Code, OpenAI Agents, or another ingestion method.
  2. Define the fields you want to alert on, such as workflow, team, model, user_id, cost_usd, eval_score, policy_status, or business_outcome.
  3. Decide where alerts should route: Slack, email, webhook, incident queue, or a downstream workflow.

Step-By-Step Setup

  1. Open Alerts in the Anosys console.
  2. Choose the signal type: anomaly, eval, cost, policy, process unit, or custom query.
  3. Select the data source and filter scope, such as checkout-agent, claims-review, or claude-code.
  4. Define the condition.
  5. Set severity and owner.
  6. Add routing destinations.
  7. Preview matching historical events.
  8. Save and run the alert in observe-only mode before enabling paging.

Example Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
name: checkout-agent-cost-spike
source: traces
scope:
  workflow: checkout-agent
condition:
  metric: cost_usd
  type: anomaly
  window: 30m
  sensitivity: high
route:
  owner: ai-platform
  channels:
    - slack:#ai-ops
    - email:ai-platform@example.com

What Appears In The Console

An alert includes the triggering signal, severity, owner, affected workflow, related traces, eval results, cost context, timeline, and recommended next steps from the AI Platform Assistant.

Common Mistakes

  • Alerting on raw thresholds without workflow, team, or customer context.
  • Paging on first-run alerts before testing noise levels.
  • Creating separate alerts for symptoms that should be grouped into one incident.
  • Missing ownership fields, which makes routing slow.