Skip to content

Observability for OpenAI ChatKit Apps

OpenAI's ChatKit SDK (@openai/chatkit-react) lets you embed a fully managed AI chat widget in any React application. The Anosys integration hooks into ChatKit's lifecycle callbacks to automatically capture every conversation turn, measure performance metrics, and correlate sessions — so you can understand how users interact with your AI assistant in production.

Why Monitor Your ChatKit App

A ChatKit-powered assistant is a production AI interface. Every conversation is a user experience that generates cost, latency, and potential failure points:

What Anosys Captures Why It Matters
Prompts & responses See exactly what users ask and what the model returns
Time-to-first-token Measure perceived responsiveness of the chat widget
Response duration End-to-end latency from user submit to full response
Thread & session IDs Correlate multi-turn conversations across page views

Architecture Overview

The integration uses a custom React hook (useChatObservability) that plugs into ChatKit's built-in callbacks:

%%{init: {"flowchart": {"curve": "linear"}}}%%
graph LR
    A[User Message] -->B["ChatKit Widget (@openai/chatkit-react)"]
    B -->C[OpenAI API]
    C -->B
    B -->D["useChatObservability Hook"]
    D -->E["Anosys Platform (POST)"]

    style B stroke:#1e88e5,color:#fff
    style D stroke:#c62828,color:#fff
    style E stroke:#388e3c,color:#fff

How it works:

  1. ChatKit fires onLog, onResponseStart, and onResponseEnd callbacks during each conversation turn.
  2. The observability hook timestamps each phase to compute time-to-first-token and response duration.
  3. On response end, the hook collects the prompt, response and correlation IDs for the turn.
  4. The hook sends one event per turn to your Anosys endpoint.

Getting Started

Prerequisites

  • A React application using @openai/chatkit-react (v1.4+)
  • An Anosys endpoint of type OpenTelemetry / AI Agents (create one under Collecting → Endpoints)
  • A small backend route that creates ChatKit sessions (your OpenAI key must stay on the server)

Keep secrets out of the browser

ChatKit runs in the browser, so anything in your frontend bundle is public. Never put your OpenAI key or an Anosys API key in frontend code or VITE_ variables. The setup below uses your endpoint's unauthorized POST URL, which identifies the endpoint but can't read data, and creates ChatKit sessions on your server.

Step 1 — Install ChatKit

1
npm install @openai/chatkit-react

Step 2 — Add the Observability Hook

Copy your endpoint's POST URL from Collecting → Integration Options → POST with Unauthorized selected. Then create a hook that provides the lifecycle callbacks:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
// src/hooks/useChatObservability.js

import { useState, useRef, useEffect } from 'react';

export function useChatObservability() {
    const [correlationIds, setCorrelationIds] = useState({
        appSessionId: crypto.randomUUID(),
        threadId: null,
        afterItemId: null,
    });

    const metricsRef = useRef({
        composerSubmitTimestamp: null,
        responseStartTimestamp: null,
    });

    // Unauthorized POST URL of your Anosys endpoint (not a secret)
    const ANOSYS_INGESTION_URL = import.meta.env.VITE_ANOSYS_INGESTION_URL;

    const sendToAnosys = async (event) => {
        await fetch(ANOSYS_INGESTION_URL, {
            method: 'POST',
            headers: { 'Content-Type': 'application/json' },
            body: JSON.stringify([event]),
        });
    };

    // --- ChatKit Callbacks ---

    const onLog = (event) => {
        if (event.name === 'composer.submit') {
            metricsRef.current.composerSubmitTimestamp = Date.now();
            metricsRef.current.responseStartTimestamp = null;
        }
    };

    const onResponseStart = () => {
        if (!metricsRef.current.responseStartTimestamp) {
            metricsRef.current.responseStartTimestamp = Date.now();
        }
    };

    const onResponseEnd = async () => {
        const endTs = Date.now();
        const { composerSubmitTimestamp, responseStartTimestamp } = metricsRef.current;

        const metrics = (composerSubmitTimestamp && responseStartTimestamp) ? {
            timeToFirstToken: responseStartTimestamp - composerSubmitTimestamp,
            duration: endTs - responseStartTimestamp,
        } : {};

        // Fetch thread messages and ship to Anosys
        // (see full implementation in the demo repo)
        sendToAnosys({
            cvs3: correlationIds.appSessionId,   // app session
            cvs4: correlationIds.threadId,       // ChatKit thread
            cvn1: metrics.duration,              // response duration (ms)
            cvn2: metrics.timeToFirstToken,      // time to first token (ms)
            // cvs1 / cvs2: prompt and response text, see the demo repo
        });
    };

    return {
        correlationIds,
        setCorrelationIds,
        onLog,
        onResponseStart,
        onResponseEnd,
    };
}

Step 3 — Wire It Into Your ChatKit Widget

Pass the callbacks from the hook into useChatKit:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// src/components/ChatWidget.jsx

import { ChatKit, useChatKit } from '@openai/chatkit-react';
import { useChatObservability } from '../hooks/useChatObservability';

const ChatWidget = () => {
    const {
        setCorrelationIds,
        onLog,
        onResponseStart,
        onResponseEnd,
    } = useChatObservability();

    const { control, ref } = useChatKit({
        api: {
            async getClientSecret() {
                // Your own backend creates the ChatKit session with your OpenAI key
                const res = await fetch('/api/chatkit/session', { method: 'POST' });
                const { client_secret } = await res.json();
                return client_secret;
            },
        },
        theme: { colorScheme: 'dark' },

        // Anosys observability callbacks
        onLog,
        onResponseStart,
        onResponseEnd,
    });

    return (
        <ChatKit ref={ref} control={control}
            style={{ width: '100%', height: '100%' }} />
    );
};

Step 4 — Set Environment Variables

Add the following to your .env file:

Frontend (.env, public):

1
VITE_ANOSYS_INGESTION_URL=https://api.anosys.ai/ingestion/YOUR_WORKSPACE/YOUR_TYPE/YOUR_ENDPOINT

Backend (server only, never in the frontend):

1
2
OPENAI_API_KEY=sk-your-openai-key
OPENAI_WORKFLOW_ID=your-chatkit-workflow-id

Your /api/chatkit/session route calls POST https://api.openai.com/v1/chatkit/sessions with OPENAI_API_KEY and the workflow ID, and returns the client_secret.

What Gets Sent to Anosys

Each conversation turn produces a structured payload with the following fields:

Field Type Description
cvs1 String User's prompt text
cvs2 String Assistant's response text
cvs3 String App session ID (unique per page load)
cvs4 String OpenAI thread ID
cvn1 Number Response duration (ms)
cvn2 Number Time-to-first-token (ms)

Name these fields in Collecting → Field Mapping (for example cvn2 → time_to_first_token_ms).

Full Reference Implementation

The complete working implementation is available as an open-source demo:

Anosys-AI/AnoSys-Demo-Customer — a React + Vite application built with @openai/chatkit-react and full Anosys observability.

Key files:

  • src/components/ChatWidget.jsx — ChatKit widget with observability callbacks
  • src/hooks/useChatObservability.js — the observability hook (metrics and thread extraction)
  • src/utils/chatHelpers.js — helper functions for message extraction and Anosys payload mapping

Next Steps