Signals

A signal is an automatic per-conversation detection: expressed user emotion, a failed tool call, an effort shortfall, or a claim contradicted by recorded execution state. The Signals page (sidebar: Observe → Signals) is where you browse them. Detections typically appear within minutes of ingest.

​

Page controls

• Date range — Last 24 hours, 3, 7, 14, 30, or 90 days, or a custom range.
• Refresh — reloads the active tab.
• Tabs — Overview, Browse, User Emotion, Tool Failures, Laziness, Hallucinations.

The active tab is addressable via the type URL parameter (overview, browse, emotion, tool-failures, laziness, hallucinations), and each detector tab supports its own deep-link parameter: tool (Tool Failures), family (User Emotion), pattern (Laziness), and rule (Hallucinations).

​

Overview

Cross-detector KPI cards, each with the counts behind the rate, the change against the previous window, and a link to the matching tab: Below the cards, What is changing lists detector patterns that are moving against the previous window, and Where conversations are concentrated shows the top use cases with a Browse use cases link to Use Cases.

​

Browse

Signal inventory — one row per detector pattern (an emotion signal, a failing tool, a hallucination rule, a laziness family) with columns: Rows can be searched, filtered by signal type, and opened into an evidence slideout.

​

User Emotion

Detects emotion expressed in the user’s own messages, classified into 6 families and 16 signals. Every analyzed conversation counts toward the KPIs; the table lists conversations with at least one detection or a negative-family risk score of 0.5 or higher. KPIs: Rows carry a status of Detected, At Risk, or Positive. Filter by status (All statuses / Detected only / At risk only), family, elicitor, and trajectory; sort by Highest score, Lowest score, Highest risk, Most recent, Oldest first, or Most messages; search by conversation ID or cause. Opening a row shows the cause, trajectory, elicitor, and verbatim user quotes — each quote links into the conversation’s Full Trace at the quoted turn.

​

Emotion taxonomy

Each detection also records:

• Elicitor — what elicited the emotion: bot, external, self, or ambiguous.
• Trajectory — how it evolved across the conversation: none, building, peak, resolved, or sustained.
• Score and risk score, a primary cause, and supporting quotes.

​

Tool Failures

“Tool invocations that returned errors or failed to execute.” Every tool call is recorded at ingest, so failures appear quickly; failed calls are then grouped into named failure categories per tool. KPIs: Failed Conversations, Total Failures, Failure Rate, Users Impacted, Tools Impacted. A time-series chart plots failure events (stacked or grouped), and a per-tool drill-down is deep-linkable with ?tool=. Each failure category carries: Individual failure events link into the conversation at the failed call. Tool-level success rates and usage live on the Tool Catalog page.

​

Laziness

“Agent effort shortfalls — skipped tools, stubs, and under-delivery.” Detections come from deterministic response patterns. Ingest-truncation artifacts are listed but excluded from headline counts. KPIs: Detections, Conversations Affected, High-Confidence Share, Patterns Firing. Detections with confidence 0.5 or higher count as laziness on their own (“high-confidence”). Weaker detections are corroborating only — they count when at least two pattern families co-occur in the same conversation. Filter by pattern family (?pattern=) and status (All conversations / High-confidence only).

​

Pattern families

When present, judge-verified detections appear as two additional families — Tool selection (judged), where the agent skipped or misused a tool the task required, and Under-delivery (judged), where the agent delivered less than the user asked for. Each carries a subtype: Missing tool call, Wrong tool, Missing required call, Excess tool call, or Under-delivery.

​

Hallucinations

“Agent claims checked against recorded execution state.” Claims in agent messages are checked against the tool receipts and task state recorded in the conversation’s world state, so checks run once the world state is built — minutes after ingest. KPIs: Contradicted, Verified Claims, Unverified Rate (unverified share of scored messages), Messages Analyzed.

​

Verdict semantics

Each scored agent message gets at most one verdict — the most severe claim in the message wins:

• Contradicted — a claim conflicts with recorded evidence. Listed per message with the claim quote, the rule that fired, and the evidence it violates.
• Verified — a claim is confirmed by a matching successful tool receipt. Listed per message.
• Unverified — a claim could not be checked either way. Counted only in aggregates (the Unverified Rate); never listed per message.

Per-message and per-conversation lists therefore show only contradictions and verified claims. The same policy applies to the Data API hallucinations endpoint.

​

Rules

Rule chips above the list filter by rule (?rule= deep link): Filter conversations by status (With contradictions / All scored conversations), sort by Most contradictions, Most verified, Most recent, or Oldest first, and search by conversation ID. Expanding a conversation lists each detection with its message number, offending quote, and the rule that fired; the conversation ID links into the conversation.

​

Next steps

• Problems — signals grouped by root cause and ranked by impact.
• Home — the same quality rates as at-a-glance KPIs.
• Conversations — where every signal’s evidence links land.
• Data API: signals — query emotions, tool failures, and hallucinations programmatically.