1.6 Multi-Agent Swarm Observability & OpenTelemetry GenAI DAG Tracing

How to visualize multi-agent handoffs, sub-agent delegations, and token trees using OpenTelemetry GenAI spans.

Why a Transcript Alone Doesn’t Explain a Swarm Failure

A single-agent conversation is easy to debug – you read the transcript top to bottom. Multi-agent frameworks (LangGraph, CrewAI, AutoGen) break that model: an orchestrator delegates to specialist sub-agents, those sub-agents call tools and sometimes delegate further, and the thing that actually failed – a sub-agent that looped back into its own parent, or one that quietly ran three times longer than the rest – is invisible in a flat message log. You would have to reconstruct the call graph by hand from timestamps and agent names, which does not scale past a handful of hops.

CertiQA avoids that reconstruction by not needing you to do it: every framework in this category already emits OpenTelemetry GenAI spans (the emerging standard for agent instrumentation), and CertiQA’s adapter parses those spans directly into a Directed Acyclic Graph – one node per agent invocation, one edge per delegation or tool call, with token counts and latency attached to each node. Whatever failed shows up as a specific node or edge in that graph, not as a paragraph you have to re-read.

Two Ways to Get Spans In: A File Export, or Live During the Run

ASSERT_AGENT_TOPOLOGY, the step action that builds and gates on the DAG, reads spans from one of two places, controlled by its source field. source: FILE reads a pre-exported OTel GenAI spans JSON file – the original path, and still the only option if your agent framework can’t be reconfigured to export anywhere else at test time. source: LIVE instead captures spans as the run happens, through an in-process OTLP/HTTP receiver the runner starts on demand (default port 4791, overridable with –otel-receiver-port); point your framework’s OTel exporter at it for the duration of the run and the assertion sees the real trace with no separate export step. Leave source unset and CertiQA picks for you: LIVE when the receiver is running and no otelSpansFile was given, FILE otherwise.

- action: ASSERT_AGENT_TOPOLOGY
  source: LIVE
  maxHops: 6
  failOnDeadlock: true

# or, against a spans file you already exported elsewhere:
- action: ASSERT_AGENT_TOPOLOGY
  source: FILE
  otelSpansFile: ./traces/refund-swarm.json
  maxHops: 6

Start the receiver by passing –otel-live to run (add –otel-receiver-port if 4791 is already taken on your machine), then point your framework at it: set OTEL_EXPORTER_OTLP_ENDPOINT to the receiver’s URL and OTEL_EXPORTER_OTLP_PROTOCOL to http/json before the agent process starts. Internally the receiver waits for the trace to go quiet – no new span for 500ms by default – before treating it as complete, and gives up after a 5 second hard timeout either way, so a genuinely still-streaming multi-agent conversation isn’t cut off mid-delegation the moment the first span arrives.

What the Graph Actually Checks

Two structural checks come from the graph itself, independent of any single agent’s behavior: maxHops caps how deep a delegation chain is allowed to get before it counts as a failure (a runaway orchestrator that keeps re-delegating instead of finishing is a real failure mode, not a hypothetical one), and failOnDeadlock catches circular delegation – agent A hands off to B, B hands back to A, forever, which looks like healthy activity in a raw log right up until the run times out. Beyond those two, the same DAG is what the per-agent delegation, handoff-content, and budget assertions in 1.10 Multi-Agent Delegation, Handoff & Budget Assertions read from – they don’t rebuild it themselves, so any case using EXPECT_DELEGATION_TO, EXPECT_HANDOFF_CONTEXT, or EXPECT_AGENT_BUDGET needs an ASSERT_AGENT_TOPOLOGY step earlier in the same case.

Reading a Captured Trace in Run Reports

• 1. Open any executed Agent run in Run Reports.
• 2. In the Transcript Drawer, switch to the “🕸️ Multi-Agent Trace DAG” tab.
• 3. Inspect node hops, per-agent token counts, and delegation latency – click a node to see the span attributes CertiQA parsed it from, which is the fastest way to confirm a suspicious edge is real and not an artifact of an odd exporter convention.

Before You Trust a Passing (or Failing) Run

• If the receiver never sees a single span under source: LIVE, the assertion fails by naming the two environment variables the agent process needs – OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_PROTOCOL=http/json – rather than a generic timeout, since a misconfigured exporter is the far more common cause than an actually-broken agent.
• A body sent as protobuf (Content-Type: application/x-protobuf) is rejected outright with a 415, not silently parsed into an empty span list – make sure your framework’s exporter is set to the JSON OTLP protocol, not the binary default some SDKs ship with.
• The Trace DAG tab only renders a real captured graph once one exists for that run; until then it shows an honest empty state rather than sample data, so an empty panel on a fresh Project means no trace has been captured yet, not that tracing is broken.