AI Agent Handoff: Don't Lose Context Between Agents

80% of bugs in multi-agent pipelines aren't in the agents. They're in the handoff.

When an orchestrator passes control to a specialized sub-agent, something gets lost. Not because of the model, but because of how we design the transfer: what information travels, in what format, and with what guarantees it arrives intact.

It's the telephone game with a 200-token budget.

Why Handoffs Break What's Working

The common pattern: the orchestrator receives the task, reasons through it, and calls the sub-agent passing the full conversation history — or worse, a summary it generates on the fly.

The problem is three-fold:

The full history won't fit. Three rounds of tool use plus the first 50 context messages is 8,000 tokens of history when the sub-agent only needs to know what to do right now.

The summary loses critical information. The orchestrator doesn't know which parts are critical to the sub-agent. It summarizes what it thinks matters. The sub-agent needs something else. The error happens silently, with no trace to explain it.

No schema versioning. You update the orchestrator; the sub-agent inherits a different format without knowing it. The system works 90% of the time. The remaining 10% fails in ways that don't reproduce.

The HandoffPayload Pattern in TypeScript

The fix is treating the handoff as a typed contract, not free text that "the model will figure out."

import { z } from 'zod'

const HandoffPayloadSchema = z.object({
  version: z.literal('1.0'),
  goal: z.string().describe('Original objective unchanged'),
  constraints: z.array(z.string()).optional(),
  completedSteps: z.array(z.object({
    tool: z.string(),
    result: z.string(),
    timestamp: z.string()
  })),
  activeContext: z.record(z.string(), z.unknown()),
  returnTo: z.string().optional()
})

type HandoffPayload = z.infer<typeof HandoffPayloadSchema>

This schema does three concrete things:

1. Separates the goal from history. The sub-agent receives the original objective uncontaminated by the orchestrator's intermediate reasoning noise.
2. Serializes results, not reasoning. completedSteps carries each tool call output — not the chain of thought.
3. Versions the contract. When the schema changes, the sub-agent detects the incompatibility before executing.

Building the Handoff in the Orchestrator

async function buildHandoff(
  originalGoal: string,
  toolResults: ToolResult[],
  context: Record<string, unknown>
): Promise<HandoffPayload> {
  const payload = {
    version: '1.0' as const,
    goal: originalGoal,
    completedSteps: toolResults.map(r => ({
      tool: r.toolName,
      result: r.output.slice(0, 500), // cap size per step
      timestamp: new Date().toISOString()
    })),
    activeContext: context,
    returnTo: process.env.ORCHESTRATOR_ID
  }

  return HandoffPayloadSchema.parse(payload) // validate before sending
}

The 500-character cap per step result is intentional. If a result is longer, the sub-agent fetches it with its own tool call to the source system. The handoff is not the data repository — it's the map.

Three Mistakes We See in Every Production Pipeline

Mistake 1: Passing the full messages array to the sub-agent

The orchestrator's chat history has {role, content}[] format. The sub-agent appends it to its own history. Context grows exponentially at each pipeline step. By the fourth agent in the chain, you're already in overflow.

Mistake 2: Missing returnTo

The sub-agent completes its work but has no destination to report to. A response is generated that no one collects. This shows up in logs as "completed successfully" when nothing actually arrived at its destination.

Mistake 3: Using activeContext as a junk drawer

Stuffing everything that "might be relevant" into activeContext creates a 30-field object where the sub-agent can't tell which fields are critical. The model weights them all equally. Define the minimum necessary context: not what could matter, but what must matter.

Handoff with Error Recovery

When a sub-agent fails, the orchestrator needs to know the handoff's final state to decide between retrying or escalating.

interface HandoffResult {
  status: 'completed' | 'failed' | 'partial'
  output?: string
  error?: {
    code: string
    step: string
    recoverable: boolean
  }
  finalContext?: Record<string, unknown>
}

async function executeWithHandoff(
  subAgent: SubAgent,
  payload: HandoffPayload
): Promise<HandoffResult> {
  try {
    const result = await subAgent.run(payload)
    return {
      status: 'completed',
      output: result.output,
      finalContext: result.context
    }
  } catch (error) {
    const isRecoverable =
      error instanceof ContextLossError ||
      error instanceof TimeoutError

    return {
      status: 'failed',
      error: {
        code: error.code,
        step: error.lastStep,
        recoverable: isRecoverable
      }
    }
  }
}

The recoverable flag lets the orchestrator distinguish between "retry with the same payload" and "escalate to a human because context is irrecoverably lost." Without it, all errors look the same and the system retries when it shouldn't.

What This Changes in Production

With this pattern in three-or-more agent pipelines:

• Context errors drop 60-70%. The sub-agent always has the original goal, never a degraded version of it.
• Timeouts are detected earlier. A version mismatch aborts the handoff at validation, before spending tokens.
• Retries are safe. HandoffPayload is stateless and can be resent without duplication risk.

The pattern adds no measurable latency. Payload serialization is microseconds; Zod validation is milliseconds. What it does eliminate is the hours of debugging when a handoff fails silently in production with no trace.

If your orchestrator passes the full messages array to sub-agents, or you have no typed handoff schema, or a sub-agent failure leaves the orchestrator not knowing what state the system is in — you have this problem.

Designing this correctly from the start saves weeks. Our AI integration service builds multi-agent pipelines with typed handoffs, observability from day one, and versioned context contracts between agents. This is the kind of architecture you design before the problem costs you money — not after.

If you already have agents in production and suspect the issues are at the handoff points, let's look at it together:

Let's review your pipeline →