AI Agent Model Routing: Pick the Right LLM per Task

AI Integration

•August 30, 2026•6 min read•By Daily Miranda Pardo

Last week I reviewed the logs of a team that had been running their agent in production for four months. Every single call went through Claude Opus. Intent classifications, response formatting, simple validations — all routed to the most expensive model available.

The result: €2,300 per month in API costs. And over 70% of those calls were tasks that Haiku would have handled just as well in a fraction of the time at 80% lower cost. The problem wasn't the AI. It was the architecture.

The Most Expensive Mistake in Production: One Model for Everything

When a team starts building agents, the logical choice is to use the most powerful model available. Quality is high, results are good, the client is happy. But nobody asks what percentage of calls actually need that level of capability.

A typical customer service agent, for example, does:

Classify incoming message intent → trivial
Extract an order number from text → trivial
Validate that a response matches the expected format → trivial
Summarize a long email thread → medium
Draft a personalized resolution proposal → complex

The first three represent 75–80% of the volume. If all of them go through the same model as the last one, you're multiplying the cost of simple operations by 10 for no reason.

What Model Routing Is and How to Implement It

Model routing is the architectural pattern that assigns each LLM call to the optimal model based on actual task complexity. In its simplest form, a lightweight classifier — often the cheapest model itself — evaluates the incoming task and decides which route to take.

type TaskComplexity = 'low' | 'medium' | 'high';

const COMPLEXITY_SIGNALS: Record<TaskComplexity, string[]> = {
  low: ['extract', 'classify', 'format', 'validate', 'check', 'parse'],
  medium: ['summarize', 'translate', 'rewrite', 'compare', 'filter'],
  high: ['generate', 'analyze', 'design', 'reason', 'strategize'],
};

function detectComplexity(taskDescription: string): TaskComplexity {
  const normalized = taskDescription.toLowerCase();
  for (const [level, signals] of Object.entries(COMPLEXITY_SIGNALS) as [TaskComplexity, string[]][]) {
    if (signals.some(s => normalized.includes(s))) return level;
  }
  return 'medium';
}

const MODEL_MAP: Record<TaskComplexity, string> = {
  low: 'claude-haiku-4-5-20251001',
  medium: 'claude-sonnet-4-6',
  high: 'claude-opus-5',
};

async function routedCall(taskDescription: string, prompt: string) {
  const complexity = detectComplexity(taskDescription);
  const model = MODEL_MAP[complexity];

  const response = await anthropic.messages.create({
    model,
    max_tokens: complexity === 'low' ? 256 : 1024,
    messages: [{ role: 'user', content: prompt }],
  });

  return { response, model, complexity };
}

This isn't magic — it's classification logic applied to infrastructure decisions. Just as you wouldn't run a distributed database to store ten rows of config, don't run your most expensive model to extract a number from a string.

The Math Nobody Does Until the Invoice Stings

With 10,000 calls per day and a realistic task mix:

Complexity	% of volume	Model	Cost/MTok input
Low	65%	Haiku	$0.80
Medium	25%	Sonnet	$3.00
High	10%	Opus	$15.00

Without routing — all Sonnet: ~$35/day → ~€1,050/month

With routing: ~$12/day → ~€360/month

Savings: €690/month, without changing a single line of business logic. And that's before counting latency. Haiku averages ~0.3s. Opus can take 8–12 seconds for complex reasoning. Correct routing reduces perceived response time across the entire application, not just the invoice.

If your agents have unexplained cost spikes or latency you can't account for, the AI integration service includes an architecture audit of LLM call patterns and model routing design from day one.

Where NOT to Apply Model Routing

Routing isn't free — it adds a decision layer and can misclassify. There are contexts where it's better to skip it:

Security-sensitive decisions. If the agent is deciding whether a user can access sensitive data, don't route. Always use your most robust model.

User-facing generation. If LLM output goes directly to a client as an official response, quality isn't negotiable.

Low volumes. Below 2,000–3,000 calls per day, the savings don't justify the added complexity of maintaining a router.

When error cost exceeds savings. If a misclassification sends a complex task to Haiku and the result is wrong, the operational cost of the error may exceed the API savings.

Advanced Version: LLM-Based Router

Once the basic router is working, the next step is using Haiku itself to classify complexity. It costs fractions of a cent extra per call but is far more robust against natural language variation:

async function llmRouter(taskDescription: string): Promise<TaskComplexity> {
  const response = await anthropic.messages.create({
    model: 'claude-haiku-4-5-20251001',
    max_tokens: 10,
    messages: [{
      role: 'user',
      content: `Classify this task as "low", "medium", or "high" complexity. Reply with one word only.\n\nTask: ${taskDescription}`,
    }],
  });

  const result = (response.content[0] as { text: string }).text.trim().toLowerCase();
  return (['low', 'medium', 'high'].includes(result) ? result : 'medium') as TaskComplexity;
}

Add automatic fallback: if Haiku fails or its output doesn't pass a confidence threshold, retry with Sonnet. This makes the routing layer transparent to the rest of the system.

The AI Driven Development service covers exactly these patterns — designing agent architectures that scale in capability without scaling the invoice along with them.

What to Measure Once the Router Is in Production

The router needs its own telemetry. At minimum:

Complexity distribution per hour — is it drifting toward more "high" calls?
Error rate per level — does Haiku fail more on certain task types?
p50/p95 latency per model — validates that the latency benefit is real
Daily cost breakdown by complexity — the metric that justifies the pattern to any stakeholder

Without that data, the router becomes another black box. Which is exactly what you were trying to avoid.

Conclusion

Model routing is one of those architectural changes teams keep postponing because "it already works." But that's precisely why API costs accumulate with no one able to justify them, and response times vary in ways nobody understands.

It's not a complex refactor. It's adding a classification layer and a model map. The return shows up in next month's invoice.

If you have agents in production and don't know what each type of call is costing you — or how much of that cost is actually necessary — let's talk for 30 minutes.