Few-shot in AI agents: one example beats 500 words
You've spent two hours refining the system prompt. You've added three paragraphs about tone, four about format, two about edge cases. The agent still misclassifies 30% of inputs.
The problem isn't the model. It's the strategy: LLMs understand examples better than instructions. That has a name — few-shot prompting — and most teams underuse it because nobody taught them when it applies.
Long instructions vs examples: why the model gets lost
When you write a long instruction in a system prompt, the model follows it approximately. The longer and more conditional it gets ("if field A is empty but field B has a value, then..."), the more it drifts from the behavior you expect.
LLMs are fundamentally pattern completion engines. Give them a pattern and they complete what follows. When you provide concrete input → output examples, you define the pattern directly. Instructions abstract that pattern; examples demonstrate it.
A system prompt with 600 words of instructions often loses to one with 150 words of instructions plus 3 well-chosen examples. That's not intuition — it's how transformers work internally.
When to use few-shot (and when not to)
Few-shot isn't a universal solution. It works well in these scenarios:
Use it when:
- You need a specific output format that doesn't emerge well from description alone
- The agent makes nuanced classifications that are hard to articulate verbally
- You want tone consistency — examples capture register better than adjectives
- There are edge cases that inevitably appear in production
Don't use it when:
- The task is simple and zero-shot works fine (examples waste tokens)
- Example content can go stale quickly
- You need different examples per user — better to make it dynamic (see below)
- Token budget is tight and each call already has a long context
How to structure few-shot in your system prompt
The cleanest structure in TypeScript with the Anthropic SDK is to separate examples inside the system prompt using clear XML tags:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const SYSTEM_PROMPT = `You classify support tickets into three categories: URGENT, NORMAL, or INFORMATIONAL.
A ticket is URGENT if the user cannot operate in production.
A ticket is NORMAL if there is a functional issue with a workaround.
A ticket is INFORMATIONAL if it's a question or feature request.
<examples>
<example>
<ticket>Payment gateway returns 500 and we cannot process any charges</ticket>
<classification>URGENT</classification>
</example>
<example>
<ticket>The date filter in reports breaks when the month has 31 days</ticket>
<classification>NORMAL</classification>
</example>
<example>
<ticket>Can you add PDF export to the reports section?</ticket>
<classification>INFORMATIONAL</classification>
</example>
</examples>
Reply with only the classification word.`;
async function classifyTicket(ticket: string): Promise<string> {
const response = await client.messages.create({
model: "claude-haiku-4-5-20251001",
max_tokens: 20,
system: SYSTEM_PROMPT,
messages: [{ role: "user", content: ticket }],
});
return (response.content[0] as { text: string }).text.trim();
}
Three details that matter in this pattern:
- The cheapest model (Haiku) is enough for classification with few-shot. You don't need Opus.
- Examples come after the short instructions, not before.
max_tokens: 20is intentional — the output is one word, don't leave it open.
If you're building classification or extraction agents, AI integration services can save you weeks of trial and error.
Dynamic few-shot: relevant examples in real time
Static few-shot has a ceiling: your 3-5 examples have to cover the space of possible cases. When the domain is broad, dynamic few-shot solves it.
The idea: you store a collection of validated input→output pairs in a database. On each call, you retrieve the N most similar to the current input and insert them dynamically into the prompt.
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
interface Example {
input: string;
output: string;
}
// In production: semantic search against your collection
async function retrieveRelevantExamples(
query: string,
n: number = 3
): Promise<Example[]> {
// Call your vector DB here (pgvector, Pinecone, etc.)
// Returns the N examples most similar to the current query
return examplesDB.slice(0, n); // simplified
}
async function dynamicAgent(input: string): Promise<string> {
const examples = await retrieveRelevantExamples(input);
const formattedExamples = examples
.map(
(e) => `<example>
<input>${e.input}</input>
<output>${e.output}</output>
</example>`
)
.join("\n");
const systemPrompt = `You are a technical support classification agent.
<relevant_examples>
${formattedExamples}
</relevant_examples>
Classify the ticket following the pattern from the examples.`;
const response = await client.messages.create({
model: "claude-haiku-4-5-20251001",
max_tokens: 50,
system: systemPrompt,
messages: [{ role: "user", content: input }],
});
return (response.content[0] as { text: string }).text.trim();
}
Dynamic few-shot has a latency cost (semantic search adds ~50-150ms), but the result is noticeably more accurate when the domain has many distinct cases. If you're building this type of agent, AI Driven Development provides the architecture so you don't reinvent the wheel.
Three cases where few-shot unlocked what instructions couldn't
Support email classification: A team had 400-word instructions for categorizing incoming emails. Error rate: 28%. We replaced it with 8 representative examples. Error rate: 6%. Processing time: the same. Token cost: slightly higher, but savings from fewer errors justified the change.
Brand-tone response generation: A customer service agent generated correct answers but with corporate phrasing that didn't fit the brand. Three examples of real responses written by the team fixed it where four paragraphs of tone description had failed.
Contract data extraction: Extracting fields from contracts with variable formats required handling dozens of different ways of writing dates, amounts, and parties. Static few-shot with 5 examples covered 80% of cases. Dynamic few-shot with 50 examples in a database covered 96%.
Why this matters in a real agent
In a production agent, every conversation turn consumes tokens. The difference between an 800-word instruction system prompt and a 300-word examples-based one can be 500 tokens per call. With 100,000 calls per month, that's real money and real latency.
Examples are also easier to maintain than instructions: when behavior needs to change, you add or modify an example. You don't have to rewrite nested conditional logic.
Are you building an agent that classifies, extracts, or generates formatted text and the results aren't consistent? Send me a message on WhatsApp and we'll review whether dynamic few-shot could be the missing lever in your pipeline.