AI Agent Middleware: Stop Duplicating Tool Logic
You have five agent tools. You need logging in each. And caching. And permission checks. And per-user rate limiting.
If you implement these directly in each tool, you write the same boilerplate five times. The sixth tool takes longer to wire up than to implement. And when you migrate your log format to Datadog, you edit five files.
This is the problem the middleware pattern solves when applied to AI agents.
The problem: cross-cutting logic scattered across every tool
In a production agent, every tool needs the same infrastructure layers:
- Logging: what the model called, with what args, how long it took, what it returned
- Caching: if the result is the same for the same inputs, don't hit the external API
- Auth: does this user have permission to run this tool
- Rate limiting: this tenant can't call
queryDB()more than 50 times per minute
With five tools and four concerns, you have 20 potentially identical code blocks. When the team grows and tool number twelve arrives, the new developer implements logging with a different format because nobody told them where the standard was. They won't find it — it's spread across twelve files.
The solution isn't documentation. It's code that makes it impossible to skip the logic.
The middleware pattern applied to agent tools
The idea is simple: instead of executing tools directly, you wrap each execution in a chain of functions that run before and after.
The middleware type
type ToolHandler = (input: unknown) => Promise<unknown>
type Middleware = (
toolName: string,
handler: ToolHandler,
next: ToolHandler
) => ToolHandler
Each middleware receives the tool name, the original handler, and a next function representing the rest of the chain. It can run logic before calling next, after, or skip it entirely — that's the short-circuit for cache hits.
Composing middlewares
function composeMiddlewares(
middlewares: Middleware[],
handler: ToolHandler,
toolName: string
): ToolHandler {
return middlewares.reduceRight(
(next, middleware) => middleware(toolName, handler, next),
handler
)
}
The four essential middlewares in production
Logger: single source of truth
const loggerMiddleware: Middleware = (toolName, _handler, next) => {
return async (input) => {
const start = Date.now()
try {
const result = await next(input)
console.log({
tool: toolName,
input,
durationMs: Date.now() - start,
status: 'ok'
})
return result
} catch (err) {
console.error({
tool: toolName,
input,
durationMs: Date.now() - start,
status: 'error',
err
})
throw err
}
}
}
Instead of logs scattered across each tool, you have a single source of truth. Switch the destination to Datadog or Sentry in one place.
Cache with TTL: fewer external API calls
const cache = new Map<string, { value: unknown; expiresAt: number }>()
const cacheMiddleware = (ttlMs = 60_000): Middleware =>
(toolName, _handler, next) => {
return async (input) => {
const key = `${toolName}:${JSON.stringify(input)}`
const cached = cache.get(key)
if (cached && cached.expiresAt > Date.now()) return cached.value
const result = await next(input)
cache.set(key, { value: result, expiresAt: Date.now() + ttlMs })
return result
}
}
A tool like searchCRM({ query: "Acme" }) can return the same result for the next 60 seconds. With this middleware you don't hit Salesforce more than necessary. In a high-volume agent, that directly reduces latency and cost.
Rate limiting per tool and user
const rateLimiter = new Map<string, number[]>()
const rateLimitMiddleware = (maxCalls: number, windowMs: number): Middleware =>
(toolName, _handler, next) => {
return async (input) => {
const userId = (input as any)?.__userId ?? 'anonymous'
const key = `${userId}:${toolName}`
const now = Date.now()
const calls = (rateLimiter.get(key) ?? []).filter(t => t > now - windowMs)
if (calls.length >= maxCalls) {
throw new Error(`Rate limit exceeded for ${toolName}`)
}
rateLimiter.set(key, [...calls, now])
return next(input)
}
}
Integrating with the Anthropic SDK
Your tools are defined as objects with name, description and inputSchema. Middleware applies to the execution handler, not the definition that the model sees:
const middlewares = [
loggerMiddleware,
cacheMiddleware(30_000),
rateLimitMiddleware(100, 60_000),
]
function wrapTools(tools: Record<string, ToolHandler>) {
return Object.fromEntries(
Object.entries(tools).map(([name, handler]) => [
name,
composeMiddlewares(middlewares, handler, name)
])
)
}
const wrappedTools = wrapTools({
searchCRM: async (input) => { /* CRM logic */ },
queryDB: async (input) => { /* DB logic */ },
sendEmail: async (input) => { /* email logic */ },
})
The tool definitions you pass to the model stay unchanged. Only how they execute changes.
If you're already implementing dynamic tool selection by relevance, middleware applies at exactly the same point: in the executor, not the registry.
What changes in production
With this pattern:
- Add a new tool = implement the handler. Logging, caching, auth and rate limiting apply automatically.
- Change log format = edit one file.
- Disable cache for debugging = remove one middleware from the array.
- Audit which tools get called most = read the logs, which exist by default.
- Add a new concern = write one middleware, add it to the array.
Cross-cutting technical debt doesn't disappear. It centralizes where it can actually be managed.
When you build a real production agent, this architecture is the difference between a system that evolves cleanly and one that becomes a maze of duplicated logic. At DAILYMP we apply it from the first tool, not when the agent already has twelve. If you need help structuring your AI integration, we can review it together.
Does your agent need proper architecture before it grows out of control? Let's talk on WhatsApp and we'll figure out how to structure your implementation.