AI Agent Map-Reduce: Process Long Docs Without Overflow

The client sends a 180-page contract. Or 200 invoices in a ZIP file. Or a 300-page annual report that needs summarizing, classifying, and data extraction.

Your AI agent receives it. And crashes.

Not because the model is bad. Because no current LLM can ingest 180 pages in a single call and return a coherent, reliable response. Context has a hard limit, cost per token scales non-linearly, and the longer the input, the more quality degrades — a well-documented phenomenon called lost in the middle.

The problem isn't technological. It's architectural. And it has a solution.

Why the obvious approaches fail

The first instinct when you get a long document is to reach for the model with the biggest context window. Claude has 200K tokens. GPT-4 has 128K. Should be enough, right?

No.

Problem 1: cost. A 200-page document is roughly 150,000 tokens. At current premium model pricing, processing that document with Sonnet costs between €0.90 and €1.50 per call. If you have 50 contracts a month, that's €75 just for that process — before retries, reprocessing, and errors.

Problem 2: quality. Models have a documented failure mode: information at the beginning and end of the context window is retained well; the middle, not so much. A 180-page contract has most of the relevant clauses between pages 40 and 140. Exactly where the model loses focus.

Problem 3: timeouts. A call with 150K input tokens can take 30-60 seconds to respond. In a serverless architecture like Vercel, that's a guaranteed timeout. In your own infrastructure, it's a resource blockage you can't sustain at real concurrency.

RAG (retrieval-augmented generation) solves a different problem: it helps you find relevant documents in a knowledge base. It's not designed to process a single long document from start to finish extracting structured information.

The right pattern is map-reduce.

The map-reduce pattern for long documents

The concept comes from distributed computing circa 2004: divide the problem into independent parts (map), process them in parallel, then combine the results (reduce).

Applied to AI agents with long documents:

1. Chunk — split the document into manageable fragments (typically 10-25 pages each)
2. Map — run a lightweight LLM agent over each chunk in parallel using Promise.all()
3. Reduce — a final agent synthesizes the partial results into a coherent response

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic();

type ChunkResult = {
  chunkIndex: number;
  keyPoints: string[];
  entities: string[];
  flags: string[];
};

async function processChunk(
  text: string,
  index: number
): Promise<ChunkResult> {
  const response = await anthropic.messages.create({
    model: "claude-haiku-4-5-20251001",
    max_tokens: 1024,
    messages: [
      {
        role: "user",
        content: `Analyze this document fragment (pages ${index * 25 + 1}-${(index + 1) * 25}).

Extract as JSON with this exact schema:
{
  "keyPoints": ["key point 1", "key point 2"],
  "entities": ["company X", "person Y", "date Z"],
  "flags": ["problematic clause if any"]
}

FRAGMENT:
${text}`,
      },
    ],
  });

  const content = response.content[0];
  if (content.type !== "text") throw new Error("Unexpected response type");

  const parsed = JSON.parse(content.text);
  return { chunkIndex: index, ...parsed };
}

async function reduceResults(
  results: ChunkResult[],
  task: string
): Promise<string> {
  const summary = results
    .map(
      (r) =>
        `[Chunk ${r.chunkIndex + 1}]\n` +
        `Key points: ${r.keyPoints.join(", ")}\n` +
        `Entities: ${r.entities.join(", ")}\n` +
        `Flags: ${r.flags.join(", ") || "none"}`
    )
    .join("\n\n");

  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 2048,
    messages: [
      {
        role: "user",
        content: `You have the results of a section-by-section analysis of a ${results.length * 25}-page document.

TASK: ${task}

RESULTS BY SECTION:
${summary}

Synthesize all the information into a coherent report, without repeating information and prioritizing the most relevant findings.`,
      },
    ],
  });

  const content = response.content[0];
  if (content.type !== "text") throw new Error("Unexpected response type");
  return content.text;
}

export async function processLongDocument(
  documentText: string,
  task: string,
  chunkSize = 8000 // ~25 pages in tokens
): Promise<string> {
  // 1. CHUNK: split the document
  const words = documentText.split(" ");
  const chunkWords = Math.floor(chunkSize / 1.3); // approx tokens → words
  const chunks: string[] = [];

  for (let i = 0; i < words.length; i += chunkWords) {
    chunks.push(words.slice(i, i + chunkWords).join(" "));
  }

  // 2. MAP: process in parallel with lightweight model
  const mapResults = await Promise.all(
    chunks.map((chunk, i) => processChunk(chunk, i))
  );

  // 3. REDUCE: synthesize with more capable model
  return reduceResults(mapResults, task);
}

Key production optimizations

Use different models at each phase. In the map phase, each chunk is a simple extraction task: Haiku handles it as well as Sonnet at a fraction of the cost. In the reduce phase, synthesis requires more complex reasoning: that's where you use the more capable model. This assignment can cut total cost by 60-70% compared to using Sonnet for everything.

Control chunk overlap. When you split a document, sentences that fall on chunk boundaries get cut. A 200-300 token overlap between consecutive chunks prevents losing critical information at the edges. Add this to the chunking loop: instead of i += chunkWords, use i += chunkWords - overlap.

Filter empty results before reduce. Not every chunk has relevant information. A 200-page contract has 40 pages of boilerplate definitions that contribute nothing to key clause extraction. Filter before reduce: mapResults.filter(r => r.keyPoints.length > 0).

Handle errors per chunk, not per document. If a chunk fails (timeout, JSON parse error), don't invalidate the whole process. Use Promise.allSettled() instead of Promise.all() and process successful results, flagging failed ones for selective reprocessing.

This is part of what we build into AI integration projects: architectures that survive the real world, where documents arrive in unexpected formats, timeouts happen, and cost needs to be predictable.

When to use map-reduce vs RAG vs direct context

The choice depends on the problem:

• Direct context — short document (under 20 pages), you need full coherence across all parts, you can't afford inter-chunk information loss. The practical quality ceiling is around 50-80K tokens with current models.

• RAG — you have a large knowledge base (hundreds or thousands of documents) and users ask questions about it. You retrieve the most relevant fragments by semantic similarity. Not designed for processing a single long document.

• Map-reduce — single long document (50+ pages), the task is extraction, classification, or summarization, and information can be analyzed in sections independently. Most real business documents fall here: contracts, reports, case files, transaction histories.

The common trap is using RAG for everything because the pipeline is already in place. RAG applied to a single 200-page document is like using a search engine to read a book: technically it works, but it's not what it's for.

If you have a process where a team manually processes long documents — reviewing contracts, extracting data from reports, classifying case files — the map-reduce pattern is likely the missing piece for automating it reliably with predictable cost. It's exactly the kind of problem we solve with the AI agents and integration service.

---

Got a document review process that eats hours of your team's time? Tell me what it looks like — we can work out in 30 minutes whether it makes sense to automate it.