Prompt Regression in CI: Catch Agent Degradation Before You Ship
You tweak three lines of your system prompt. Better tone, clearer instructions, a constraint that was missing. Unit tests pass. Linter is green. You merge.
Two days later someone on the team says the agent "doesn't respond the same anymore." Support tickets creep up. The agent is producing technically valid outputs — but functionally worse ones. And the most painful part: you have no idea which commit caused it.
This is called silent prompt degradation. Unlike a code bug, it throws no exceptions and fails no conventional tests.
Why unit tests aren't enough for prompts
A unit test checks that a function returns a deterministic value given an input. AI agents are not deterministic. The same prompt can produce slightly different outputs on every call, and "slightly different" can mean "perfectly correct" or "functionally broken."
The most common failures that slip past conventional tests:
- Semantic regression: the agent answers the literal question but loses the tone, structure or context that made the response useful
- Lost implicit guardrails: an instruction that was limiting behavior disappears when you rephrase the prompt
- Format drift: the output JSON is still valid but field names shift slightly
- Personality fade: in agents with a defined role, the character gradually dissolves across prompt iterations
The root problem: you change something in natural language, and the effect can only be measured by running the agent. No compiler catches it.
The pattern: golden dataset + automated scoring
The solution is to treat prompts like code that needs a regression suite. The pattern has three components:
1. Golden dataset: a set of test cases with known input and expected output. They don't need to match the actual output exactly — they're quality references to score against.
2. Eval runner: runs the agent on each golden case with the new prompt, collects the responses.
3. Automated scorer: compares each response against the reference and assigns a score. A minimum threshold blocks the merge if it isn't reached.
// types/eval.ts
interface GoldenCase {
id: string;
input: string;
expectedOutput: string;
criteria: string[]; // which dimensions to score
}
interface EvalResult {
caseId: string;
actualOutput: string;
score: number; // 0.0 – 1.0
breakdown: Record<string, number>;
passed: boolean;
}
interface RunSummary {
totalCases: number;
passed: number;
failed: number;
avgScore: number;
failedCases: EvalResult[];
}
LLM-as-Judge: semantic scoring without deterministic tests
The scorer is the critical component. For natural language outputs, scoring must be semantic, not syntactic. This is where the LLM-as-Judge pattern comes in: you use another LLM (or the same one with a different prompt) to evaluate response quality.
// lib/eval-runner.ts
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const JUDGE_PROMPT = `
You are an AI agent quality evaluator.
Given an ACTUAL_OUTPUT and an EXPECTED_OUTPUT, score the output on these dimensions:
- accuracy: does it correctly answer the question or task? (0.0 – 1.0)
- format: does it follow the expected format? (0.0 – 1.0)
- tone: does it maintain the agent's expected tone and persona? (0.0 – 1.0)
Return ONLY JSON with this exact schema:
{ "accuracy": 0.0, "format": 0.0, "tone": 0.0, "overall": 0.0 }
`;
async function scoreCase(
actual: string,
expected: string,
criteria: string[]
): Promise<{ score: number; breakdown: Record<string, number> }> {
const response = await client.messages.create({
model: "claude-opus-5-5",
max_tokens: 256,
messages: [
{
role: "user",
content: `EXPECTED_OUTPUT:\n${expected}\n\nACTUAL_OUTPUT:\n${actual}\n\nCRITERIA TO EVALUATE: ${criteria.join(", ")}`,
},
],
system: JUDGE_PROMPT,
});
const text =
response.content[0].type === "text" ? response.content[0].text : "{}";
const parsed = JSON.parse(text);
return {
score: parsed.overall ?? 0,
breakdown: parsed,
};
}
export async function runEvals(
dataset: GoldenCase[],
agentFn: (input: string) => Promise<string>,
threshold = 0.75
): Promise<RunSummary> {
const results: EvalResult[] = [];
for (const c of dataset) {
const actualOutput = await agentFn(c.input);
const { score, breakdown } = await scoreCase(
actualOutput,
c.expectedOutput,
c.criteria
);
results.push({
caseId: c.id,
actualOutput,
score,
breakdown,
passed: score >= threshold,
});
}
const passed = results.filter((r) => r.passed).length;
return {
totalCases: results.length,
passed,
failed: results.length - passed,
avgScore: results.reduce((a, r) => a + r.score, 0) / results.length,
failedCases: results.filter((r) => !r.passed),
};
}
Two important notes about this pattern:
- Use a smarter model than the one you're evaluating when possible. If your agent runs on Haiku, evaluate with Sonnet. The judge must be more reliable than the evaluated.
- The judge prompt is also code. It must be versioned and part of the same regression pipeline.
Wiring it into GitHub Actions
The critical point is that this runs on every PR, not just on releases. A prompt change that passes conventional tests but drops the eval score below threshold automatically blocks the merge.
# .github/workflows/eval.yml
name: Prompt Regression Eval
on:
pull_request:
paths:
- "prompts/**"
- "lib/agent/**"
jobs:
eval:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
cache: "npm"
- run: npm ci
- name: Run prompt regression eval
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: npx tsx scripts/run-evals.ts
- name: Upload eval report
if: always()
uses: actions/upload-artifact@v4
with:
name: eval-report
path: eval-results.json
// scripts/run-evals.ts
import { runEvals } from "../lib/eval-runner";
import { goldenDataset } from "../tests/golden-dataset";
import { myAgent } from "../lib/agent";
import * as fs from "fs";
const THRESHOLD = 0.75;
const summary = await runEvals(goldenDataset, myAgent, THRESHOLD);
fs.writeFileSync("eval-results.json", JSON.stringify(summary, null, 2));
console.log(`\n📊 Eval summary:`);
console.log(
` Passed: ${summary.passed}/${summary.totalCases} (avg score: ${summary.avgScore.toFixed(2)})`
);
if (summary.failed > 0) {
console.error(`\n❌ ${summary.failed} cases below threshold:`);
for (const f of summary.failedCases) {
console.error(` ${f.caseId}: score ${f.score.toFixed(2)}`);
}
process.exit(1); // CI fails, merge blocked
}
console.log(`\n✅ All cases above threshold. Safe to deploy.`);
The paths filter on the GitHub Actions workflow is essential: evals only trigger when prompts or agent code change, not on every PR in the project.
What golden dataset you need (and how many cases)
The practical question: how many cases do you need? The honest answer: 20 well-chosen cases are worth more than 200 generic ones.
Criteria for building a good golden dataset:
- Cover your main happy paths: the flows used by 80% of your users
- Include known edge cases: inputs that have historically caused problems
- Capture past regressions: when you catch a behavior bug in prod, add that case to the dataset
- Vary tone and difficulty: short inputs, long inputs, ambiguous inputs, multi-language if relevant
The golden dataset grows over time. Every production bug you catch is a new case to add.
The cost of not having this
A prompt change that degrades behavior and reaches production has a real cost: extra support load, users leaving, team confidence in the agent slowly eroding. The most dangerous part is that it's hard to spot — the agent "works," it just works worse.
A regression pipeline that runs in 3-5 minutes per PR and blocks problematic merges is the difference between iterating with confidence and moving with fear of breaking things.
If you're building AI agents in production and want to implement a solid eval system, at DAILYMP we bring this type of architecture to teams that already have agents running. You can also review how we structure output validation guardrails for the layer that complements this system.
Does your team need an eval system for your agents? → Let's talk for 30 min