Why Most Enterprise Agent Pilots Never Reach Production (And the 5 Patterns That Do)

We've seen more than 30 enterprise agent pilots in the last 18 months. About eight of them reached production. The other 22 are still "in progress" — which, in most organizations, means dead.

Pattern 1: Tool design that only works in demos

The most common failure isn't the model — it's the tools. Teams spend weeks on the agent orchestration and five minutes on tool design. The result: tools that return ambiguous outputs, don't handle partial failure, and give the model no way to recover from errors.

Production tool design requires: typed return values, error states the model can interpret and reason about, idempotency where it matters, and explicit documentation that gets injected into the system prompt.

Pattern 2: No eval harness means no improvement

If you can't measure it, you can't improve it. Agents that ship have an eval harness from day one — a set of test cases that represent real inputs and expected outputs. Teams that skip this spend months "tuning prompts" with no way to know if they're making things better or worse.

Pattern 3: Hallucinated output formats

Agents that generate structured output (JSON, function calls, API parameters) will eventually hallucinate the format. Without validation and retry logic at the tool layer, this crashes silently and creates subtle bugs that are nearly impossible to debug in production.

Pattern 4: No escalation path

Every agent needs a way to say "I can't handle this." Agents that only have a success path will hallucinate rather than admit uncertainty. Build explicit escalation into the tool set: `escalate_to_human(reason, context)` is often the most important tool in the set.

Pattern 5: Wrong scope for the pilot

The best pilots are narrow, high-volume, and low-risk. "Automate our entire customer support operation" is not a pilot — it's a project. "Handle the top 20% of ticket categories by volume, with a human review step" is a pilot that can succeed and prove ROI.

The three patterns that ship

The agents that reach production share three traits: they have narrow, well-defined tool sets; they have an eval harness with >50 test cases before going live; and they have explicit escalation logic baked in from the first iteration.

Everything else is secondary.