How to Evaluate LLM Agents: 5 Patterns That Catch Failures
Eval shortcuts fail on honesty-trained models. Five assertion patterns that survived production.
July 11, 2026 · 12 min
Cloud Architecture · AI Engineering · Distributed Systems
Latest · Post
Fixing two-column extraction broke academic title pages. The benchmark never noticed.
Eval shortcuts fail on honesty-trained models. Five assertion patterns that survived production.
July 11, 2026 · 12 min
Six retrieval decisions for agent RAG, each settled by a benchmark, not a vibe. Defaults included.
July 10, 2026 · 10 min
pdf_search went from a pivot tool to a terminal tool. The thesis still holds.
July 04, 2026 · 8 min
MCP tutorials stop at tool integration. The real work is auth, guardrails, and untrusted data.
April 11, 2026 · 7 min
Tool boundaries shape agent behavior more than prompts. What I learned shipping pdf-mcp.
April 04, 2026 · 10 min
Tools, resources, prompts, SQLite persistence, in-process testing. Under 100 lines of Python.
February 22, 2026 · 9 min
Circuit breakers, validation gates, sagas, budget guards. Five patterns I use in production.
February 12, 2026 · 10 min
Claude's limits: 100 pages visual, 30MB per file. Hit them? pdf-mcp reads in chunks.
January 31, 2026 · 10 min
What threading, multiprocessing, and async actually do inside a Lambda function. Benchmarked.
January 17, 2026 · 4 min
Three MCP design patterns from giving an agent full API access. And watching it go wrong.
December 24, 2025 · 4 min
How to wrap a legacy Redmine API with MCP. No rewriting, no destabilizing anything.
May 27, 2025 · 6 min
Hybrid wins at page grain. BM25 wins at section grain. Granularity decides.
June 06, 2026 · 10 min
Page-mode PDF search costs 2 to 6 extra tool calls per query depending on the document. Section-aware search delivers...
May 30, 2026 · 10 min
Architecture, error handling, monitoring, security, cost control. 8 disciplines production agents need.
March 11, 2026 · 10 min
I audited my MCP server at 3,000 downloads and found 8 vulnerabilities. Every fix is here.
February 25, 2026 · 7 min
Five lessons from real failures: structured output, constraints over roles, architecture over model.
November 29, 2025 · 4 min
Event-driven sync across the China boundary. No VPN, no persistent infrastructure.
April 12, 2025 · 5 min