Have you ever struggled with your RAG/agent working perfectly on toy documents, but falling apart on real PDFs with OCR, tables, multi-lingual, or long hops? It’s probably one of a small number of repeatable failure modes. I’ve compiled a Problem Map with reproducible tests and minimal fixes, no fine-tuning or model jailbreaks required. This map works with Claude, GPT-4/5, Qwen, and Llama via Ollama.
The Problem Map is a collection of 16+ failure modes with tiny failing datasets or synthetic reproducers, exact stages that break, and minimal fixes. It’s designed to help you identify and fix the root causes of your RAG/agent’s failures.
Each failure mode has a concise explanation, a reproducible test, and a minimal fix. You can try the fixes today and see the improvements for yourself. The map is open-source, MIT-licensed, and has no tracking or SaaS dependency.
I’m looking for edge cases I might have missed, so if you have a corpus where correct citations still yield wrong answers, I’d love to add the case and fix. Let’s work together to make our RAG/agents more reliable and efficient.