Your RAG isn't broken. It's answering a different question.
— 3 min read
Vector search finds text that resembles the question. When the answer has to be assembled by following relationships, that is the wrong operation — and no embedding model fixes it.
Your RAG pipeline works. Then someone asks: "which of our services break if the payments team's dependency goes down?"
You get five chunks about outages, three about payments, and nothing connecting them. The model assembles a confident answer out of fragments that were never actually related.
No embedding model fixes this. No larger top_k fixes it either.
Similarity is not connection
Vector search finds text that resembles your question. That is the right tool when the answer is sitting in a passage and you just need to find which one.
It fails when the answer is not in any chunk — when it has to be assembled by following relationships. No document contains the sentence "service X breaks if payments goes down." That fact lives in the links between three of them, and similarity has no way to say and then follow that.
CognoDB is the store that does the walking
CognoDB is a managed graph database built for retrieval. Across a ten-engine benchmark it posted the lowest two-hop latency and the highest read throughput of the group — which is the exact operation graph RAG performs on every query.
It speaks Bolt and Cypher. The official Neo4j drivers for Python, JavaScript, Go, Java and .NET connect unmodified, so adopting it is one line:
MATCH (e:Entity {name: $entity})<-[:ABOUT]-(f:Fact)-[:SOURCED_FROM]->(doc)
RETURN f.statement, doc.title ORDER BY f.observed_at DESC LIMIT 25
No new SDK, no proprietary query language, no rewrite (the docs have the specifics). Every Cypher tutorial and Stack Overflow answer already applies. Every instance also ships an MCP server, so an agent queries the graph as a tool with no glue code.
And to be clear about the architecture: this does not replace your vector store. The strongest systems use both — embeddings to find the entry point, traversal to expand and explain it.
The token bill is where you feel it
Prompt stuffing scales with your corpus, not your question. Every document you add makes every query more expensive, permanently.
Traversal scales with the neighbourhood. Retrieving the relevant subgraph over a 2,000-entity knowledge base costs about 2,668 tokens against 202,285 to dump the same base into the prompt — 98.7% fewer.
The ratio depends on your data. The curve does not: across a twentyfold increase in corpus size, graph retrieval stayed roughly flat at ~2,700 tokens.
You also get something a similarity score cannot give you — a path. When an auditor asks why the system answered as it did, "these three facts, from these two documents, connected in this order" is an explanation.
Three things worth building this week
- Agent memory that doesn't grow the prompt. The agent retrieves the neighbourhood it needs per turn, so cost stops climbing with session length.
- Impact analysis. Services, owners, dependencies. "What breaks, and who do I tell?" becomes one traversal.
- Entity resolution. Accounts, devices, cards. Fraud is rarely visible in one record — it is visible in the shape between them.
Point a driver at it and find out
If your retriever keeps returning plausible chunks that don't add up, you already have the diagnostic. Load a slice of your real corpus and run the traversal your vector store cannot express.
A free instance takes a minute and no card. It is a connection-string change either way, so an afternoon tells you whether your data has this shape.
