Why Basic RAG Fails at Multi-Hop Reasoning  🔗 **Source:** [Why Basic RAG Fails at Multi-Hop Reasoning](https://thenewstack.io/graphrag-multi-hop-reasoning-python/) > [!note]+ Document Notes > > [!info] Provenance > > **Analysis generated:** 2026-08-27 by Claude (Opus 5) via the `readwise-deep-read` pipeline. > > **Source:** The New Stack — https://thenewstack.io/graphrag-multi-hop-reasoning-python/ > > **Published:** 2026-08-27 · **Length:** 1,007 words · 8 headings · article (technical tutorial) > > **Byline:** **Not present in the captured markup.** Reader's `author` field is the bare domain. No individual author is asserted. > > **Register key:** `[STATED]` author-stated · `[EVIDENCE]` demonstrated · `[INFERENCE]` reviewer inference · `[RECOMMEND]` reviewer recommendation. > > **Coverage:** Full article read. **All four outbound links point to The New Stack's own prior articles** — no link to Neo4j documentation, the `neo4j-graphrag` package, or the GraphRAG literature. `[EVIDENCE]` **The code blocks themselves are largely absent from the extraction** — only the `pip install` line survives — so the four implementation steps are assessed as *described*, not as written. > > **Extraction artifact:** `[EVIDENCE]` A promotional link — *"A Claude Code skill was eating 200,000 tokens before answering a single question"* — is captured mid-paragraph, splitting the Acme/BetaTech example. Not the author's text. > > # 🏷️ Title: Why basic RAG fails at multi-hop reasoning (and how GraphRAG fixes it) > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References `[EVIDENCE]` Four links, **all to The New Stack's own prior coverage**. Named but unlinked: - **Neo4j** and the **`neo4j-graphrag`** package — the framework the entire tutorial depends on - **Cypher** — Neo4j's query language (see [the spelling problem](#the-spelling-problem-is-not-cosmetic)) - `VectorCypherRetriever`, `FixedSizeSplitter` — the two classes doing the work - GPT-4o, as the extraction model ## 📝 Summary **The diagnosis, and it is correct.** `[STATED]` *"The standard RAG assumes that semantic similarity implies relevance, which is not necessarily the case."* **The demonstration is the article's best asset** — two chunks and one question: > `[STATED]` Chunk 1: *"Acme Corp acquired BetaTech in 2022."* Chunk 2: *"Sarah Connor was appointed CEO of BetaTech in 2023."* > Query: *"Who leads the company that Acme Corp acquired?"* `[STATED]` *"chunks with high similarity to 'Acme Corp' and 'leadership' will be found, but Chunk 2 will be overlooked because 'Sarah Connor' and 'CEO of BetaTech' have nothing to do with 'Acme Corp' semantically."* `[INFERENCE]` **Two sentences, and the failure is undeniable.** The answer is present in the corpus, distributed across chunks that share no vocabulary with the query, and no amount of better embedding recovers it — because the missing thing is a *relationship*, not a similarity. **The proposal.** Use an LLM at ingestion to extract entities and relationships into a Neo4j graph; at query time, find an entry node by vector search, then traverse relationships to assemble a subgraph. **The honest part, and the reason to read it.** `[STATED]` Three trade-offs: ingestion cost is high (an LLM call per chunk), schema design is mandatory (*"the moment you do not provide a proper schema, your LLM will hallucinate entity labels, duplicating nodes of 'Acme Corp', 'Acme Corporation,' and 'Ame'"*), and — the underrated one — `[STATED]` *"Observability is superior: To debug your naive RAG, you need to examine big floating-point arrays. To debug GraphRAG, you only need to look at your Neo4j dashboard to view the nodes."* **Bottom line.** `[INFERENCE]` A clear, correct explanation of a real failure mode, with **one substantive gap**: the title promises multi-hop and **the implementation described performs a single hop**, working on the example only because the entry point happens to be the middle node. ## 📖 Main Content ### The title promises multi-hop; the implementation does one hop `[STATED]` Step 4: *"We will embed the query, identify the semantic entry point, and finally perform a Cipher query on all the **1-hop connected neighbors**."* `[INFERENCE]` **Trace the worked example against that and something becomes visible.** The graph is: ``` (Acme Corp) -[ACQUIRED]-> (BetaTech) <-[LEADS]- (Sarah Connor) ``` The query is *"Who leads the company that Acme Corp acquired?"* — which is **two hops from Acme Corp**: Acme → BetaTech → Sarah Connor. `[EVIDENCE]` But the retrieval enters at the **semantic entry point**, and the article reports the retrieved context as *"BetaTech, its acquisition by Acme Corp, and the fact that Sarah Connor leads it."* `[INFERENCE]` **The entry point is BetaTech**, and from BetaTech both facts are one hop away. **The example is solvable in one hop only because the entry node sits in the middle of the chain.** Move the query one step further out — *"Who leads the companies acquired by the firm that acquired BetaTech?"* — and one hop fails exactly as vector search did. `[RECOMMEND]` **Traversal depth is a parameter you have to choose, and the article never says so.** Anyone implementing this should treat depth as a design decision with a cost curve: each additional hop expands the subgraph combinatorially, which is why 1-hop is a sensible default and why the article is right to use it. **What is missing is naming it as a limit**, so the reader knows what to change when a genuinely three-entity question fails. `[INFERENCE]` This is not a fatal objection — the technique is real and the entry-point-plus-neighbourhood pattern solves a large class of questions vector search cannot. It is a **gap between the title's promise and the implementation's reach**, and it is the kind of thing that surfaces in production rather than in a tutorial. ### Entity resolution is the hard part and gets one sentence `[STATED]` *"The moment you do not provide a proper schema, your LLM will hallucinate entity labels, duplicating nodes of 'Acme Corp', 'Acme Corporation,' and 'Ame.'"* `[INFERENCE]` **This is the single most practically important line in the article and it is a warning rather than a solution.** A schema constrains *labels* — that a node is a `Company` rather than an `Organisation` — and does considerably less for *identity*: whether "Acme Corp" appearing in document 40 is the same node as "Acme Corporation" in document 12. `[INFERENCE]` **Entity resolution is where real GraphRAG deployments fail**, and it fails silently. A duplicated entity does not error — it produces a graph where traversal from one node misses facts attached to its twin, which looks exactly like the original problem the graph was built to solve. The article's own example is the illustration: if the acquisition attaches to "Acme Corp" and a later filing attaches to "Acme Corporation," the multi-hop query breaks and the failure is invisible in the query result. `[RECOMMEND]` Anyone building this needs a **deduplication and canonicalisation pass** after ingestion, and the article does not mention that one exists. `[INFERENCE]` **A schema is necessary and not sufficient**, and treating it as sufficient is the most likely way to follow this tutorial correctly and still get a broken graph. ### The observability argument is the best reason to adopt, and it is buried in a bullet `[STATED]` *"To debug your naive RAG, you need to examine big floating-point arrays. To debug GraphRAG, you only need to look at your Neo4j dashboard to view the nodes. It is easy to see if the LLM has hallucinated."* `[INFERENCE]` **This deserves to be the headline rather than the third trade-off.** The stated benefit of GraphRAG is retrieval quality; the *durable* benefit is that **the retrieval step becomes inspectable.** With vector search, a wrong answer gives you a similarity score and no account of why those chunks were selected. With a graph, the retrieval is a path you can read. `[INFERENCE]` **And it pairs with a finding from elsewhere in this batch, on the same axis and in the opposite direction.** The Syntax episode covered four CSS features that move computation out of JavaScript and into CSS, and the cost I recorded there was exactly this: **JavaScript layout math is ugly and inspectable; CSS layout math is elegant and opaque** — no breakpoints, no unit tests, no stack trace. **Same variable, opposite direction. GraphRAG trades elegance for inspectability; declarative CSS trades inspectability for elegance.** `[INFERENCE]` Which yields the general point neither article makes: **the debuggability of a system is a design variable you choose, not a property you inherit** — and it is usually traded away for concision without anyone recording the trade. ### The spelling problem is not cosmetic `[EVIDENCE]` The article writes **"Cipher"** three times — *"dynamic Cipher traversal,"* *"The vector Cipher retriever,"* *"perform a Cipher query"* — and **"Cypher"** once, inside the class name `VectorCypherRetriever`. `[INFERENCE]` Neo4j's query language is **Cypher**. `[RECOMMEND]` This matters practically rather than pedantically: **a reader who searches "Cipher query Neo4j" finds cryptography.** The one correct spelling is inside a class name, where it is least likely to be read as the language's name. `[INFERENCE]` The inconsistency — correct in code, wrong in prose, three times to one — reads like prose written separately from the code it describes. ### What this is actually useful for `[INFERENCE]` The article frames the use case as enterprise document retrieval, and the reasoning applies anywhere **the answer is distributed across sources that share no vocabulary.** Two observations that follow: **The failure mode is specific and diagnosable.** `[RECOMMEND]` Before adopting anything, characterise your failed queries. **If they fail because the relevant text was never retrieved, that is the vector-search defect this article describes and a graph addresses it. If they fail because the right text was retrieved and the model reasoned badly, a graph changes nothing.** The article does not offer that diagnostic, and it is the first thing to establish. **A vault with explicit links is already the structure being proposed.** `[INFERENCE]` The article's argument is that *explicit typed relationships* outperform *inferred semantic similarity* for questions spanning multiple documents. **A wiki-linked note corpus already has explicit relationships** — hand-authored, curated, and typed by convention rather than by an LLM at ingestion. The expensive part of GraphRAG, the part requiring a GPT-4o call per chunk, is the part such a corpus does not need. `[INFERENCE]` The instructive difference is what a graph adds that links alone do not: **typed edges and traversal**. `[[Acme Corp]]` records *that* two notes relate; `-[ACQUIRED]->` records *how*, which is what makes multi-hop traversal answerable rather than merely followable. `[RECOMMEND]` **The cheap, useful version of this idea is not building a knowledge graph — it is making link types explicit where they carry meaning**, which is a note-writing habit rather than an infrastructure project. ### The trade-off the article states and does not quantify `[STATED]` *"Ingestion cost is high, whereas RAG costs are low because embedding models are inexpensive. To ingest each document chunk in GraphRAG, you need an LLM like GPT-4o to extract entities from the document. Filter your enterprise data carefully."* `[EVIDENCE]` **No number.** Not a per-chunk cost, not a ratio against embedding, not a worked example on a stated corpus size. `[INFERENCE]` **Third instance of this pattern in this batch.** Wordfence reported *"roughly two hours"* for the Argus run that succeeded, with no count of targets attempted. Graze reported 41 billion posts and 7,000 feeds with **no creator payout figure**. Here: *"ingestion cost is high"* in an article whose purpose is helping a reader decide whether to pay it. `[RECOMMEND]` **A single worked example — 1,000 chunks at model X costs approximately Y — would convert the article's central trade-off from a caution into a decision.** `[EVIDENCE]` The rate-limit warning, by contrast, is specific and useful: `[STATED]` *"Without configuring a rate-limit handler, your pipeline will fail with a RateLimitError on the first real document."* `[INFERENCE]` That is the kind of operational detail that only comes from having run it. ## 🗒️ Notes - **The opening framing is sharper than the usual version:** `[STATED]` *"According to the echo chamber's view, solving LLM hallucinations is easy: simply… break your PDFs into 1,000-token chunks, embed them, insert them into a vector database, and perform cosine similarity searches. It works perfectly well…until you actually deploy it."* `[INFERENCE]` Naming the received recipe precisely, then locating the failure at deployment rather than in principle, is the right structure. - **The global-summarisation failure is a second, distinct defect** and gets one line: `[STATED]` *"RAG also falls apart at global summarization ('what are the major risk factors discussed in all of our compliance reports?')."* `[INFERENCE]` This is a **different** problem from multi-hop — it needs corpus-wide aggregation, not relationship traversal — and the graph approach described does not obviously solve it either. Raised and dropped. - **The engineering note is honest and easy to miss:** `[STATED]` *"Under the hood, this pipeline still splits your text using a FixedSizeSplitter. The difference is that it uses the LLM to extract the semantic architecture from those chunks before saving them."* `[INFERENCE]` **Chunking does not go away** — the graph is built *on top of* the same fixed-size splitting, so chunk-boundary artifacts still affect what the extractor sees. A useful corrective to the impression that GraphRAG replaces chunking. - **"Stop doing it randomly" is the article's actual thesis** and is better than the title: `[STATED]` *"Chunking text is fine, but you should stop doing it randomly."* - **The pull-quotes repeat body text verbatim** — three of them, each duplicating a sentence already present. `[INFERENCE]` A CMS convention that inflates the word count and, in the extraction, makes the article read as if it repeats itself. - **Dependencies are named precisely:** `[STATED]` `pip install neo4j neo4j-graphrag openai python-dotenv`, plus *"an active Neo4j database (local or cloud)."* `[INFERENCE]` Naming the infrastructure requirement up front is right — **Neo4j is the real adoption cost**, not the Python packages, and the article does not soften it. - **`python-dotenv` and the credential note are a small good sign:** `[STATED]` *"Production-ready pipelines require proper credential management and resilience."* `[INFERENCE]` Tutorials that hardcode API keys are common; this one does not. ## 📚 References `[EVIDENCE]` **Four links, all internal to The New Stack.** `[EVIDENCE]` **Neo4j is not linked. `neo4j-graphrag` is not linked. The GraphRAG literature is not cited.** One of the four links is a promotional insert unrelated to the subject. `[INFERENCE]` For a hands-on tutorial this is a real defect: **the reader is told to install a package and stand up a database, and given no route to either's documentation.** The framework is described as making the architecture *"accessible to everyone"* — accessibility that the article then withholds by not pointing at it. `[INFERENCE]` The pattern is now familiar across this batch: the Argus piece had 7 of 12 links self-referential, the Flipboard piece 7 of 12, the changelog guide **zero** links while naming six external tools. `[EVIDENCE]` **The WPpodcast episode remains the outlier**, with 21 links essentially all to primary sources. **Internal linking is an engagement strategy; external linking is a service to the reader**, and most of this batch chooses the first. ## 🗣️ Paraphrase The standard advice for stopping hallucination — chunk your PDFs, embed them, put them in a vector database, search by cosine similarity — works until you deploy it. It breaks because it assumes similar text is relevant text. Take two chunks: "Acme Corp acquired BetaTech in 2022" and "Sarah Connor was appointed CEO of BetaTech in 2023." Now ask who leads the company Acme acquired. Vector search finds things resembling "Acme Corp" and "leadership" and misses the second chunk entirely, because "Sarah Connor" and "CEO of BetaTech" have no semantic resemblance to "Acme Corp." The answer is in the corpus, spread across chunks that share no vocabulary with the question. You don't need better similarity — you need to know that Acme *acquired* BetaTech and Sarah Connor *leads* it. GraphRAG builds that. At ingestion, an LLM reads each chunk and extracts entities and the relationships between them into a knowledge graph. At query time, you find an entry point by vector search, then walk the relationships out from it to gather a relevant subgraph, and hand that to the model. Neo4j's `neo4j-graphrag` makes this practical: install the packages, connect with proper credential handling and a rate-limit handler (without one, you'll fail on the first real document), define a schema before ingesting — skip that and the model will invent labels and create separate nodes for "Acme Corp," "Acme Corporation" and "Ame" — then retrieve with a vector-plus-traversal retriever. Three things to know before adopting. Ingestion is expensive, because every chunk needs an LLM call. Schema design isn't optional. But debugging gets much better: instead of squinting at float arrays, you look at the nodes and see whether the model invented something. ## 📰 Newsletter **The clearest two-sentence demonstration of why RAG breaks that I've seen.** Chunk one: *"Acme Corp acquired BetaTech in 2022."* Chunk two: *"Sarah Connor was appointed CEO of BetaTech in 2023."* Ask *"who leads the company that Acme Corp acquired?"* and vector search misses chunk two entirely — *"'Sarah Connor' and 'CEO of BetaTech' have nothing to do with 'Acme Corp' semantically."* **The answer is in your corpus, split across chunks that share no vocabulary with the question.** No better embedding fixes that, because the missing thing is a relationship, not a similarity. **The fix is to have an LLM extract entities and relationships at ingestion into a graph**, then retrieve by finding an entry node and walking outward. The article's diagnosis line is the keeper: *"The standard RAG assumes that semantic similarity implies relevance, which is not necessarily the case."* **Three things it gets right and one it doesn't.** The best argument is buried third in a bullet list: *"To debug your naive RAG, you need to examine big floating-point arrays. To debug GraphRAG, you only need to look at your Neo4j dashboard."* **The durable win isn't retrieval quality — it's that retrieval becomes inspectable.** (Neat mirror of the Syntax episode this week, where CSS absorbing JavaScript's layout math trades inspectability *away* for elegance. Same variable, opposite direction — **debuggability is a design choice, not a property you inherit.**) The schema warning is the most practically important line and it's a warning rather than a solution: skip it and you get *"'Acme Corp', 'Acme Corporation,' and 'Ame'"* as three nodes. **Entity resolution is where real deployments fail, and it fails silently** — a duplicated entity doesn't error, it just makes traversal miss facts attached to its twin. That looks exactly like the problem you built the graph to solve. **A schema is necessary and not sufficient; you also need a dedup pass, and the article doesn't mention one.** **And the title promises multi-hop while the implementation does one hop.** Step 4 retrieves *"1-hop connected neighbors."* The example works because the entry point lands on BetaTech — the middle of the chain — so both facts are one step away. Push the question one entity further out and one hop fails exactly as vector search did. **Traversal depth is a parameter you have to choose, and it's never named as one.** Also: it's **Cypher**, not "Cipher." The article gets it wrong three times in prose and right once inside a class name — and searching "Cipher query Neo4j" gets you cryptography. ## ✅ Action Items #### DevRel - **Developer advocacy:** `[RECOMMEND]` The Acme/BetaTech example is the best two-sentence explanation of RAG's limits in circulation. Worth reusing, with the entry-point subtlety made explicit rather than glossed. - **Education:** `[RECOMMEND]` Teach the **diagnostic** the article omits: characterise your failed queries first. **Retrieval failure and reasoning failure look identical from the answer and have completely different fixes.** - **Developer experience:** `[INFERENCE]` The rate-limit detail — *"your pipeline will fail with a RateLimitError on the first real document"* — is the kind of operational gotcha that belongs in a quickstart and rarely is. - **Developer marketing / Community management / Cross-project:** Not applicable to this source. #### OSPO - **Policy & Governance / License Compliance / Security Management / Software Inventory / Developer Support / Community Engagement / Strategic Alignment:** Not applicable to this source — a technical retrieval-architecture tutorial with no governance surface. #### Community - **Maintainers / Contributors / Sponsors / Ecosystem:** Not applicable to this source. #### WordPress - **All teams:** Not applicable to this source. `[INFERENCE]` Enumerating teams here would manufacture relevance the article does not have. #### The WPOCC - **Programs / Projects / Funding experiments:** Not applicable to this source. ### FAIR - Not applicable to this source in any respect. ### Personal knowledge practice `[RECOMMEND]` This is where the article earns its place, and the connection is specific: - **A wiki-linked vault already has the structure being proposed.** `[INFERENCE]` The article's claim is that **explicit typed relationships beat inferred semantic similarity** for questions spanning documents. A hand-linked note corpus has explicit relationships already — curated, and typed by convention rather than by an LLM at ingestion. **The expensive part of GraphRAG, the GPT-4o call per chunk, is the part such a corpus does not need.** - **What a graph adds that links alone do not is *typed edges*.** `[INFERENCE]` `[[Acme Corp]]` records **that** two notes relate; `-[ACQUIRED]->` records **how**. The second is what makes traversal answerable rather than merely followable. - `[RECOMMEND]` **The cheap version of this idea is not building a knowledge graph — it is making link types explicit where they carry meaning.** A note-writing habit, not an infrastructure project. `[INFERENCE]` The deep-read pipeline already does a version of this: atom tags (`#atom/zettel/`, `#action/`, `#fallacy/`) are typed annotations, and cross-document connections drawn in highlight notes are hand-authored edges. - **The entity-resolution warning applies to vaults too**, and more strongly. `[INFERENCE]` A vault with `[[Acme Corp]]` and `[[Acme Corporation]]` as separate notes has exactly the duplication the article warns about — **and unlike a graph database, nothing will ever flag it.** The human-readable-wikilink discipline already in use is partly a defence against this; **periodic alias consolidation is the dedup pass.** ## 🧪 Hypotheses 1. **H1 — Entry-point selection determines multi-hop success at fixed depth.** With 1-hop retrieval, success on multi-entity questions depends primarily on whether vector search lands on a node topologically between the query's entities. 2. **H2 — Entity duplication is the dominant GraphRAG failure mode.** In deployed GraphRAG systems, more retrieval failures trace to duplicated or unresolved entities than to insufficient traversal depth. 3. **H3 — Schemas constrain labels but not identity.** Enforcing a schema materially reduces label hallucination and does not materially reduce duplicate-entity creation. 4. **H4 — Inspectability changes debugging outcomes.** Engineers locate the cause of a wrong retrieval faster with a graph-based pipeline than with a vector-only one, holding system quality constant. 5. **H5 — GraphRAG's advantage is confined to relationship questions.** On single-hop factual queries, GraphRAG shows no retrieval-quality advantage over vector search sufficient to justify its ingestion cost. 6. **H6 — Curated link corpora need no extraction step.** For hand-linked note corpora, graph traversal over existing links performs comparably to LLM-extracted graphs at a fraction of the ingestion cost. ## 📊 Methodology - **H1 — the one that tests the gap I identified, and it is cheap.** Construct a synthetic corpus with known graph topology; generate questions at varying entity distance; vary which node vector search returns as the entry point; measure success at fixed 1-hop depth. **Fully controlled, no confounds worth naming, and it would tell an implementer exactly when to increase depth.** - **H2 — the most practically valuable and the hardest to instrument.** Requires access to deployed systems and a failure taxonomy fixed in advance. **Confound:** duplication and depth failures can co-occur, and duplication is easier to spot once you look — so an unblinded coder will over-attribute to it. **Pre-register the taxonomy or the result follows the analyst.** - **H3 — directly runnable.** Ingest the same corpus with and without a schema; count label variants and duplicate entities separately. `[INFERENCE]` **Separating the two counts is the whole point** — the article treats the schema as addressing both, and the distinction is what determines whether you also need a dedup pass. - **H4:** Fault-injection study. Introduce a known retrieval fault into matched pipelines; measure time to diagnosis. **Confound:** familiarity with graph databases dominates; counterbalance and report it as a covariate. - **H5:** Benchmark both on a question set stratified by hop count, with cost per query recorded alongside quality. `[RECOMMEND]` **The cost axis is the point** — the article asserts the trade-off and quantifies neither side. - **H6 — the one relevant to a personal vault.** Compare traversal over hand-authored links against an LLM-extracted graph on the same corpus, same questions. **Confound:** hand-authored links are typed only by convention, so the comparison partly measures link-writing discipline rather than architecture. **Interesting either way**, since discipline is the variable a person controls. ## ❓ Questions - **What traversal depth do real deployments use, and how is it chosen?** `[INFERENCE]` The article's implicit answer is 1, and never treats it as a decision. - What does ingestion actually cost per thousand chunks? **The article's central trade-off, unquantified.** - **How do production systems handle entity resolution after extraction?** The article names the problem and skips the solution. - Does GraphRAG address the global-summarisation failure it raises in the second paragraph? `[INFERENCE]` Traversal solves relationships; corpus-wide aggregation is a different problem and gets no answer. - `[EVIDENCE]` Since chunking persists via `FixedSizeSplitter`, how much do chunk boundaries degrade entity extraction — and does a relationship split across two chunks survive? - `[INFERENCE]` What is the maintenance cost when documents change? A vector index re-embeds; a graph needs its affected subgraph reconciled, and nothing here addresses updates. ## 🔮 Future Research - **Entity resolution in LLM-extracted knowledge graphs.** `[INFERENCE]` The article identifies it as the failure mode and offers a schema, which addresses a different problem. **This is the gap between the tutorial and a working system.** - **Cost-quality curves for GraphRAG at varying traversal depth and corpus size.** `[INFERENCE]` The decision every adopter faces and nobody appears to have published. - **Whether curated link corpora obviate the extraction step.** Directly relevant to anyone with a substantial wiki-linked vault. - **Graph maintenance under document churn.** `[INFERENCE]` Re-embedding is cheap and idempotent; re-extraction is neither, and the incremental-update story is unwritten. ## 💡 Implications - **Practice.** `[RECOMMEND]` **Diagnose before adopting.** If failed queries fail because the right text was never retrieved, a graph helps. If the right text was retrieved and the model reasoned badly, it changes nothing. - **Policy.** Not applicable to this source. - **Security.** `[INFERENCE]` One item worth noting and not raised: **an LLM extracting entities from documents into a persistent graph creates a new data path**, and the graph will contain whatever the documents contained. Access control over the graph is not the same as access control over the source documents, and the article does not mention it. - **Education.** `[EVIDENCE]` The Acme/BetaTech example is excellent pedagogy — minimal, concrete, and the failure is undeniable in two sentences. - **Funding models.** Not applicable to this source. ## 🧠 Learning Styles Assessment **Reading level estimate.** `[INFERENCE]` **Flesch–Kincaid grade 12–13** — moderate sentence length with dense technical noun phrases. Hand estimate on 1,007 words, ±1.5 grades. `[EVIDENCE]` Jargon is heavy and mostly unglossed: *cosine similarity*, *multi-hop*, *nodes and edges*, *schema enforcement*, *1-hop neighbors*, *FixedSizeSplitter*. `[INFERENCE]` The intended reader already builds RAG systems, and the article is correctly pitched for that reader. #### 👀 Visual learners - **Present:** `[EVIDENCE]` One inline relationship notation — `("Acme Corp") - [ACQUIRED] -> (BetaTech)` — which does real explanatory work in a single line. - **Improve:** `[EVIDENCE]` **An article about graphs contains no graph.** A three-node diagram would show the traversal path, make the entry-point subtlety visible, and expose the 1-hop limitation immediately. `[INFERENCE]` **The missing diagram is the same thing as the missing analysis** — drawing it is what reveals that the example is solvable in one hop only because the entry node sits in the middle. #### 🔊 Aural learners - **Present:** Nothing. - **Improve:** `[INFERENCE]` Low value — this is reference material with code, consumed by scanning. Saying so is more useful than reflexively recommending audio. #### 🗣️ Verbal learners - **Present:** `[EVIDENCE]` **The strongest dimension.** *"The standard RAG assumes that semantic similarity implies relevance"* is a precise one-line diagnosis, and *"an unstructured graph is nothing more than an unorganized database"* is a memorable compression of the schema argument. - **Improve:** `[EVIDENCE]` The **Cipher/Cypher** inconsistency is a verbal failure specifically — the term a reader most needs to search for is misspelled three times in prose and correct only inside a class name. #### 🧩 Kinesthetic learners - **Present:** `[EVIDENCE]` Four numbered steps, a real `pip install` line, and named classes. - **Improve:** `[EVIDENCE]` **The code blocks are largely absent from the extraction** — only the install line survives — so a hands-on reader working from this capture cannot follow the steps. `[INFERENCE]` Whether that is a capture artifact or thin original code is not determinable from here, and it matters, because steps 2–4 are described rather than shown either way. #### 🧠 Logical learners - **Present:** `[EVIDENCE]` Clean problem → mechanism → implementation → trade-offs structure, and the trade-offs section is genuinely balanced rather than promotional. - **Improve:** `[EVIDENCE]` **The title/implementation gap** — multi-hop promised, 1-hop implemented — is exactly what a logical reader will catch by tracing the example. `[INFERENCE]` And the global-summarisation failure is raised in paragraph two and never returned to. #### 🧑‍🤝‍🧑 Social learners - **Present:** Nothing. - **Improve:** `[INFERENCE]` No practitioner voice, no deployment report, no community reference. **For an article whose framing is "this is what happens after you deploy,"** the absence of anyone describing an actual deployment is a real gap. #### 🧘‍♀️ Solitary learners - **Present:** Self-contained in structure. - **Improve:** `[EVIDENCE]` **All four links are internal.** Neo4j, `neo4j-graphrag`, and the underlying literature are unlinked. `[INFERENCE]` The solitary reader told to install a package and stand up a database is given no route to either's documentation — the article withholds precisely the accessibility it credits the framework with providing. #### 🌿 Naturalistic learners - **Present:** Nothing. - **Improve:** `[INFERENCE]` No honest fit. The nearest available metaphor — a knowledge graph as an ecosystem of relationships — is decoration rather than explanation, and saying so beats reaching for it. ## 📊 Accessibility Assessment Assessed against **WCAG 2.2 AA**, from the captured markup only. `[INFERENCE]` **Scope limit:** content structure and text as retrieved. Focus, keyboard operation, live contrast, and reflow require the rendered page and are **not assessed**; absence of a finding is not a pass. #### Screen reader semantics - **Present:** `[EVIDENCE]` **Proper heading hierarchy** — three `<h2>` sections with four nested `<h3>` steps, correctly ordered with no skipped levels. `[INFERENCE]` **Passes 2.4.6**, which three news articles in this batch failed outright. - **Improve:** `[EVIDENCE]` **Pull-quotes duplicate body text verbatim**, three times. `[INFERENCE]` Unless marked `aria-hidden`, a screen-reader user hears each of those sentences twice — a well-known and easily-fixed pattern failure. Not confirmable from the extraction; flagged as the thing to check. - **Improve:** `[EVIDENCE]` A **promotional link is embedded mid-paragraph**, splitting the Acme/BetaTech example in the extraction. `[INFERENCE]` Inline promotional inserts interrupt reading order for anyone consuming linearly, which is every screen-reader user. #### Keyboard navigation - **Not testable from the captured markup.** #### Contrast and readability - **Not testable** — no CSS captured. #### Text resizing and reflow - **Not testable from the captured markup.** `[EVIDENCE]` The article contains code blocks; **wide preformatted content is the standard 1.4.10 hazard** and must scroll within its own container. Flagged as the thing to check. #### Media and captions - **Present:** One credited image (`[EVIDENCE]` *"Rick Rothenberg for Unsplash+"*). - **Improve:** `[EVIDENCE]` **No alt text captured for it** — the credit line is the only accompanying text, and a credit is not a description. **WCAG 1.1.1.** `[INFERENCE]` If the image is a decorative stock photo, `alt=""` is correct and the credit should not be its accessible name; either way the current state is ambiguous. #### Forms and interactions - **Not applicable.** #### Standards reference - **1.3.1 Info and Relationships (A)** — appears satisfied; heading structure is correct. - **2.4.6 Headings and Labels (AA)** — **satisfied.** - **1.1.1 Non-text Content (A)** — **indeterminate**; image alt not captured, only a credit line. - **1.4.10 Reflow (AA)** — **not assessable**; code blocks are the risk. ## 🌍 DEIB Assessment #### Cultural breadth - **Present:** Technical subject with no regional framing. - **Improve:** `[INFERENCE]` One substantive and non-obvious point: **entity resolution is a culturally-loaded problem the article treats as clerical.** Its example — "Acme Corp" / "Acme Corporation" / "Ame" — is the easy case. **Names transliterated from non-Latin scripts, names with varying ordering conventions, and organisations whose legal names differ by jurisdiction all produce the same duplication with far higher failure rates.** A schema does nothing for any of it, and a system built and tested on Western corporate names will underperform silently elsewhere. #### Inclusive language - **Present:** Neutral throughout. - **Improve:** `[INFERENCE]` Jargon density is the barrier rather than word choice, and it is an audience decision. #### Neurodiversity support - **Present:** `[EVIDENCE]` **Strong** — numbered steps, descriptive headings, consistent structure, and a summarising trade-offs list at the end. - **Improve:** `[EVIDENCE]` The verbatim pull-quotes disrupt reading flow, and the mid-paragraph promotional insert breaks the article's single most important example in half. #### Representation - **Present:** Nothing to assess. - **Improve:** `[EVIDENCE]` **No byline**, and no practitioner is quoted or referenced. `[INFERENCE]` For an article whose authority rests on deployment experience — *"companies quickly discover"*, *"until you actually deploy it"* — **nobody with that experience is identified**, including the author. #### Psychological safety - **Present:** `[EVIDENCE]` `[STATED]` *"Standard vector search is a very powerful solution, but treating it as a one-size-fits-all fix… is an easy way out."* `[INFERENCE]` The criticism lands on a **practice** rather than on practitioners, and the article credits the thing it is arguing against rather than dismissing it. - **Improve:** `[INFERENCE]` *"The echo chamber's view"* is the one dismissive phrase, aimed at an unnamed collective. Mild, and it does set a slightly superior tone in the opening paragraph that the rest of the article does not sustain. ## ✅ Logical Validity #### Strong reasoning - **The Acme/BetaTech demonstration is the article's strongest work and is close to airtight.** Two chunks, one query, and an explicit statement of *why* retrieval fails: `[STATED]` *"'Sarah Connor' and 'CEO of BetaTech' have nothing to do with 'Acme Corp' semantically."* `[INFERENCE]` The reader can verify the failure by inspection, which is the best property an example can have. - **The core diagnosis names an assumption rather than a symptom.** `[STATED]` *"The standard RAG assumes that semantic similarity implies relevance."* `[INFERENCE]` Locating the defect in a **premise** rather than in performance is what makes the argument generalise beyond the example. - **The trade-offs section argues against the article's own recommendation** — high ingestion cost, mandatory schema work — which is what distinguishes a technical article from a pitch. - **The engineering note corrects a misimpression the article could have profited from.** `[STATED]` *"Under the hood, this pipeline still splits your text using a FixedSizeSplitter."* `[INFERENCE]` Volunteering that chunking has not gone away removes an inflated reading the framing invites. #### Fallacies - **Overclaim in the title relative to the implementation.** Multi-hop is promised; **1-hop retrieval is implemented**, and the example succeeds only because the entry point lands mid-chain. **Weakness:** a reader adopting this for genuinely multi-entity questions will hit the same failure the article diagnoses. **Fix:** name traversal depth as a parameter with a cost curve. - **Problem named, solution omitted — entity resolution.** The schema is offered as the remedy for label hallucination *and* node duplication; **it addresses the first far better than the second. Fix:** state that a deduplication pass is required. - **Unquantified central trade-off.** *"Ingestion cost is high"* in an article whose purpose is helping the reader decide whether to pay it. **Fix:** one worked example — N chunks, model X, approximate cost. - **Raised and dropped — global summarisation.** Introduced as a second RAG failure in paragraph two, never addressed. `[INFERENCE]` Traversal does not obviously solve corpus-wide aggregation, and the reader is left assuming it does. - **Dismissive framing of the opposition.** *"The echo chamber's view"* characterises the received approach without engaging anyone who holds it. Mild, and the article's substantive treatment of vector search is fair. #### Tone of voice `[EVIDENCE]` Confident practitioner register — direct address, imperatives (*"stop doing it randomly,"* *"filter your enterprise data carefully"*), and deployment-flavoured authority (*"until you actually deploy it,"* *"companies quickly discover"*). `[INFERENCE]` The tone claims operational experience the article never evidences, and the **rate-limit warning is the one detail that genuinely reads as earned** — it is too specific to be secondhand. `[EVIDENCE]` The closing is measured rather than triumphal: `[STATED]` GraphRAG *"demands more intentionality in its design, strict guidelines to follow, and greater upfront computational effort."* `[INFERENCE]` Ending on the costs rather than the benefits is the right note and is not the usual choice. ## 🛡️ DARVO Analysis **No DARVO pattern present, and the framework is inapplicable.** `[EVIDENCE]` DARVO — Deny, Attack, Reverse Victim and Offender — describes a response to accusation. This is a technical tutorial with no accusation, no accused party, and no adversarial dimension. Checking each element explicitly rather than omitting the section: - **Deny:** None. `[EVIDENCE]` The article **affirms the limits of its own recommendation** — high ingestion cost, mandatory schema work, chunking not eliminated — rather than minimising any of them. - **Attack:** `[EVIDENCE]` One mild instance: *"the echo chamber's view"* characterises proponents of standard RAG collectively and without engagement. `[INFERENCE]` **This is dismissive framing, not DARVO** — there is no accuser and no accusation being deflected — and the article's substantive treatment of vector search is fair, calling it *"a very powerful solution."* - **Reverse victim/offender:** None. `[INFERENCE]` One adjacent observation, recorded separately because it is **not** DARVO: the article's authority rests on claimed deployment experience (*"companies quickly discover"*) that is nowhere evidenced, and it carries **no byline**. **That is an unevidenced appeal to experience**, already logged under Representation and Tone — not a rhetorical manoeuvre against anyone. ## 🎯 Conclusion **Bottom line.** `[INFERENCE]` A correct, clearly-argued explanation of a real failure mode, carrying **one substantive gap and one that will bite in production.** 1. **The diagnosis is right and the demonstration is excellent.** `[STATED]` *"The standard RAG assumes that semantic similarity implies relevance, which is not necessarily the case."* Two chunks and one question prove it in a way no amount of argument would: the answer is in the corpus, distributed across chunks that share no vocabulary with the query, and **the missing thing is a relationship, not a similarity.** 2. **The title promises multi-hop; the implementation does one hop.** `[EVIDENCE]` Step 4 retrieves *"1-hop connected neighbors."* `[INFERENCE]` The worked example succeeds only because vector search enters at **BetaTech — the middle of the chain** — putting both facts one step away. Push the question one entity further out and one hop fails exactly as vector search did. **Traversal depth is a parameter with a cost curve, and it is never named as one.** 3. **Entity resolution is where this actually breaks, and it gets one sentence.** `[STATED]` Skip the schema and you get *"'Acme Corp', 'Acme Corporation,' and 'Ame'"* as separate nodes. `[INFERENCE]` But **a schema constrains labels, not identity** — and a duplicated entity doesn't error, it makes traversal miss facts attached to its twin, **which looks exactly like the problem the graph was built to solve.** A dedup pass is required and no mention of one appears. 4. **The best argument is buried third in a bullet list.** `[STATED]` *"To debug your naive RAG, you need to examine big floating-point arrays. To debug GraphRAG, you only need to look at your Neo4j dashboard."* `[INFERENCE]` **The durable win is not retrieval quality — it is that retrieval becomes inspectable.** `[INFERENCE]` And it mirrors the Syntax episode from this batch exactly: CSS absorbing JavaScript's layout math trades inspectability *away* for elegance. **Same variable, opposite direction — debuggability is a design choice, not a property you inherit**, and it is usually traded away for concision without anyone recording the trade. **For a wiki-linked vault specifically.** `[INFERENCE]` The argument is that **explicit typed relationships beat inferred semantic similarity** for questions spanning documents — and a hand-linked note corpus already has explicit relationships, curated, without needing the GPT-4o call per chunk that makes GraphRAG expensive. **What a graph adds is *typed* edges:** `[[Acme Corp]]` records *that* two notes relate; `-[ACQUIRED]->` records *how*, which is what makes traversal answerable rather than merely followable. `[RECOMMEND]` **The cheap version is not building a knowledge graph — it is making link types explicit where they carry meaning.** And the duplication warning applies with more force to a vault than to a database: `[[Acme Corp]]` and `[[Acme Corporation]]` as separate notes is the same defect, and **nothing will ever flag it.** **Practical next steps.** 1. **Diagnose before adopting.** Retrieval failure and reasoning failure look identical from the answer and have completely different fixes. 2. Treat traversal depth as an explicit decision; assume 1 hop unless you have checked. 3. Plan a deduplication pass. **A schema is necessary and not sufficient.** 4. `[EVIDENCE]` It is **Cypher**, not "Cipher" — wrong three times in prose here, and searching the misspelling gets you cryptography. <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/Why Basic RAG Fails at Multi-Hop Reasoning.md|Why Basic RAG Fails at Multi-Hop Reasoning by The New Stack]] ## Highlights - An unstructured graph is nothing more than an unorganized database. The moment you do not provide a proper schema, your LLM will hallucinate entity labels, duplicating nodes of “Acme Corp”, “Acme Corporation,” and “Ame.” We will build our architecture beforehand and feed it into the pipeline. ([View Highlight](https://read.readwise.io/read/01m1239r9187yt1btff8x4mme1)) - Note: THE MOST PRACTICALLY IMPORTANT LINE IN THE ARTICLE — AND IT'S A WARNING, NOT A SOLUTION. A schema constrains LABELS (this node is a Company, not an Organisation). It does much less for IDENTITY: whether "Acme Corp" in document 40 is the same node as "Acme Corporation" in document 12. ENTITY RESOLUTION IS WHERE REAL GRAPHRAG DEPLOYMENTS FAIL, AND IT FAILS SILENTLY. A duplicated entity doesn't error — it produces a graph where traversal from one node MISSES FACTS ATTACHED TO ITS TWIN, WHICH LOOKS EXACTLY LIKE THE PROBLEM THE GRAPH WAS BUILT TO SOLVE. The article's own example is the illustration: if the acquisition attaches to "Acme Corp" and a later filing to "Acme Corporation," the multi-hop query breaks and the failure is invisible in the result. A SCHEMA IS NECESSARY AND NOT SUFFICIENT — you also need a dedup/canonicalisation pass after ingestion, and the article doesn't mention one exists. AND THE HARD VERSION IS WORSE THAN THE EXAMPLE: "Acme Corp"/"Acme Corporation"/"Ame" is the EASY case. Names transliterated from non-Latin scripts, varying name-ordering conventions, organisations whose legal names differ by jurisdiction — all produce the same duplication at far higher rates, and a schema does nothing for any of it. A system built and tested on Western corporate names UNDERPERFORMS SILENTLY ELSEWHERE. THIS APPLIES TO MY VAULT WITH MORE FORCE, NOT LESS: [[Acme Corp]] and [[Acme Corporation]] as separate notes is the identical defect, and unlike a graph database NOTHING WILL EVER FLAG IT. The human-readable-wikilink discipline is partly a defence against this; PERIODIC ALIAS CONSOLIDATION IS THE DEDUP PASS. #atom/zettel/z2 #action/h1 - This is where multi-hop reasoning happens. An ordinary vector retriever will provide us with nodes. To travel across the graph, we use a VectorCypherRetriever. We will embed the query, identify the semantic entry point, and finally perform a Cipher query on all the 1-hop connected neighbors. ([View Highlight](https://read.readwise.io/read/01m1239aa8d5pajgsbj09nq9e7)) - Note: THE TITLE PROMISES MULTI-HOP; THIS IMPLEMENTATION DOES ONE HOP. Trace the article's own example against it. The graph is (Acme Corp)-[ACQUIRED]->(BetaTech)<-[LEADS]-(Sarah Connor). The query "Who leads the company that Acme Corp acquired?" is TWO HOPS FROM ACME. But retrieval enters at the SEMANTIC ENTRY POINT, and the reported context is "BetaTech, its acquisition by Acme Corp, and the fact that Sarah Connor leads it" — SO THE ENTRY POINT IS BETATECH, THE MIDDLE OF THE CHAIN, and from there both facts are one step away. THE EXAMPLE IS SOLVABLE IN ONE HOP ONLY BECAUSE THE ENTRY NODE SITS IN THE MIDDLE. Push it one entity further out — "who leads the companies acquired by the firm that acquired BetaTech?" — and 1-hop fails exactly as vector search did. TRAVERSAL DEPTH IS A PARAMETER YOU HAVE TO CHOOSE AND THE ARTICLE NEVER SAYS SO. Each extra hop expands the subgraph combinatorially, which is WHY 1-hop is a sensible default — the gap is not naming it as a limit, so the reader doesn't know what to change when a genuinely three-entity question fails. TESTABLE CHEAPLY: synthetic corpus with known topology, questions at varying entity distance, vary which node vector search returns as entry point, measure success at fixed depth. Fully controlled, no real confounds, and it tells an implementer exactly when to increase depth. ALSO NOTE: it's CYPHER, not "Cipher." The article gets it wrong three times in prose and right once inside the class name VectorCypherRetriever — and searching "Cipher query Neo4j" returns cryptography. #atom/zettel/z1 #fallacy/overclaim