Beyond Either: Knowledge Graphs and Hybrid RAG as a Single System
Solving problems that neither architecture can solve alone
A worked industrial case in which the answer exists in no document, in no database, and in no single retrieval — and an architecture that recovers it.
1. Executive summary
Two architectures dominate enterprise efforts to make institutional knowledge machine-usable. The knowledge graph offers governed structure: entities with durable identity, explicit relationships, formal semantics and a defensible record of what was true when. Retrieval-augmented generation, in its mature hybrid form, offers access to the far larger body of knowledge that was never structured at all — the reports, drawings, service records, change justifications and shift narratives in which most industrial understanding actually resides.
Each is routinely presented as an alternative to the other. In practice each has a hard boundary, and the boundaries are complementary rather than overlapping. A knowledge graph can only answer questions about relationships someone has already modelled; it cannot surface a causal mechanism that no one thought to represent. Hybrid retrieval can surface any statement that appears in the corpus; it cannot determine that two documents which share no vocabulary refer to physically connected equipment, cannot tell which version of a procedure was in force during an event three years ago, and cannot establish that four separately-stated conditions were simultaneously true.
Between these boundaries lies a class of problem that neither architecture can address at any level of investment. This paper characterises that class, works through a detailed industrial example of it, and sets out an architecture in which the two systems are bound into one — not chained together, but structurally coupled, with each constraining the other.
The central argument is that the value of the combination does not come from having two retrieval sources. It comes from three specific mechanisms that only exist when the graph and the corpus are bound: the graph restricts where retrieval looks, allowing evidence to be found that no similarity measure would rank; the graph supplies the identity and temporal scaffolding that lets fragments from unrelated documents be recognised as statements about the same thing at the same time; and the graph acts as a falsifier, rejecting generated hypotheses that are physically or temporally impossible. The third mechanism is the one most implementations omit and the one that determines whether the system can be trusted.
|
Relationship
to the companion paper This paper assumes the knowledge graph disciplines
described in the companion volume — opaque identifiers, an equivalence-based
identity layer, provenance on every assertion, and a bitemporal model
separating valid time from transaction time. Those are not optional
refinements here; the architecture in Section 5 is unbuildable without them,
and Section 9 explains why. |
2. Where each architecture stops
2.1 The boundary of a knowledge graph
A knowledge graph answers questions by traversing relationships that have been explicitly asserted. This gives it capabilities no text system has: it knows that two identifiers denote the same asset, it knows what is physically connected to what, it knows what was true on a given date, and it can distinguish a fact that changed from a fact that was recorded in error.
Its boundary follows directly from the same property. The graph contains only what has been modelled. Every ontology is a deliberate abstraction produced against a finite set of questions, and the questions that matter most are frequently the ones nobody anticipated. A graph built to support maintenance planning will contain the relationship between a pump and its functional location; it will not contain the relationship between a corrosion inhibitor dosing change and a downstream heat exchanger fouling rate, because no one modelled that relationship, because no one yet knew it existed.
The second boundary is coverage. In a typical operating facility the overwhelming majority of accumulated knowledge is unstructured: root cause analyses, vendor field service reports, inspection narratives, management-of-change justifications, operator log entries, commissioning notes, correspondence. This material is not merely uningested — most of it is not reducible to structured assertions without discarding the reasoning it contains. A change justification is valuable precisely because of its argument, not merely its outcome.
• A knowledge graph cannot: propose a mechanism it was not taught, explain why a decision was taken, capture nuance or hedged judgement, or answer a question that requires reading an argument rather than following a link.
2.2 The boundary of hybrid retrieval
Hybrid retrieval — lexical search fused with dense vector search, followed by reranking — is a genuine advance over either technique alone. Lexical matching handles the identifiers, tag numbers and technical terminology that embeddings handle poorly; dense retrieval handles paraphrase and conceptual similarity that keyword matching misses. Fused and reranked, the combination is strong across a wide range of enterprise question types.
Its boundary is that every retrieval mechanism it contains is a similarity measure over text. This produces four specific limitations, and they are the limitations that matter for hard problems:
1. Similarity is not connection. A document describing a dilution steam superheater and a document describing a radiant coil have almost no lexical or semantic overlap. That they concern physically coupled equipment is a fact about the plant, not a fact about the text. No embedding model recovers it, because the information is not present in the language.
2. Identity is not resolvable from text. The same asset appears across fifteen years of documents as a tag number, a legacy tag number, a vendor item number, a functional location and three informal names. Retrieval treats these as unrelated tokens. Merging them requires an authority the corpus does not contain.
3. Time is not a retrievable property. Retrieval can find the procedure document; it cannot determine which revision was in force during an incident, nor that a set point stated in a 2019 document had been superseded by the time of a 2021 event. Document dates are a weak proxy and are frequently absent or wrong.
4. Conjunction is not retrieval. If an explanation requires four conditions to have held simultaneously, and each is stated in a different document, no top-k retrieval reliably returns all four. Each is individually a poor match for the query; the relevance of each depends entirely on the presence of the others, which is a circularity that ranking cannot resolve.
2.3 The irreducible class
Combining the two boundaries yields a precise characterisation. A problem is irreducible to either architecture alone when its solution requires all of the following:
• A causal or explanatory mechanism that exists only in narrative form and was never modelled as a relationship
• Evidence distributed across multiple documents that share no meaningful vocabulary
• A connection between those documents that is physical, organisational or temporal rather than linguistic
• Identity resolution across naming regimes that the documents themselves do not reconcile
• A determination that several conditions held at the same time, where the relevant time is historical rather than current
Problems of this shape are not exotic. They are the standard form of the persistent, expensive, repeatedly-investigated problem in any facility more than a decade old — precisely the problems that justify the investment in the first place. Sections 3 and 4 work through one in detail.
3. A worked problem
The following case is hypothetical but constructed from the standard components of real recurring-failure investigations. It is described at enough resolution to test any proposed architecture against it.
3.1 The situation as presented
A world-scale olefins complex, in operation since 2004, has experienced accelerated coking in the radiant coils of one cracking heater, designated F-1104, at irregular intervals since 2016. Events occur roughly every fourteen to twenty months. Each requires an unplanned decoke, and each costs the operator several million dollars in lost throughput. Seven sister heaters on the same feedstock and the same firing control philosophy are unaffected.
Three formal root cause investigations have been conducted. The first attributed the problem to burner maldistribution and led to a burner replacement, after which the problem recurred. The second attributed it to a metallurgy variance in a coil section replaced during the 2015 turnaround, and led to a further replacement, after which the problem recurred. The third concluded that the cause was indeterminate and recommended increased monitoring.
The operator has a mature knowledge graph covering equipment, connectivity, functional locations, maintenance history and process tag identity, built to the standards described in the companion paper. It has also deployed a competent hybrid retrieval system over a corpus of approximately four hundred thousand documents spanning the facility's operating life. Neither system has produced an explanation. Neither is defective.
3.2 The actual mechanism
The true explanation, which no single source states, is a four-condition conjunction:
1. A minority of naphtha cargoes originating from one specific supply terminal carry an elevated trace oxygenate content. This is recorded in cargo assay data, which is structured, retained, and correctly associated with the receiving tank.
2. Feedstock from those cargoes reaches F-1104 specifically only under a particular header lineup, adopted during reduced-rate operation following a charge gas compressor trip. The lineup is a documented operating practice; which heater it favours is a consequence of header hydraulics stated in no document at all, but derivable from the connectivity model in the graph.
3. A 2018 replacement of the dilution steam superheater tube bundle, performed by a vendor under a service contract, slightly altered the steam quality delivered to the 1100-series heaters. The vendor field service report notes the change as an incidental observation. The report references the superheater and the unit number; it never mentions F-1104 or any individual heater.
4. A 2019 management-of-change reduced the set point of the sulphur additive dosing skid — the additive passivates coil metallurgy against catalytic coking — in order to reduce sulphur dioxide emissions. The justification document discusses emissions compliance and the dosing skid, designated U-31. It contains no discussion of coking and no reference to any heater.
None of the four conditions is individually sufficient, and none is individually anomalous. Coking events occur only when all four coincide: an affected cargo in service, during a post-trip lineup that routes it preferentially to F-1104, under the post-2018 steam quality, with post-2019 sulphur dosing. The apparent fourteen-to-twenty-month period is simply the joint frequency of two independent stochastic events — the arrival of an affected cargo and a compressor trip — within the same operating window.
The three failed investigations each examined the heater. The explanation is not located in the heater.
3.3 Why the knowledge graph cannot solve it
The graph contains every structural fact required and none of the causal ones.
• The mechanism was never modelled. There is no relationship type connecting additive dosing set points to coil coking propensity, because that dependency was not part of any competency question the ontology was built against. The graph cannot traverse an edge that does not exist, and cannot invent one.
• The rationale is not in the graph. The management-of-change record may exist as a node with a date, a status and an approver. Its reasoning — that sulphur reduction was traded against an emissions target, with passivation considered a secondary effect — is prose in an attached document.
• The vendor observation was never structured. A field service report noting an incidental change in steam quality is not the kind of statement that any ingestion pipeline converts into an assertion, because it is a hedged observation rather than a specification.
A graph query for "what is connected to F-1104" returns a correct and complete answer that contains no explanation. This is not a modelling failure; it is the boundary described in Section 2.1 operating exactly as expected.
3.4 Why hybrid retrieval cannot solve it
The retrieval system contains every causal fragment required and cannot assemble them.
Consider the natural query: why does F-1104 coke prematurely? Lexical retrieval matches on the tag string and returns the three prior investigation reports, the decoke work orders and the heater datasheet. Dense retrieval adds documents semantically similar to coking and heater fouling — general coking literature, sister-heater records, other units' fouling analyses. Reranking sharpens the ordering. The result is a well-formed, entirely useless retrieval set: it contains the three investigations that already failed.
|
Evidence fragment |
Terminology it actually uses |
Why hybrid retrieval misses it |
|
Cargo assay data |
Terminal identifier, oxygenate ppm, cargo number |
Structured tabular data; not in the document corpus at all,
and not text-retrievable in any case |
|
Post-trip header lineup |
Valve tags, "reduced rate operation", compressor
trip narrative |
Never states which heater is favoured; the routing
consequence is hydraulic, not documented |
|
Vendor field service report |
Superheater bundle, unit 1100, "slightly drier steam
observed" |
Zero lexical overlap with the query; semantically distant;
never names the heater |
|
Management-of-change 19-441 |
Sulphur dioxide, emissions target, skid U-31, dosing set
point |
Zero overlap with coking vocabulary; its relevance depends
on knowing U-31 serves F-1104 |
|
Shift logs, March 2021 |
Free-text operator notes, informal equipment names |
Individually unremarkable; relevance emerges only in
conjunction with the other three |
The deeper problem is visible in the table's final column. Each fragment is relevant only because the others are relevant. A ranking function scores documents against a query independently; it has no mechanism for recognising a conjunction whose members are individually weak. Increasing k does not help — the fragments are not at rank 50, they are at rank 4,000, below tens of thousands of documents that mention coking or F-1104 with far greater surface relevance.
3.5 Why the obvious combinations also fail
Three architectures that appear to combine the approaches do not, in fact, address this problem. Each is worth naming because each is commonly deployed.
Corpus-derived graphs
A widely-used pattern extracts entities and relationships from the document corpus itself and builds a graph from them, using that graph to organise retrieval and summarisation. This is genuinely useful for corpus navigation and thematic synthesis. It does not solve the problem here, because the graph it produces is derived from the same text that is missing the necessary connections. It cannot contain the plant connectivity model, because no document states it. It cannot contain a bitemporal record, because documents do not carry one. A graph extracted from a corpus inherits every gap in the corpus.
The graph as an additional retriever
A second pattern treats the graph as one more source in the fusion — retrieve from vectors, retrieve from lexical index, retrieve from graph, merge, rerank. This preserves the failure exactly. The graph is being queried by similarity to the question, so it returns the same neighbourhood of F-1104 that the text retrieval returned. The graph's distinctive contribution is not that it holds more documents; it is that it can restrict and verify. Used as a peer retriever, that contribution is discarded.
Sequential handoff
A third pattern runs the graph query first, passes the result to the retrieval system as context, and generates an answer. This is closer, and it is where most implementations stop. It fails on this problem because a single forward pass cannot express the necessary reasoning: the relevant graph traversal is not the one implied by the question. Nothing in "why does F-1104 coke" indicates that the answer requires expanding to the additive dosing skid two hops away, or that the expansion must be temporally scoped to the state of the plant during specific historical windows. The traversal must be driven by evidence that has not been retrieved yet, and the retrieval must be driven by a traversal that has not been performed yet. This is a loop, not a pipeline.
4. The binding layer
The combined system is not two systems with an interface between them. Its central artefact is a binding — a persistent set of edges connecting every unit of retrievable text to the graph entities it makes statements about. Everything in Section 5 depends on this binding existing and being of good quality. It is built once, maintained continuously, and is where most of the engineering effort in this architecture is spent.

4.1 Chunking for anchorable evidence
Chunking strategy in a bound system differs from conventional practice. The unit of retrieval should be the unit that makes a coherent, independently interpretable assertion — because each chunk will be anchored to entities and later assembled with chunks from other documents, and a fragment that cannot stand alone will produce spurious anchors and unusable evidence.
• Prefer structural segmentation over fixed-length windows: section, clause, table row, log entry, drawing annotation
• Retain document-level context on every chunk — document type, author role, issuing organisation, revision, approval status — because these become filters and trust signals later
• Keep tables intact and separately typed; a specification table is a different kind of evidence from a narrative paragraph and should be retrievable as such
• Preserve the parent-child relation so that a retrieved chunk can be expanded to its surrounding context during evidence assembly without re-retrieval
4.2 Entity linking and the anchoring edge
Every chunk is passed through an entity linker that identifies mentions of graph entities and resolves them to node identifiers. This is the same identity problem described in the companion paper, in its hardest form: the corpus spans decades of naming conventions, includes handwritten annotations transcribed by optical character recognition, and contains informal names that appear in no register.
The linker should therefore be layered, and should be the same resolution service the graph's own ingestion pipelines use:
1. Exact and normalised matching against every known identifier for every entity, including superseded identifiers with their historical validity periods — this alone resolves the majority of industrial mentions, because tag numbers are distinctive strings
2. Pattern-based extraction for identifier formats that follow a known grammar, including partial and abbreviated forms
3. Contextual disambiguation using the graph itself: where a mention is ambiguous, prefer the candidate that is structurally close to entities already confidently linked in the same document
4. Model-based linking for informal references, with confidence scored and recorded
The output is an anchoring edge carrying the chunk identifier, the entity identifier, the character offsets of the mention, the linking method, and a confidence value. Anchors are assertions like any other and are governed like any other: they carry provenance, they can be retracted, and their precision is measured.
|
Anchor
quality is the ceiling on system performance Every subsequent capability in this architecture is bounded
by the recall of the anchoring layer. A chunk that was never anchored to the
entity it discusses is invisible to frontier-scoped retrieval no matter how
good the retrieval is — it has been excluded before ranking begins. This is
the principal risk of the architecture and the principal thing to measure.
Section 8 treats it as its own failure mode. |
4.3 Temporal stamping
Each chunk carries two independent time attributes, mirroring the bitemporal discipline of the graph itself: the period the chunk makes assertions about, and the moment the chunk was authored. A 2023 incident report describing a 2019 event asserts about 2019 and was authored in 2023. Conflating the two makes historical reasoning impossible and is a common defect in document indexes, which typically carry only a file date.
Assertion periods are extracted where stated, inherited from the document where available, and left explicitly unknown otherwise. An unknown assertion period should be represented as unknown rather than defaulted to the authoring date, because a default that is silently wrong is worse than an absence that can be handled.
4.4 Assertion typing
Chunks should be classified by the kind of claim they make, because different claim types warrant different treatment in evidence assembly:
• Specification — a designed or required value. Authoritative for intent, not for actual state.
• Observation — a measured or witnessed condition at a stated time. The most valuable class for the reasoning in Section 5.
• Decision — a change made, with its rationale. Management-of-change records, approvals, deviation permits.
• Interpretation — an analytical conclusion, which may be wrong. Prior root cause findings belong here, and must not be treated as ground truth. In the worked case, two of the three prior investigations reached conclusions that were subsequently disproven; a system that weights them as fact will reproduce their errors.
• Negative evidence — an explicit statement that something was checked and found normal. Frequently the most diagnostically useful class and almost always discarded by conventional indexing, because such statements are semantically bland and rank poorly against every query.
5. Query-time architecture
With the binding in place, the query-time system operates as a loop between structure and text rather than a pipeline through them. Figure 2 shows the arrangement; the subsections that follow describe each stage and what it contributes to the worked problem.

5.1 Decomposition
The question is decomposed into subgoals classified by which system can answer them. "Why does F-1104 coke prematurely" yields structural subgoals (what is F-1104, what supplies it, what has changed in its supply chain and when) and explanatory subgoals (what mechanisms cause accelerated coking, what changed and why). The decomposition is itself revisable: subgoals are added as evidence accumulates, which is what makes this a loop.
5.2 Resolve and scope
Named entities in the question are resolved to graph identifiers through the same resolution service used at binding time. The temporal scope is established — here, the windows of the recorded coking events, which the graph holds as maintenance history. This produces a set of seed entities and a set of time windows, and every subsequent step is bounded by both.
This stage also expands the identity: F-1104 in 2016 documents may appear under a pre-revamp designation. All historical identifiers are added to the lexical query set, which is a capability the corpus alone cannot provide.
5.3 Topological expansion — the retrieval frontier
This is the mechanism on which the architecture rests. Rather than retrieving against the whole corpus, the system traverses the graph outward from the seed entities along semantically meaningful relationship types, and collects the set of chunks anchored to any entity reached. This set is the retrieval frontier.
Traversal is typed, not undirected. For a fouling or degradation question, the relevant relationship classes are material flow, utility supply, chemical dosing, thermal coupling and shared control. Traversal along, say, cost centre or document custody relationships is excluded — untyped n-hop expansion produces a frontier containing most of the plant and defeats the purpose.
The effect on the worked case is decisive. The management-of-change document for skid U-31 has no lexical or semantic relationship to the query. But U-31 is two typed hops from F-1104 in the graph — it doses the feed header that supplies the heater — so every chunk anchored to U-31 enters the frontier. Within a frontier of a few thousand chunks rather than four hundred thousand documents, a chunk describing a set point reduction on a passivating additive is a strong match for a subgoal about protective chemistry. It was never a weak match; it was a strong match that could not be found.
|
The
inversion Conventional retrieval asks: which documents are most
similar to this question? Frontier retrieval asks: which documents are about
things that are physically connected to the subject of this question, and
which of those bear on the mechanism? The second question is answerable only
with a structural model, and it is the question that finds evidence sharing
no vocabulary with the query. |
5.4 Fused retrieval within the frontier
Standard hybrid retrieval now runs, but over the frontier rather than the corpus. Three signals are fused: lexical matching including all historical identifiers, dense semantic matching, and graph proximity — the inverse traversal distance from the seed, which serves as a structural prior. Fusion by reciprocal rank is robust and avoids score-scale calibration between heterogeneous signals.
Two further filters apply that unbound retrieval cannot express. Temporal filtering restricts to chunks whose assertion period intersects the event windows, or whose authoring predates them and which have not been superseded. Assertion-type weighting boosts observations and decisions for causal subgoals, and explicitly down-weights prior interpretations so that failed investigations do not dominate the evidence set.
5.5 Co-temporal evidence assembly
Retrieved chunks are assembled into evidence bundles organised by subgoal and by time window, and each candidate condition is projected onto the event timeline. This is where conjunction becomes visible, and it is a graph operation rather than a text one: the graph knows when the additive set point changed, when the superheater bundle was replaced, when each cargo was received and routed, and when each compressor trip occurred.
Projecting these onto the seven recorded coking events produces the discriminating observation. Every coking event falls within a window in which all four conditions held. No coking event falls outside such a window. Equally important, there are windows in which three of the four conditions held and no coking occurred — which is what elevates the finding from correlation to a specific conjunction and identifies which condition is load-bearing.
Neither system produces this alone. The corpus supplies the candidate conditions; the graph supplies the timeline against which they are tested.
5.6 Constrained hypothesis generation
The language model now receives the assembled evidence together with the relevant subgraph — the entities, their typed relationships, and the temporal record — and is asked to propose mechanisms consistent with both. The instruction that matters most is a prohibition: every proposed causal link must be expressible either as a traversable path in the supplied subgraph or as a mechanism explicitly stated in a supplied chunk, with a citation. Mechanisms that require an unstated connection are to be flagged as speculative rather than asserted.
This constraint is what converts generation from a source of plausible narrative into a source of testable hypotheses. Without it, a model given rich context will produce a fluent explanation that reads exactly like a correct one.
5.7 The graph as falsifier
Each generated hypothesis is then mechanically tested against the graph. This stage is what most implementations omit, and it is the difference between a system that produces convincing answers and one that produces defensible ones.
• Path existence. Does a physical route actually exist between the proposed cause and the proposed effect? A hypothesis implicating a dosing skid that supplies a different train is rejected outright.
• Temporal consistency. Did the proposed cause precede every instance of the effect? A cause dated 2019 cannot explain events in 2016, and the graph knows both dates.
• Co-temporal validity. Were all conjunct conditions simultaneously in force under the graph's bitemporal record, using valid time rather than the date on the document?
• Specificity. Does the hypothesis explain why F-1104 and not its seven sister heaters? A proposed cause common to all eight fails this test, which is precisely where the first two historical investigations went wrong.
• Negative-case consistency. Are there windows in which the proposed cause held and no event occurred? If so, the hypothesis is incomplete and returns for refinement with the anomaly stated.
Rejected hypotheses return to generation with the specific reason attached, which is materially more useful than a bare rejection: a hypothesis failing only the specificity test usually needs one additional differentiating condition, and the failure tells the system what to look for.
5.8 Answer with a traceable path
The output is not prose alone. It is a structured finding: the proposed mechanism, the subgraph path that carries it, the specific chunks that evidence each link with document and revision identifiers, the temporal windows over which the conjunction held, the falsification tests passed, and the residual uncertainty.
In an engineering context this structure is not a presentational nicety. A conclusion that a change to emissions-driven sulphur dosing has an unrecognised interaction with coil passivation will be challenged, and correctly so. It is defensible only if every link can be traced to a source and every temporal claim to the record.
5.9 Write-back and the learning loop
Once a finding is validated by a competent human, it is written back to the graph as new assertions — a dependency between additive dosing and coil coking propensity, with provenance pointing to the evidencing chunks, a confidence value, and the identity of the approver.
This closes the loop in a way that compounds. The relationship that had to be discovered through the full nine-stage process is thereafter a traversable edge, available to every future query at no cost. The system's structural model grows from its own reasoning, under governance. The companion paper's disciplines apply without exception: these are curated assertions, they carry full provenance, they are subject to the ontology change process if they require a new relationship type, and they can be retracted if later disproven.
6. Why the combination exceeds the parts
It is worth stating precisely what the combination provides that neither component does, because this is the argument that must survive scrutiny from a sceptical architect or a budget holder.
|
Mechanism |
What it does |
Why neither system has it alone |
|
Frontier restriction |
Uses structural connectivity to determine which text is
eligible for retrieval, independently of similarity |
The graph has connectivity but no narrative; retrieval has
narrative but no connectivity model |
|
Cross-source conjunction |
Recognises fragments from unrelated documents as statements
about the same entity in the same period |
Requires identity resolution and a bitemporal record that
no corpus contains, applied to content no graph contains |
|
Falsification |
Mechanically rejects hypotheses that are physically or
temporally impossible |
Retrieval systems have no model of possibility; graphs have
no mechanism to test against |
|
Historical scoping |
Determines what was in force at the time of an event rather
than now |
Documents carry authoring dates, not validity; only a
bitemporal graph carries the distinction |
|
Compounding structure |
Converts each validated finding into a permanent
traversable relationship |
Retrieval systems do not accumulate structure; graphs do
not generate hypotheses |
The unifying observation is that each system supplies the other's missing constraint. Retrieval without structural constraint searches a space too large and too flat to find weak-signal evidence. Structure without narrative has nothing to reason over. Generation without falsification produces fluent output that cannot be trusted in an engineering decision. The three failures are eliminated by the same coupling.
7. Evaluation
Conventional retrieval metrics do not measure this system. Answer relevance, faithfulness and context precision all assume that the correct evidence is retrievable by similarity, which is the assumption the architecture exists to escape. A system scoring well on standard benchmarks may fail entirely on the problems that motivated its construction.
|
Measure |
Definition |
What it protects against |
|
Anchor recall |
Proportion of entity mentions in a labelled sample
correctly linked to the right node |
Silent exclusion of evidence before ranking begins — the
architecture's primary risk |
|
Anchor precision |
Proportion of anchors that are correct |
Frontier dilution, where wrong anchors admit irrelevant
chunks and degrade ranking |
|
Frontier recall |
Proportion of known-relevant chunks that fall within the
generated frontier |
Over-tight traversal typing that excludes the answer |
|
Frontier efficiency |
Ratio of frontier size to corpus size at fixed recall |
Over-loose traversal that recreates the original search
problem |
|
Conjunction recall |
Proportion of multi-source conjunctions in a labelled set
that the system assembles completely |
The core capability; nothing else measures it |
|
Falsification rate |
Proportion of generated hypotheses rejected by graph
adjudication |
A rate near zero means the falsifier is inert, not that
generation is excellent |
|
Trace completeness |
Proportion of asserted causal links carrying a citation or
a graph path |
Unevidenced assertion presented as finding |
|
Temporal correctness |
Proportion of historical questions answered against the
state in force at the time |
Silent use of current state to answer historical questions |
Two of these deserve emphasis. Falsification rate is diagnostic rather than directional: a system rejecting almost nothing is not producing excellent hypotheses, it has a falsifier that is not actually testing anything, and this is the most common way the architecture degrades into an ordinary retrieval system with additional latency. Conjunction recall requires a deliberately constructed evaluation set — a handful of known multi-source problems with their evidence chains labelled by domain experts — which is expensive to build and is the only direct measure of whether the system does the thing it was built for.
8. Failure modes
|
Failure mode |
Symptom |
Mitigation |
|
Anchoring gap |
The system performs no better than plain retrieval on hard
problems |
Measure anchor recall on a labelled sample before anything
else; treat sub-90% recall on identifier mentions as a blocker |
|
Frontier explosion |
Traversal returns most of the plant; retrieval quality
collapses |
Type the traversal by relationship class and question type;
cap hop count; rank frontier entities by structural relevance |
|
Frontier starvation |
Correct answer excluded because its entity was outside the
traversal |
Widen typed traversal for exploratory questions; always
retain an unrestricted retrieval channel as a fallback and report when it
disagrees |
|
Inert falsifier |
Near-zero rejection rate; confident answers that are wrong |
Adversarial test set of hypotheses known to be impossible;
require the falsifier to catch them |
|
Interpretation contamination |
The system reproduces the conclusions of prior failed
investigations |
Assertion typing with explicit down-weighting of prior
interpretations relative to observations |
|
Temporal collapse |
Historical questions answered with current configuration |
Bitemporal filtering enforced at retrieval, not applied as
a post-hoc caveat |
|
Write-back pollution |
Ungoverned machine-generated edges accumulate in the graph |
Human validation mandatory before promotion; staging
container; full provenance; retraction path |
|
Latency rejection |
Users abandon the system because answers take minutes |
Route simple questions to direct retrieval; reserve the
full loop for questions that warrant it — see Section 10 |
9. Building it
The build sequence matters, because the components have a strict dependency order and the temptation is to start with the visible one.
Prerequisite — the graph disciplines
Three properties from the companion paper are load-bearing here and cannot be retrofitted cheaply. Without an identity layer, anchoring cannot resolve historical designations and the binding is unreliable from the first document. Without a bitemporal model, historical scoping and co-temporal assembly are simply not implementable — the system will answer 2016 questions with 2026 configuration and give no indication that it has done so. Without provenance, write-back cannot be governed and the graph degrades as the system runs.
An organisation without these should build them first. The combined architecture applied to an ungoverned graph produces confident, traceable-looking, wrong answers, which is a worse outcome than no system.
Phase 1 — Bind a narrow corpus
Select one document class with high identifier density and high diagnostic value — vendor field service reports, management-of-change records, or incident investigations. Build chunking, entity linking and temporal stamping for that class alone. Measure anchor precision and recall against a hand-labelled sample of several hundred mentions. Do not proceed until recall on identifier mentions is strong; every later capability is capped by this number.
Phase 2 — Frontier retrieval
Implement typed traversal and frontier-scoped hybrid retrieval. Evaluate against a small set of known-answer questions whose evidence sits outside the query's vocabulary — the deliberately hard cases. At this stage the system already delivers value: it finds relevant material that conventional retrieval cannot rank, without any generation involved.
Phase 3 — Assembly and falsification
Add co-temporal evidence assembly, constrained generation and graph adjudication. Build the adversarial hypothesis set before building the falsifier, so that the falsifier is developed against tests rather than tested after the fact.
Phase 4 — Write-back under governance
Add the learning loop last. It requires a validation workflow with named accountable reviewers, a staging container, and integration with the ontology change process for findings that need new relationship types. Deploying write-back before this governance exists is the fastest route to an untrustworthy graph.
Phase 5 — Broaden
Extend to further document classes and further domains, re-measuring anchor quality for each new class. Anchor quality is class-specific: a linker tuned on structured service reports will underperform on scanned handwritten log books, and the difference will not be visible in aggregate metrics.
10. Cost, latency and when not to build this
The architecture is expensive in three ways. The binding layer requires sustained engineering effort and does not degrade gracefully — a partially anchored corpus behaves like an unanchored one. Query latency is materially higher than direct retrieval, since the loop involves several traversals, a retrieval pass, one or more generation calls and an adjudication pass. And the governance overhead of write-back is real and recurring.
Against this, the system should be reserved for questions that justify it. A practical routing policy sends factual lookup and single-document questions to direct retrieval, sends structural questions to the graph, and invokes the full loop only for explanatory, diagnostic and multi-source questions. Most enterprise queries are not of the third kind; the ones that are, are disproportionately valuable.
There are situations in which this architecture is the wrong answer:
• The corpus is small enough that an entire relevant subset fits in a model context window — the frontier mechanism is solving a problem that does not exist
• The domain has no meaningful physical or organisational topology, so typed traversal has nothing to traverse
• Questions are overwhelmingly current-state rather than historical, removing the temporal contribution
• No governed graph exists and none is planned, in which case the honest answer is to build hybrid retrieval well and revisit later
Conversely, the case is strongest where problems are recurrent and expensive, where the facility is old enough that institutional memory has dispersed, where naming regimes have changed across ownership or contractor transitions, and where explanations must withstand engineering and regulatory challenge. Those conditions describe most large process facilities.
11. Closing observations
The framing of knowledge graphs and retrieval-augmented generation as competing approaches has persisted longer than it should. They are not competitors; they fail in orthogonal directions. A graph fails by omission — it cannot contain what was never modelled. Retrieval fails by flatness — it cannot see connections that are not linguistic. The class of problems that defeats both is not a narrow edge case; it is the standard shape of the expensive recurring problem in a mature industrial facility.
What resolves it is not adjacency but coupling. The graph must scope where retrieval looks, supply the identity and temporal scaffolding that lets scattered fragments be recognised as a conjunction, and adjudicate what generation proposes. Retrieval must supply the mechanisms, rationales and observations that the graph structurally cannot hold. Each constrains the other, and the constraint is the value.
One design decision separates the systems that work from those that merely demonstrate well. It is not the retrieval strategy, the model, or the graph platform. It is whether the graph is permitted to reject what the language model produces. A system that can only retrieve and generate will produce a fluent, well-cited, entirely wrong explanation for the coking in F-1104, and it will produce it quickly. A system that must first prove the path exists, that the timing is consistent, and that the mechanism explains why this heater and not its seven identical siblings, will either produce a defensible finding or produce nothing — and in an engineering context, nothing is a legitimate and valuable answer.
Build the falsifier. Everything else is retrieval.
About this paper
This paper is a companion to "Knowledge Graphs for the Industrial Enterprise" and forms part of an ongoing whitepaper series on industrial digitalisation, covering operational technology architecture, cybersecurity, sustainability and the energy transition. The F-1104 case is hypothetical and is constructed from the standard components of recurring-failure investigations; any resemblance to a specific facility is coincidental. The paper is vendor-neutral: capabilities are described functionally and no specific platform, model or graph engine is recommended.