Regulated data work needs lineage. Most pipelines ship trust instead.
A knowledge graph is a set of claims about the world, and building one is not the hard part. Defending it six months later is. I built a pipeline where provenance is structural: sources are pinned and versioned, every record validates before it enters the graph, every node and edge carries its source, and a query with no support returns an explicit abstention instead of a plausible answer.
106 passing tests, clean under a full automated code review. Four public reference sources acquired, validated, and built into a live graph of 80 nodes and 70 edges. Twelve of fourteen competency questions pass. Every source pinned and hash-verified.
Completed and independently reviewed, on a curated scope rather than the full source databases. The two unimplemented competency questions are documented in the project's own reports, not hidden.
The friction
A graph says this entity relates to that one, and you can rely on it. Then the questions arrive, months after the build: which source version was each fact taken from, what changed upstream since, and what happens when the graph is asked something it has no support for. Most pipelines answer none of that and present the graph as finished.
For regulated work, that is not a documentation gap. An auditor tracing a single fact back through a pipeline that never recorded its sources is stopped cold, and so is anyone trying to rebuild the result later. The lineage is not an appendix to the deliverable. It is the deliverable.
What I built
A pipeline that turns public reference sources into an audited graph, with the guarantees built into the stages rather than asserted afterward. Three stages carry them.
Acquisition with version honesty
Four public reference sources are acquired: UniProt, Rhea, ChEBI, and NCBI Taxonomy. Each record carries its source version, and where a source publishes no release identifier, the pipeline records that fact instead of inventing one. Version honesty at acquisition is what makes a later rebuild comparable to the original.
A validated build with provenance on every edge
Records validate against schemas before they enter the graph, and conflicts are logged, not merged. The build is a NetworkX graph exported to Neo4j and PyG formats, and a live authenticated Neo4j read-back verified the export at 80 nodes and 70 edges. Every node and edge carries its source record, so a reviewer can trace any fact to where it came from without the person who built the graph in the room.
Queries that abstain
Fourteen competency questions define the graph's behavior contract, and twelve execute and pass, including correct abstention when evidence is missing. All eight required edge cases demonstrate: ambiguity, contradiction, competing routes, missing evidence, unsupported assertions, out-of-scope entities. A query with no supported chain returns an abstention with its reason, not a plausible-looking substitute.
What exists today
The pipeline completed and was verified end to end: 106 passing tests with ruff and mypy clean, the live Neo4j read-back above, and an independent software review that returned accept with follow-ups, with the follow-ups remediated. The repository manifest is SHA256-pinned, so the exact inputs the claims were measured against are reconstructable. This was delivered for a client; client identity and outcome details are withheld.
How it's built
Validation happens at the boundaries, provenance travels with the data, and the build is deterministic. Acquisition records what it fetched and from which version. Curation validates each record against its schema and logs conflicts rather than merging them quietly. The graph build and its exports are deterministic, and the competency suite pins behavior, so a rebuild that changes an answer is a finding rather than a surprise.
What it does not do
The graph covers a curated scope, not the full breadth of its source databases. Two of the fourteen competency questions are not implemented, and both facts appear in the project's own reports. There is no production service and no live sync from upstream sources. It is a completed, reviewed build of the pattern, sized to prove the guarantees.
What a client engagement would deliver
The guarantees transfer to any data a business has to defend: supplier records, clinical data, financial lineage, safety dossiers. A pilot takes one of your datasets, pins its sources, validates records against contracts you approve, and hands back a graph or register where every fact traces to its origin and every unanswerable question abstains. A production build adds the refresh path and access controls on top.
How to engage
Three ways to engage, in order:
- Scoping call (free, 30 minutes). Identify the dataset your business has to defend and decide together whether a pilot is worth it.
- Pilot. One dataset, your sources, one audited deliverable, scoped before it starts.
- Production build. Scoped after the pilot. The pipeline becomes tooling your team operates under its normal review workflow.
All engagement is on standard McIntosh Consulting LLC terms.