What the literature says and how well it says it are different questions.
A curated database records what a claim says and where it came from. Almost none of them record how much the evidence actually supports the claim, which is the question anyone acting on the database needs answered. I built a curation pipeline that assigns that grade, row by row, keyed to the citation it was computed from, and ran it at organism scale on the published literature.
3,500 curated rows across 571 organisms, each keyed to its entry in a public protein database. 1,840 rows carry a numeric evidence grade; 1,402 of those sit at 0.8 confidence or above. 2,642 distinct PubMed citations behind the annotations.
Advisory tooling. The grades inform review; a domain expert reads the evidence before anything is acted on. Built on public data, delivered for a client.
The friction
When a business or an agency has to act on scientific claims, someone has to decide how much each claim is worth. A supplier attestation, a safety dossier, a risk register, a literature review: each is a list of claims that someone downstream has to trust, partly trust, or reject.
Curated databases usually record what a claim says and where it came from, then stop. The confidence question is left to the reader, who has neither the time nor the access to re-derive it. A row backed by a single dated abstract and a row backed by a stack of recent studies present the same face, and the person acting on the list cannot tell them apart.
The expensive part is reading each source and deciding how much support it actually gives. At thousands of rows that does not get done by hand, so the trust assessment either never happens or happens once, informally, in somebody's head.
What I built
A curation pipeline that does the expensive part: it reads the retrieved sources behind each claim and records how strong the support is, with the citation attached. Three pieces carry the weight.
A curated reference database
The current build holds 3,500 curated rows spanning 571 organisms, each row keyed to its UniProt accession so the identity claim is checkable by anyone. Claims about protein function are organized under a 32-category functional-security ontology grouped into six super-categories, so a reviewer reads a named category with its evidence, not free text.
Evidence grading
Each row is graded by reading what the retrieved sources actually say and scoring the strength of support, on a scale a reviewer can read at a glance. The current database carries 1,840 graded rows, 1,402 of them at 0.8 confidence or above, with 2,642 distinct PubMed citations behind the annotations. A grade is computed from retrieved source text, never from a model's memory.
A contract-bound agent pipeline
The curation runs as a ten-stage pipeline across eleven agent specifications, and the output of each stage has to validate against a JSON Schema contract before the next stage can consume it. A stage that produces malformed output stops the pipeline instead of passing garbage downstream. The same inputs produce the same result, every run.
What exists today
The database and the pipeline both run. The numbers on this page were re-measured from the current output files: 3,500 rows, 571 organisms, 1,840 graded rows, 1,402 at 0.8 or above. This was delivered for a client. The single-process version, 1,596 lines of Python, produced the shipped database; the newer multi-agent pipeline has since been demonstrated end to end on the same curation path.
How it's built
Retrieval before generation, contracts between stages, and no grade without a source. The language model works from retrieved literature text inside a labeled advisory role and never originates a grade on its own. Where the ontology classifies a finding, the database records which category was chosen and what evidence backed the choice, so a reviewer can disagree with a specific decision instead of auditing the whole database at once. Deterministic stages mean the same corpus rebuilds to the same grades.
What it does not do
It does not predict anything. If the literature is silent on a protein, the database says so rather than extrapolating from a neighbor, and ungraded rows stay ungraded until sources are read. It is single-operator tooling: no team review interface, no scheduled refresh from upstream sources, no production deployment.
What a client engagement would deliver
The pattern transfers to any list of claims a business has to act on: supplier attestations, safety dossiers, regulatory registers, literature reviews. A pilot takes one of your claim lists, grades each claim against its evidence with citations attached, and hands back the graded register, so the review conversation starts at confidence levels instead of raw assertions. A production build adds the refresh path and the review workflow on top.
How to engage
Three ways to engage, in order:
- Scoping call (free, 30 minutes). Identify the claim list your business has to act on and decide together whether grading it is worth a pilot.
- Pilot. One claim register, your sources, one graded deliverable, scoped before it starts.
- Production build. Scoped after the pilot. The pipeline becomes tooling your team operates under its normal review workflow.
All engagement is on standard McIntosh Consulting LLC terms.