Skip to content

Add a NodeNorm skill for coding agents, served at /llms.txt#405

Open
gaurav wants to merge 7 commits into
improve-documentationfrom
add-skill
Open

Add a NodeNorm skill for coding agents, served at /llms.txt#405
gaurav wants to merge 7 commits into
improve-documentationfrom
add-skill

Conversation

@gaurav

@gaurav gaurav commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Makes NodeNorm usable from a coding agent without the agent having to reverse-engineer the API from its OpenAPI schema.

Stacked on #403 — targets improve-documentation, not main, because the skill links to documentation/Babel.md which that PR introduces. Merge #403 first and this retargets cleanly.

Before merging

  • Run a real docker build. Docker was not available in the session that wrote this, so the image was never built — see Not verified. The COPY ./skills skills line is the only thing standing between this and a /llms.txt that works locally and 404s in every deployment.
docker build -t nn-skill-check .
docker run --rm nn-skill-check python -c \
  "from node_normalizer.config import SKILL_PATH; assert SKILL_PATH.is_file(), SKILL_PATH; print('skill present:', SKILL_PATH)"

Why a skill rather than MCP or a CLI

#403 already fixed the reference problem: every endpoint has a real description, the response schema is documented, examples are served. What an agent still lacked is judgement — when to reach for NodeNorm at all, which conflation flags to set, how to batch, and which behaviours fail silently rather than erroring.

  • MCP server — NodeNorm is an unauthenticated public API whose useful surface is essentially one endpoint. A server to build, host, version and register, plus per-user configuration, buys little. NameRes reached the same conclusion: its "Add MCP endpoints" issue (NameResolution#251) produced a skill file instead. Worth revisiting if usage shows agents actually struggling.
  • Python package / CLI here — a new packaging and release surface in a repo that ships a service and a loader, and Translator_sdk already wraps NodeNorm. The skill teaches batching against a documented contract instead.

What is in it

  • skills/nodenorm/SKILL.md — the skill, in the <name>/SKILL.md + frontmatter layout Claude Code actually discovers. Body is plain Markdown, usable pasted into any agent.
  • GET /llms.txt — the same document served by the instance itself, frontmatter stripped, linked from the OpenAPI description. An agent given only a base URL can bootstrap. One copy of the text: the route reads the skill file directly.
  • documentation/LLMs.md + a README pointer.
  • tests/frontend/test_llms_txt.py — 4 tests, no Redis needed.

Note the Dockerfile change. The image copies only node_normalizer/, config.json, redis_config.yaml and load.py, so without COPY ./skills skills this route works locally and 404s in every deployment.

Every claim is measured, not remembered

The numbers in the skill came from hitting the live RENCI instance, and the three traps it exists to prevent were all confirmed there:

Trap Evidence
GET/POST conflation defaults differ (#398) Identical query returns 206 equivalent identifiers via GET, 30 via POST
/get_setid parameters are inverted and silently ignored GET takes conflation, POST body takes conflations; wrong name returns HTTP 200 with an unconflated hash and empty conflations
Prefix matching is a coin flip The DMD clique has three ENSEMBL: entries — one gene, two proteins

The /get_setid one matters most: anyone comparing a gene against its protein with default flags concludes they are different concepts, with no error.

Validation

The skill was tested twice by agents given only the skill file plus network access, with no repo access.

First pass completed both target use cases and found no factual errors, but hit two traps — and both were the documents fault. /get_setid was a one-line stub whose "set both flags on every request" advice generalised wrongly, and the flagship recipe told it to match on prefix, which nearly returned ENSP00000288447 as "the Ensembl ID". Both fixed.

Second pass made zero wrong turns — but correctly pointed out I had tested it on its own worked examples, so it was really checking whether the doc could be copied, not whether it taught the model. Its substantive finding: the skill stated "30 vs 206" three times without ever giving the rule those numbers illustrate.

So the last commit adds the rule, measured across all four flag combinations for seven CURIEs: only the conflation matching the concepts kind does anything; the other flag is a no-op. Genes and proteins move only with conflate; chemicals only with drug_chemical_conflate; diseases and phenotypes not at all.

One reported finding was wrong and deliberately not applied — include_taxa was said to populate only individual entries. It populates both a top-level taxa array and the entries. Agent findings were verified rather than taken on faith.

Not verified

The Docker image was never built. Docker was not running. Instead the container layout was replicated (/code with node_normalizer/ and skills/ alongside) and SKILL_PATH confirmed to resolve, and there is no .dockerignore to exclude skills/ — so the COPY should be correct, but that is inference, not evidence. Please run the build in Before merging above.

Follow-ups filed

#404 (the Dockerfile still pins --root-path /1.3, the last source of the deprecated prefix), NameResolution#290 (that repos skill install instructions point at a path Claude Code does not discover, so the two services can converge).

🤖 Generated with Claude Code

gaurav and others added 6 commits July 27, 2026 05:54
PR #403 fixed the reference problem — every endpoint now carries a real
description and the response schema is documented. What an agent still lacks
is judgement: when to reach for NodeNorm at all, which conflation flags to
set, how to batch a whole column of identifiers, and the handful of behaviours
that silently produce wrong answers rather than errors.

This is that document. Every factual claim in it was measured against the live
RENCI instance rather than written from memory, including the equivalent-
identifier counts used to illustrate what conflation actually does.

Three things it exists to prevent, all of which fail silently:

- GET and POST disagree on the conflation defaults, so the same query returns
  206 equivalent identifiers one way and 30 the other (#398). The rule taught
  is to set both flags on every request.
- /get_setid takes `conflation` (singular) on GET and `conflations` (plural) in
  the POST body, defaults to no conflation unlike GET /get_normalized_nodes,
  and silently ignores the wrong name — returning HTTP 200 and an unconflated
  hash. Anyone comparing a gene against its protein this way concludes they are
  different concepts.
- A conflated clique routinely holds several identifiers sharing one prefix
  (the DMD clique has three ENSEMBL entries: one gene, two proteins), so
  matching on prefix alone is a coin flip. The recipe uses individual_types
  and filters on type.

Format is skills/nodenorm/SKILL.md with YAML frontmatter, which is what Claude
Code actually discovers; the body is plain Markdown usable by any agent. The
`description` names concrete CURIE prefixes, since that is what the model
matches on when deciding whether the skill is relevant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An agent handed a NodeNorm URL and nothing else has no way to find the skill
file, which lives in a GitHub repository it has no reason to know about. This
serves the same document from the instance itself, so the instructions are
always reachable from the thing being described and always match the deployed
version. It is linked from the OpenAPI description too, so an agent reading
/openapi.json will find it.

The YAML frontmatter is stripped on the way out — it is Claude Code packaging
metadata and noise in an llms.txt. There is still only one copy of the text:
the route reads skills/nodenorm/SKILL.md directly.

Note the Dockerfile change. The image copies only node_normalizer/, config.json,
redis_config.yaml and load.py, so without `COPY ./skills skills` this route
works locally and 404s in every deployment — the kind of failure that shows up
long after the PR is merged. A missing file returns a 404 naming where to read
the instructions instead, rather than a 500.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mirrors the structure of NameResolution#259 so the two services read the same
way, but with install instructions that work: Claude Code discovers skills as
<name>/SKILL.md inside a skills directory, so the whole skills/nodenorm/
directory has to be copied, not just the file. Filed as NameResolution#290 so
the sibling repo can converge on the same format.

Also documents fetching the instructions straight from a running instance with
/llms.txt, which is the shortest path for an agent that has a URL and nothing
else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two cold-read validations, each given only the skill file and network access,
turned up real gaps. The first hit two traps the document caused; both are
fixed in the preceding commits. This addresses what the second found.

The main change is generalization. The document stated "30 vs 206" three times
but never gave the rule those numbers illustrate, so a reader handed any CURIE
outside the three worked examples had to re-derive the flag matrix by trial and
error — exactly the work the skill claims to have already done for them. The
rule, measured across all four flag combinations for seven CURIEs spanning
genes, proteins, chemicals, diseases and phenotypes: only the conflation
matching the concept's kind does anything, and the other flag is a no-op.
NCBIGene:1756 and UniProtKB:P11532 move only with conflate (5->22, 4->22);
CHEBI:15377, MESH:D014867 and CHEBI:15365 move only with drug_chemical_conflate
(30->206, 30->206, 21->436); MONDO:0005148 and HP:0002465 do not move at all.

Also stated: POST defaults conflate to true and only drug_chemical_conflate
differs, so the GET/POST divergence bites chemicals and not genes; POST
/get_setid takes a bare JSON array, with an example, since "a list of objects"
left the shape ambiguous; per-entry `type` is a single string where the
clique-level `type` is a list; and malformed input, including the empty string,
returns null with HTTP 200 rather than an error, so it fails silently.

One reported finding was wrong and is not applied: include_taxa was said to add
taxa only to individual entries. It populates both a top-level `taxa` array and
the individual entries, which is what the document now says.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
construct_open_api_schema() regenerates `paths` from the route decorators and
merges back only document-level metadata, so both halves fail silently: metadata
that stops being copied just vanishes from the served schema, and endpoint
documentation written into openapi.yml is never served at all. That is how 235
lines of prose sat unserved until #403.

Five tests: the yml metadata (including the license, which was dropped for a
long time) reaches the schema; openapi.yml has no `paths` section and the routes
supply them; no operation is left on FastAPI's placeholder "Successful Response";
construct_open_api_schema returns the cached schema rather than calling a dict;
and get_app_info reads what it claims to.

Also records the split in CLAUDE.md, next to the note that the Dockerfile has to
COPY ./skills for /llms.txt to work in a deployed image.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two remaining master URLs, both in this PR's files. The install curl in
LLMs.md matters most: it is the one line a reader actually executes.

Both targets 404 right now, because skills/nodenorm/SKILL.md and
documentation/Babel.md only exist on these two branches. main is still the
right target -- an install instruction should hand people the merged file, and
pinning it to a branch would rot as soon as the branch is deleted. They resolve
once #403 and this PR land.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The link tests only looked at Markdown plus two hand-listed source files. A
hand-written list of "where links live" is exactly what lets a link rot
unnoticed, and it silently stops covering anything added later, so the file set
is now globbed by extension.

It immediately found one the Markdown-only scan could not see: a captured
/status sample in documentation/NodeNormalization.ipynb still showed a
blob/master babel_version_url. Normalized to main, matching the equivalent
sample in NameRes's API.md.

Worth knowing that this one is aspirational rather than descriptive: the value
comes from BABEL_VERSION_URL, which the deployment chart sets to a master URL,
so a live instance really does return master today. The fix for that belongs in
translator-devops.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant