Skip to content

OPENNLP-1888: Document annotation container with typed offset-anchored layers - #1182

Open
krickert wants to merge 32 commits into
mainfrom
OPENNLP-1888-DocumentShape
Open

OPENNLP-1888: Document annotation container with typed offset-anchored layers#1182
krickert wants to merge 32 commits into
mainfrom
OPENNLP-1888-DocumentShape

Conversation

@krickert

Copy link
Copy Markdown
Contributor

Adds the document annotation container discussed on OPENNLP-1888: an immutable Document over the original text, typed LayerKey identities, span-anchored Annotation values, DocumentAnnotator with declared requires/provides, and a DocumentAnalyzer whose pipeline ordering is validated at build time. Four adapters over the existing single-task interfaces (sentence detector, tokenizer, POS tagger, name finder) plus lemmatizer and stemmer layer adapters come with it; the container itself never learns about specific layers.

Contract behavior, each pinned by a test asserting the exact message where one is thrown: spans are structurally mandatory and validated against the text length; key equality is the (id, type) pair; layers preserve insertion order and are never sorted; layers are immutable once added and detached from caller input; providing an already-present layer is rejected; reading an absent layer returns an empty immutable list. 33 tests, including a full pipeline example and a contract suite.

Follow-ups planned as commits on this PR, from the review discussion on the ticket: namespaced identifiers for the standard keys; per-key positional versus document scope for whole-document facts; the invariants above transcribed into the specification text; and the documented convention for gold versus predicted layers. Opening as a draft until those land.

The acceptance criterion suggested in that discussion, that a new layer can be added without touching the container package, is already observable: the feature branches on the ai-pipestream fork (glossary, PII, coreference, dependencies, relations, money/quantity/temporal, geo, embeddings) each add their layers with no container edits.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants