OPENNLP-1888: Document annotation container with typed offset-anchored layers - #1182
Open
krickert wants to merge 32 commits into
Open
OPENNLP-1888: Document annotation container with typed offset-anchored layers#1182krickert wants to merge 32 commits into
krickert wants to merge 32 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the document annotation container discussed on OPENNLP-1888: an immutable
Documentover the original text, typedLayerKeyidentities, span-anchoredAnnotationvalues,DocumentAnnotatorwith declaredrequires/provides, and aDocumentAnalyzerwhose pipeline ordering is validated at build time. Four adapters over the existing single-task interfaces (sentence detector, tokenizer, POS tagger, name finder) plus lemmatizer and stemmer layer adapters come with it; the container itself never learns about specific layers.Contract behavior, each pinned by a test asserting the exact message where one is thrown: spans are structurally mandatory and validated against the text length; key equality is the (id, type) pair; layers preserve insertion order and are never sorted; layers are immutable once added and detached from caller input; providing an already-present layer is rejected; reading an absent layer returns an empty immutable list. 33 tests, including a full pipeline example and a contract suite.
Follow-ups planned as commits on this PR, from the review discussion on the ticket: namespaced identifiers for the standard keys; per-key positional versus document scope for whole-document facts; the invariants above transcribed into the specification text; and the documented convention for gold versus predicted layers. Opening as a draft until those land.
The acceptance criterion suggested in that discussion, that a new layer can be added without touching the container package, is already observable: the feature branches on the ai-pipestream fork (glossary, PII, coreference, dependencies, relations, money/quantity/temporal, geo, embeddings) each add their layers with no container edits.