[DEV-1824] Document the data source order contract and fix contradicting examples - #292
Open
galisufyan-327 wants to merge 5 commits into
Open
[DEV-1824] Document the data source order contract and fix contradicting examples#292galisufyan-327 wants to merge 5 commits into
galisufyan-327 wants to merge 5 commits into
Conversation
…xamples `POST /v1/data-sources/:id/data/query` validates `order` before running the query, but the rules lived nowhere in the docs: the array-of-arrays shape was only shown by example, and the five-column unprefixed allowlist existed solely as a comment inside a code fence. Meanwhile several examples sorted by a data column with no `data.` prefix, which the API rejects with 400. - API/fliplet-datasources.md: `## Sorting and Ordering` now states the four rules in a table, warns that a flat array reports a single-character column name, and carries a Common mistakes block. Adds the first multi-word column example in the corpus (`data.Start Time UTC`), since the verbatim-with-spaces rule previously had no positive evidence to copy. Prefixes the five `Name` sorts that were missing `data.`. - API/datasources/joins.md: scopes the join-level `order`, which is a single flat `[column, direction]` pair sorted in memory after the fetch and is NOT the top-level nested shape. Left flat, because flat is what the join code reads; the two shapes are now explicitly told apart. - REST-API/fliplet-datasources.md: the `order` bullet pointed at a dead anchor, so an agent following it landed on the page top. Points at the real heading and restates the rules in JSON form next to the endpoint. - coding-standards.md, code-api-patterns.md: prefix the data column, point at the rule. Regenerates .well-known/llms-full.txt from source. No frontmatter changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review follow-ups on the previous commit. Every claim below was re-checked in `routes/v1/data-sources.js` before rewording. - API/fliplet-datasources.md: the intro promised "four rules" above a five-row table. Scoped to "the shape or column rules" instead, which also fixes the wider problem — the 400 framing covered the Direction row, but `direction` is never validated (`sortDirection = direction || 'ASC'` is interpolated raw into the SQL literal), so a bad direction surfaces as a database error, not a 400. Direction is now presented as a convention. - API/fliplet-datasources.md: the contract's example block declared `connection` four times in one fence, so pasting it whole threw a redeclaration SyntaxError. Split into four fences, each independently runnable. - API/fliplet-datasources.md, REST-API/fliplet-datasources.md: "a one-character column name always means the shape is flat" was false — a declared one-character column used unprefixed reports the same error. Softened, with the second cause named. - API/fliplet-datasources.md: the undeclared-column error is conditional. The check only runs when the data source declares columns; with none declared the sort silently yields NULLs. Qualified. - API/datasources/joins.md: a join `order` also accepts a bare string, coerced to `[order, 'ASC']`. Added. - API/fliplet-datasources.md, REST-API/fliplet-datasources.md, coding-standards.md: state that the `data.` prefix is specific to `order`. `where` keys are nested under `data` by the API and `attributes` are picked off `entry.data`, so both take unprefixed column names — without saying so, a reader could over-generalize the prefix and break working queries. Regenerates .well-known/llms-full.txt from source. No frontmatter changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The prefix note said the data. prefix was specific to order and told readers not to generalize it. Join `on` keys and aggregate field paths address the raw entry row, so they require the prefix; dropping it there fails silently. Also correct the direction row (an invalid direction is still a 400, only the message differs), note that the direction must be a literal because it reaches the query unescaped, and soften the "only other way" claim about one-character column names in the error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The `data.` prefix also applies to join `on` keys/values and `aggregate` field paths, which read the raw entry row. Keep only the accurate half: `where` and `attributes` take unprefixed column names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The direction row described how the API handles the value, not just what to write. Keep the actionable half — write a literal, never build it from user input, ASC is the default — and drop the implementation detail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deploying fliplet-cli with
|
| Latest commit: |
30753bf
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://72284a0f.fliplet-cli.pages.dev |
| Branch Preview URL: | https://fix-dev-1824-ds-order-contra.fliplet-cli.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Product areas affected
Developer documentation — data sources (
docs/API/fliplet-datasources.md,docs/API/datasources/joins.md,docs/REST-API/fliplet-datasources.md,docs/coding-standards.md,docs/code-api-patterns.md) and the regenerated.well-known/llms-full.txt.What does this PR do?
States the data source
ordercontract forPOST /v1/data-sources/:id/data/query, and removes the examples that contradicted it.These docs are fetched at runtime by the V3 AI builder and override its own system prompt, so a wrong example here actively teaches the wrong shape. Before this PR the contract existed only as a code comment inside a fence, while eight-plus examples across the corpus showed forms the API rejects.
The contract, now stated as a table in
## Sorting and Ordering:orderis an array of arrays — never flat.id,order,createdAt,deletedAt,updatedAtmay appear unprefixed.data.<ColumnName>.data.must match the declared column verbatim, spaces included.Changes:
API/fliplet-datasources.md— contract table, a warning explaining the single-character error a flat array produces, an info block on undeclared columns, the corpus's first multi-word column example (data.Start Time UTC), and a WRONG/RIGHT block. Five unprefixedNameexamples fixed todata.Name. The one oversizedjsfence was split into four independently runnable fences.API/datasources/joins.md— scoped, not "fixed". See below.REST-API/fliplet-datasources.md— dead anchor#sort--order-the-results→#sorting-and-ordering, plus the same rules in JSON form.coding-standards.md,code-api-patterns.md— corrected/annotated examples.JIRA ticket
DEV-1824
Result
Verified against the API implementation (
routes/v1/data-sources.js) rather than against the existing prose — every quoted error string matches byte-for-byte, and each documented failure mode was traced to the line that produces it.Corpus state after the change:
orderexample remains. Surviving flat examples are either inside ajoin:block (where flat is correct) or explicitly labelled WRONG.order: [['X', …]]with an unprefixed data column remains, except WRONG-labelled examples andAPI/core/analytics.md(a different endpoint, where the column is anattributesalias).Docs CI (the exact gate in
.github/workflows/docs-validate.yml):llms-full.txtis a faithful regeneration — re-running the generator leaves it byte-identical.Checklist
Documentation-only; no runtime surface and no test seam to add. Verification is the CI gate above plus corpus-wide greps proving no contradicting example survives.
Deployment instructions
None beyond the normal docs deploy. No frontmatter
description:ortitle:changed, so the search index entries are unchanged;llms-full.txtis regenerated and committed in-tree, so the freshness check is satisfied.Companion Studio PR (Fliplet/fliplet-studio#8868) states the same contract in the builder prompt. Since these docs override the prompt, both should land together.
Author concerns
1.
joins.mdwas deliberately NOT converted to the nested shape. The original plan called those flat examples the same defect class. They are not: a join-levelorderis a different parameter, sorted in memory after fetch (_.get(entry, order[0]),order[1] === 'DESC'), and it never reaches theallowedOrderColumnsvalidator. Nesting it would have madeorder[0]an array andorder[1]undefined, silently breaking join ordering and publishing a false doc. The examples are scoped instead, and the bare-string form (coerced toASC) is now documented too.2. The
data.prefix is not exclusive toorder, and saying so was a real bug in an earlier revision of this branch.wherekeys andattributesentries take bare names, but joinonkeys andaggregatepaths do require the prefix. An earlier draft told readers not to generalize the prefix beyondorder— which would have led an agent to write bareonkeys and get a silently empty join (or, on arequiredjoin, a dropped row). All three sites now say only what is true, and the corpus was grepped to confirm no fourth site survives.3. Pre-existing fence defects remain.
API/fliplet-datasources.mdhas 11 fences that redeclareconst connection, all pre-existing and untouched here. This branch removed one such fence and introduced none (12 → 11). Fixing the rest is out of scope.4. One over-broad edge-case description. The docs say a prefixed one-character name that is "not a letter, digit, space, underscore or hyphen" produces the one-character error. The validator regex is
/^[A-z0-9 _-]*$/, whoseA-zspan also admits[ \ ] ^ `. Thedata.@example given is correct; only the class description is marginally wider than the regex.