Skip to content

feat: page the enrichment queue (CMP-92) - #160

Merged
ripgrim merged 3 commits into
mainfrom
rg/enrichment-queue-paging
Aug 18, 2026
Merged

feat: page the enrichment queue (CMP-92)#160
ripgrim merged 3 commits into
mainfrom
rg/enrichment-queue-paging

Conversation

@github-actions

@github-actions github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Follows #159 — three findings from that review, plus the two limits it shipped with.

The queue endpoint took no input and always read 20 rows, while the footer counted every due task. Past 20 the widget promised records it could not open. It now takes a limit. The footer offers Show all for rows already fetched, and Load more for rows the server held back.

ENRICHMENT_PAGE and ENRICHMENT_PAGE_MAX live in @crm/validation/enrichment-queue, so the widget and the service cannot disagree about the maximum.

The footer decision moved out of the component into queueFooter, covered for every combination of total, fetched, shown and limit. That is where the miscount was.

The PR title job died with a permissions error on a fork pull request, taking the whole check down with it. It now reports the suggested title and lets the linter run.

Verified in a browser against 32 due rows: 27 more → Show all → 12 more → Load more → footer gone, every record reachable.

🤖 Generated with Claude Code


Summary by cubic

Pages the enrichment queue so the widget can reach all due tasks. Previously the API always returned 20 rows while the footer counted all due tasks; now the API accepts a limit, and the footer offers Show all for fetched rows or Load more to request more up to the maximum.

  • API: enrichment.queue now takes { limit } validated by @crm/validation/enrichment-queue; the service clamps with pageSize, defaults to ENRICHMENT_PAGE, and applies the limit to both due and scheduled lists. Constants live in @crm/validation/enrichment-queue to keep client and server in sync.
  • UI: footer logic moves to queueFooter with tests. Uses keepPreviousData to hold fetched rows while increasing the limit. Shows “Show all” when more rows are already fetched and “Load more” until ENRICHMENT_PAGE_MAX; stops offering beyond the max. Resets the limit on popover close and passes { limit } to the query.
  • Tests: prove clamping (including NaN/Infinity), defaulting, exact row counts vs total, input validation boundaries, and footer states.
  • CI: the PR title workflow keeps the check green on fork permissions; it posts the suggested title and lets the linter run.

Migration/Review

  • Callers of trpc.enrichment.queue should pass { limit } (or {}); omitting limit uses the server default (20). Check any external consumers that relied on a no-input signature.

Written for commit 2632b3a. Summary will update on new commits.

Review in cubic

@vercel

vercel Bot commented Aug 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
crm-agent Ready Ready Preview Aug 18, 2026 3:24pm
crm-api Ready Ready Preview Aug 18, 2026 3:24pm
crm-app Ready Ready Preview Aug 18, 2026 3:24pm

Request Review

@ripgrim ripgrim changed the title feat: page the enrichment queue beyond the first twenty rows feat: page the enrichment queue (CMP-92) Aug 18, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 10 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/api/test/enrichment-queue.spec.ts">

<violation number="1" location="apps/api/test/enrichment-queue.spec.ts:101">
P2: These three pagination tests don't actually exercise the limit. The `beforeAll` fixture inserts exactly one due task and one scheduled task, so `queue(1)`, `queue(10_000)` and `queue(0)` each return at most one row regardless of the service's `take` logic. Every assertion (`rows.length/scheduled.length` <= 1 or <= ENRICHMENT_PAGE_MAX) trivially passes even if `enrichment.queue` dropped its limit entirely, and the "nonsense limit as one row, never as none" test's `<= 1` assertion also passes if the implementation returns zero rows. Seed more than ENRICHMENT_PAGE_MAX due (and scheduled) rows so the cap and the "one row never none" behavior are actually verified against `total`/`scheduledTotal` rather than tautologically.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/api/test/enrichment-queue.spec.ts Outdated
expect(row?.line).toBe("Waiting");
});

it("hands back no more rows than the caller asked for", async () => {

@cubic-dev-ai cubic-dev-ai Bot Aug 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: These three pagination tests don't actually exercise the limit. The beforeAll fixture inserts exactly one due task and one scheduled task, so queue(1), queue(10_000) and queue(0) each return at most one row regardless of the service's take logic. Every assertion (rows.length/scheduled.length <= 1 or <= ENRICHMENT_PAGE_MAX) trivially passes even if enrichment.queue dropped its limit entirely, and the "nonsense limit as one row, never as none" test's <= 1 assertion also passes if the implementation returns zero rows. Seed more than ENRICHMENT_PAGE_MAX due (and scheduled) rows so the cap and the "one row never none" behavior are actually verified against total/scheduledTotal rather than tautologically.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/api/test/enrichment-queue.spec.ts, line 101:

<comment>These three pagination tests don't actually exercise the limit. The `beforeAll` fixture inserts exactly one due task and one scheduled task, so `queue(1)`, `queue(10_000)` and `queue(0)` each return at most one row regardless of the service's `take` logic. Every assertion (`rows.length/scheduled.length` <= 1 or <= ENRICHMENT_PAGE_MAX) trivially passes even if `enrichment.queue` dropped its limit entirely, and the "nonsense limit as one row, never as none" test's `<= 1` assertion also passes if the implementation returns zero rows. Seed more than ENRICHMENT_PAGE_MAX due (and scheduled) rows so the cap and the "one row never none" behavior are actually verified against `total`/`scheduledTotal` rather than tautologically.</comment>

<file context>
@@ -96,4 +97,25 @@ describe("what the enrichment widget reads", () => {
 		expect(row?.line).toBe("Waiting");
 	});
+
+	it("hands back no more rows than the caller asked for", async () => {
+		const queue = await enrichment.queue(1);
+
</file context>
Fix with cubic

Comment thread apps/app/components/enrichment-queue.tsx
Comment thread .github/workflows/pr-title.yml Outdated
Comment thread apps/app/components/enrichment-queue.tsx Outdated
@ripgrim
ripgrim merged commit 8c1abb1 into main Aug 18, 2026
9 checks passed
@ripgrim
ripgrim deleted the rg/enrichment-queue-paging branch August 18, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant