feat(data modeling): merge records sync and sync_resume into a chunk-yielding generator

[evertoncolling]

Description

client.data_modeling.records.sync() and sync_resume() never returned whenever a page held fewer records than limit.

RecordsAPI._sync delegated to the generic _list paging helper, whose loop terminates on:

# cognite/client/_api_client.py:328
if total_retrieved == limit or next_cursor is None:
    break

Neither condition can be reached for this endpoint:

• The sync endpoint always returns a nextCursor — that is precisely what makes the change feed resumable. It signals end-of-feed with hasNext: false, which the paging loop never read.
• So the loop could only exit when the feed happened to return exactly as many records as were asked for.

The new API

Following the review discussion, the fix grew into a redesign: sync and sync_resume are merged into a single generator method that issues one POST per chunk and yields each SyncRecordList as it arrives, until the feed is exhausted (hasNext: false). Nothing is accumulated in memory.

# Start from a relative time, iterate the whole feed:
for chunk in client.data_modeling.records.sync(stream_id="my-stream", initialize_cursor="7d-ago"):
    process(chunk)
    last_cursor = chunk.cursor  # persist after processing each chunk

# Later, resume from the stored cursor:
for chunk in client.data_modeling.records.sync(stream_id="my-stream", cursor=last_cursor):
    ...

# Manual single-chunk control:
chunk = next(client.data_modeling.records.sync(stream_id="my-stream", cursor=last_cursor))

• Exactly one of initialize_cursor / cursor is required; @overload stubs enforce this at type-check time (a lazy runtime ValueError backstops untyped callers).
• chunk_size (1–1000, default 1000) replaces limit as the per-request page size. It is deliberately not named limit: SDK-wide, limit means "total items, None fetches everything", while chunk_size is the established name for per-yielded-chunk size (see the __call__ iterators). There is no chunk_size=None single-record mode since each chunk carries the cursor needed to resume.
• The generator terminates at end-of-feed rather than polling forever — the endpoint has no server-side poll timeout (unlike datapoint subscriptions), so tailing is done by calling sync(cursor=...) again later.
• The sync client gets the equivalent Iterator[SyncRecordList] via codegen ("sync" added to SYNC_METHODS_TO_KEEP because mypy requires async-generator overload stubs to be plain def).

Breaking changes

Records is an alpha feature behind FeaturePreviewWarning.

• sync_resume() is removed — pass cursor= to sync() instead.
• sync() now returns a generator of SyncRecordList chunks instead of a single list.
• The limit parameter is replaced by chunk_size; the unlimited (limit=None) accumulate-everything mode is gone — iterate the generator instead.

Checklist:

• Tests added/updated.
• Documentation updated. Documentation is generated from docstrings - these must be updated according to your change.
  If a new method has been added it should be referenced in cognite.rst in order to generate docs based on its docstring.
• The PR title follows the Conventional Commit spec.

[gemini-code-assist]

Code Review

This pull request refactors the record synchronization logic in _sync to use a direct POST request instead of the generic paging helper, preventing infinite loops when receiving partial pages. It also adds comprehensive integration and unit tests for the sync functionality. The reviewer suggests adding an explicit check to ensure that initialize_cursor and cursor are not provided simultaneously, as they are mutually exclusive parameters.

[codecov]

Codecov Report

❌ Patch coverage is 99.09091% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.80%. Comparing base (2ea6d56) to head (87b639b).

Files with missing lines	Patch %	Lines
...ration/test_api/test_data_modeling/test_records.py	98.07%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2755      +/-   ##
==========================================
- Coverage   93.81%   93.80%   -0.01%     
==========================================
  Files         513      514       +1     
  Lines       52620    52806     +186     
==========================================
+ Hits        49364    49536     +172     
- Misses       3256     3270      +14     

Files with missing lines	Coverage Δ
cognite/client/_api/data_modeling/records.py	98.96% <100.00%> (+1.43%)	⬆️
cognite/client/_sync_api/data_modeling/records.py	100.00% <100.00%> (ø)
...gnite/client/data_classes/data_modeling/records.py	98.22% <ø> (ø)
...s_unit/test_api/test_data_modeling/test_records.py	100.00% <100.00%> (ø)
...ration/test_api/test_data_modeling/test_records.py	98.07% <98.07%> (ø)

... and 10 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[andersfylling]

Left two comments to make the test suite more robust. @asosnovski and I were just talking about the sync endpoint so this was a nice change to see!

[haakonvt]

Great fix, I fully agree to move away from the existing helper method 👌 Some comments:

[haakonvt]

The TimeRange class should be taught how to deal with relative time strings 🚀

[evertoncolling]

That's fair! I was discussing something around time anchors in the API like what data points support ("1h-ago"), but until that lands we could handle something like this in the SDK. That would be a future dedicated PR though.

[andersfylling]

should we consider a generator to reduce memory?

[haakonvt]

As in continuously yielding chunks back to the user? I’m all in favor

[evertoncolling]

Then maybe we should add it via a separate method that has this pattern, no? We can refactor the existing method to sync a single page (either start the sync or resume) with manual control and add a dedicated method that exhausts the cursor until hasNext is false with the ability to yield chunks. Kind of like iterate_data in datapoints. WDYT?

Also, since we are changing a lot here, we could also merge sync and sync_resume as suggested. I guess we can do some overload so a single method accepts the two signatures.

[evertoncolling]

Or just a single "generator" method that handles everything? Both manual pagination control by yeidling one page at the time and iterating over the whole dataset?

[evertoncolling]

Ended up testing out the latter :)