Skip to content

[SYNPY-1894] feat: synchronize active grid session - #1440

Open
linglp wants to merge 8 commits into
developfrom
SYNPY-1894
Open

[SYNPY-1894] feat: synchronize active grid session #1440
linglp wants to merge 8 commits into
developfrom
SYNPY-1894

Conversation

@linglp

@linglp linglp commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Problem

A grid session captures the schema version of its bound RecordSet at creation time and never picks up a newer version on its own. Once a data manager registers and binds a new JSON schema version to a RecordSet, contributors' already-open grid sessions have no way to see the new columns.

The old Grid.synchronize() only supports synchronizing recordset-based grid sessions.

Solution

  • Relaxed Grid.synchronize()/synchronize_async() to support both recordset-based grids and file-view based grid.
  • Added CurationTask.synchronize_active_grid_session()/_async() as the task-level "synchronize the tasks" function: it creates an active grid session first if the task doesn't have one yet, otherwise reuses the existing active session, then delegates to Grid.synchronize().

Design decision

For FileBasedMetadataTaskProperties, there's only one meaningful synchronize behavior — PULL_PUSH — since file-based grids write annotations directly back to files with no intermediate versioned state to preview; there's nothing to choose.

For RecordBasedMetadataTaskProperties, the choice between PULL and PULL_PUSH is consequential: PULL_PUSH commits the grid's current state as a new RecordSet version, an action with a lasting, visible side effect for every other collaborator on that RecordSet, while PULL does not commit anything.

If it defaulted to PULL_PUSH (which is the implicit default of the synapse server), users might be surprised by the JSON Schema version bump. If it defaulted to PULL, users might get an unexpected uncommitted preview when they meant to finalize all their changes and push them. Currently, the method requires the caller to explicitly state their intent and will raise an error if sync_type is not provided, rather than silently picking a default and letting the wrong choice surface later as a data surprise.

@linglp linglp changed the title [SYNPY-1894] [SYNPY-1894] feat: synchronize active grid session Aug 5, 2026
task = cls().fill_from_dict(synapse_response=task_dict)
yield task

@otel_trace_method(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's integration test for grid_synchronize so I skipped adding integration test for this function synchronize_active_grid_session_async and only added a unit test.

```

### Step 4: Download record-based metadata as a local CSV
### Step 4: Synchronize the grid session to pick up schema updates

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I currently place the synchronization step right after session creation (Step 3), framed around picking up a schema change before starting to edit. There's also a valid case for calling synchronize after importing local CSV edits (Step 6). Please let me know if you would want me to move it. @cconrad8

@linglp
linglp marked this pull request as ready for review August 6, 2026 05:27
Copilot AI lite review requested due to automatic review settings August 6, 2026 05:27
@linglp
linglp requested a review from a team as a code owner August 6, 2026 05:27
@linglp
linglp requested a review from cconrad8 August 6, 2026 05:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds explicit synchronization support for active Grid sessions used by curation tasks, enabling already-open sessions to pick up newer RecordSet schema versions and extending synchronization to file-view-backed grids.

Changes:

  • Introduces SyncType and updates Grid.synchronize() / synchronize_async() to accept a sync mode.
  • Adds CurationTask.synchronize_active_grid_session_async() (and sync wrapper) to create/reuse the active session and delegate to Grid.synchronize.
  • Expands unit/integration tests and updates curator docs/guides to document the new behavior and recommended workflows.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit/synapseclient/models/async/unit_test_curation_async.py Adds unit tests for task-level active-session synchronization and SynchronizeGridRequest syncType serialization/validation.
tests/integration/synapseclient/models/async/test_grid_async.py Updates/extends integration coverage for synchronizing entity-view and RecordSet-backed grid sessions.
synapseclient/models/curation.py Adds SyncType, implements async task-level synchronization, and threads sync_type through grid sync requests.
synapseclient/models/init.py Exports SyncType from the public synapseclient.models namespace.
docs/reference/experimental/sync/curator.md Adds reference docs entry for synchronize_active_grid_session and SyncType.
docs/reference/experimental/async/curator.md Adds reference docs entry for synchronize_active_grid_session_async and SyncType.
docs/guides/extensions/curator/metadata_contribution.md Updates the contributor workflow guide to include a schema-refresh synchronization step and renumbers subsequent steps.
Suppressed comments (2)

synapseclient/models/curation.py:3914

  • Grid.synchronize() is documented and tested with string values (e.g., "PULL_PUSH"), and SynchronizeGridRequest.sync_type explicitly allows Union[SyncType, str]. Update the Grid.synchronize() type hint to include str so IDE/type-checking matches the supported API.
        self,
        *,
        sync_type: Optional[SyncType] = None,
        timeout: int = 120,
        synapse_client: Optional[Synapse] = None,

synapseclient/models/curation.py:5051

  • Grid.synchronize_async() is used with string values in integration tests (e.g., sync_type="PULL_PUSH"), and SynchronizeGridRequest.sync_type accepts Union[SyncType, str]. Update the sync_type annotation here as well so type hints reflect the supported input types.
        self,
        *,
        sync_type: Optional[SyncType] = None,
        timeout: int = 120,
        synapse_client: Optional[Synapse] = None,
    ) -> "Grid":

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +2284 to +2289
client = Synapse.get_client(synapse_client=synapse_client)

if not self.task_properties:
await self.get_async(synapse_client=synapse_client)

if isinstance(self.task_properties, FileBasedMetadataTaskProperties):

@thomasyu888 thomasyu888 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 I'll defer to the team on the fine grained technical review after sage week but on a high level, this looks great - just had a tiny comment.

- Download the CSV (Step 5) as a local reference, make your edits locally, then copy-paste the values back into the Grid UI
- Make edits directly in the Synapse Grid UI — Step 3 prints the session URL (`https://www.synapse.org/Grid:default?sessionId=...`) after creating the session

**Use `synchronize()` instead of `export_to_record_set()`.** After editing in the Grid UI, push your changes back to the underlying files:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For discussing, should we deprecate the export to record set function and push people to use this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually noticed some problems if we remove export_to_record_set_async. As an example, the following is working as expected:

grid = Grid(session_id="MTM4Mzc0")
grid = await grid.import_csv_async(path="test_grid_import.csv")
await grid.export_to_record_set_async()
grid = await task.synchronize_active_grid_session_async(sync_type="PULL")
grid.download_csv(destination="/Users/lpeng/Downloads")

But if I remove export_to_record_set_async and change sync_type to PULL_PUSH, then I noticed that I can't actually "push" the changes from my CSV to the grid. I will ask other devs in curator-dev channel after Sage week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants