[SYNPY-1894] feat: synchronize active grid session - #1440
Conversation
| task = cls().fill_from_dict(synapse_response=task_dict) | ||
| yield task | ||
|
|
||
| @otel_trace_method( |
There was a problem hiding this comment.
There's integration test for grid_synchronize so I skipped adding integration test for this function synchronize_active_grid_session_async and only added a unit test.
| ``` | ||
|
|
||
| ### Step 4: Download record-based metadata as a local CSV | ||
| ### Step 4: Synchronize the grid session to pick up schema updates |
There was a problem hiding this comment.
I currently place the synchronization step right after session creation (Step 3), framed around picking up a schema change before starting to edit. There's also a valid case for calling synchronize after importing local CSV edits (Step 6). Please let me know if you would want me to move it. @cconrad8
There was a problem hiding this comment.
Pull request overview
Adds explicit synchronization support for active Grid sessions used by curation tasks, enabling already-open sessions to pick up newer RecordSet schema versions and extending synchronization to file-view-backed grids.
Changes:
- Introduces
SyncTypeand updatesGrid.synchronize()/synchronize_async()to accept a sync mode. - Adds
CurationTask.synchronize_active_grid_session_async()(and sync wrapper) to create/reuse the active session and delegate toGrid.synchronize. - Expands unit/integration tests and updates curator docs/guides to document the new behavior and recommended workflows.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/synapseclient/models/async/unit_test_curation_async.py | Adds unit tests for task-level active-session synchronization and SynchronizeGridRequest syncType serialization/validation. |
| tests/integration/synapseclient/models/async/test_grid_async.py | Updates/extends integration coverage for synchronizing entity-view and RecordSet-backed grid sessions. |
| synapseclient/models/curation.py | Adds SyncType, implements async task-level synchronization, and threads sync_type through grid sync requests. |
| synapseclient/models/init.py | Exports SyncType from the public synapseclient.models namespace. |
| docs/reference/experimental/sync/curator.md | Adds reference docs entry for synchronize_active_grid_session and SyncType. |
| docs/reference/experimental/async/curator.md | Adds reference docs entry for synchronize_active_grid_session_async and SyncType. |
| docs/guides/extensions/curator/metadata_contribution.md | Updates the contributor workflow guide to include a schema-refresh synchronization step and renumbers subsequent steps. |
Suppressed comments (2)
synapseclient/models/curation.py:3914
Grid.synchronize()is documented and tested with string values (e.g., "PULL_PUSH"), andSynchronizeGridRequest.sync_typeexplicitly allowsUnion[SyncType, str]. Update theGrid.synchronize()type hint to includestrso IDE/type-checking matches the supported API.
self,
*,
sync_type: Optional[SyncType] = None,
timeout: int = 120,
synapse_client: Optional[Synapse] = None,
synapseclient/models/curation.py:5051
Grid.synchronize_async()is used with string values in integration tests (e.g., sync_type="PULL_PUSH"), andSynchronizeGridRequest.sync_typeacceptsUnion[SyncType, str]. Update thesync_typeannotation here as well so type hints reflect the supported input types.
self,
*,
sync_type: Optional[SyncType] = None,
timeout: int = 120,
synapse_client: Optional[Synapse] = None,
) -> "Grid":
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| client = Synapse.get_client(synapse_client=synapse_client) | ||
|
|
||
| if not self.task_properties: | ||
| await self.get_async(synapse_client=synapse_client) | ||
|
|
||
| if isinstance(self.task_properties, FileBasedMetadataTaskProperties): |
| - Download the CSV (Step 5) as a local reference, make your edits locally, then copy-paste the values back into the Grid UI | ||
| - Make edits directly in the Synapse Grid UI — Step 3 prints the session URL (`https://www.synapse.org/Grid:default?sessionId=...`) after creating the session | ||
|
|
||
| **Use `synchronize()` instead of `export_to_record_set()`.** After editing in the Grid UI, push your changes back to the underlying files: |
There was a problem hiding this comment.
For discussing, should we deprecate the export to record set function and push people to use this?
There was a problem hiding this comment.
I actually noticed some problems if we remove export_to_record_set_async. As an example, the following is working as expected:
grid = Grid(session_id="MTM4Mzc0")
grid = await grid.import_csv_async(path="test_grid_import.csv")
await grid.export_to_record_set_async()
grid = await task.synchronize_active_grid_session_async(sync_type="PULL")
grid.download_csv(destination="/Users/lpeng/Downloads")
But if I remove export_to_record_set_async and change sync_type to PULL_PUSH, then I noticed that I can't actually "push" the changes from my CSV to the grid. I will ask other devs in curator-dev channel after Sage week.
Problem
A grid session captures the schema version of its bound RecordSet at creation time and never picks up a newer version on its own. Once a data manager registers and binds a new JSON schema version to a RecordSet, contributors' already-open grid sessions have no way to see the new columns.
The old
Grid.synchronize()only supports synchronizing recordset-based grid sessions.Solution
CurationTask.synchronize_active_grid_session()/_async()as the task-level "synchronize the tasks" function: it creates an active grid session first if the task doesn't have one yet, otherwise reuses the existing active session, then delegates to Grid.synchronize().Design decision
For FileBasedMetadataTaskProperties, there's only one meaningful synchronize behavior — PULL_PUSH — since file-based grids write annotations directly back to files with no intermediate versioned state to preview; there's nothing to choose.
For RecordBasedMetadataTaskProperties, the choice between PULL and PULL_PUSH is consequential: PULL_PUSH commits the grid's current state as a new RecordSet version, an action with a lasting, visible side effect for every other collaborator on that RecordSet, while PULL does not commit anything.
If it defaulted to
PULL_PUSH(which is the implicit default of the synapse server), users might be surprised by the JSON Schema version bump. If it defaulted toPULL, users might get an unexpected uncommitted preview when they meant to finalize all their changes and push them. Currently, the method requires the caller to explicitly state their intent and will raise an error if sync_type is not provided, rather than silently picking a default and letting the wrong choice surface later as a data surprise.