[SYNPY-1880] feat: added per row validation to Grid data class by linglp · Pull Request #1428 · Sage-Bionetworks/synapsePythonClient

linglp · 2026-07-17T02:04:06Z

Problem:

Add the ability to query rows in a Synapse Grid session and retrieve row level validation results

Solution:

All dataclasses added:
Please note that some data classes are renamed to avoid name collision with the data classes in table (i.e. Query -> GridQuery)
GridQuery Query
QueryRequest QueryRequest
GridQueryJobRequest GridQueryJobRequest / GridQueryJobResponse
GridQueryResult QueryResult
GridRow Row
SelectColumn SelectColumn
GridQueryValidationResult ValidationResults
SelectItem SelectItem
SelectByName SelectByName
SelectAll SelectAll
CountStar CountStar
SelectSelection SelectSelection
Filter Filter
RowValidationResultFilter RowValidationResultFilter
CellValueFilter CellValueFilter
RowSelectionFilter RowSelectionFilter
RowIsValidFilter RowIsValidFilter
RowIdFilter RowIdFilter
ValidationOperator ValidationOperator
CellValueOperator CellValueOperator
GridReplica GridReplica
CreateReplicaRequest CreateReplicaRequest

Replica support

Added the create_grid_replica API call (api/curation_services.py, POST /grid/session/{sessionId}/replica)
Added GridReplica and CreateReplicaRequest dataclasses
Added Grid.create_replica_async/create_replica methods

Grid.validate_rows_async / validate_rows
New method that submits a QueryRequest against a grid session and returns per-row validation results

Grid.connect_async / connect

Added as an async context manager (@asynccontextmanager) and its true sync counterpart
Creates the grid session and one replica up front, binds the replica to the Grid instance for the duration of the with/async with block, and clears it on exit

…reate replica

…; add them to documentation; add GridQueryValidationResult data class; add a lot of examples to validate_rows

linglp · 2026-07-17T02:14:59Z

                `Synapse.allow_client_caching(False)` this will use the last created
                instance from the Synapse class constructor.

        Returns:


All these changes are confusing. I did not delete these and they are still there if you use VS code to see.

linglp · 2026-07-17T02:15:05Z

            syn = Synapse()
            syn.login()

-            # Export modified grid data back to the record set


All these changes are confusing. I did not delete these and they are still there if you use VS code to see.

linglp · 2026-07-17T02:15:31Z

-        return self
+        return None

-    def synchronize(


Same above. The function is still there.

linglp · 2026-07-17T15:51:33Z

+
+    @skip_async_to_sync
+    @asynccontextmanager
+    async def connect_async(


Based on John Hill's suggestion here, we will need to use connect function to interact with the replica.

linglp · 2026-07-17T15:54:41Z

+            timeout=timeout, synapse_client=synapse_client
+        )
+
+        if not request.query_result or not request.query_result.rows:


Initially I raised it as an error but eventually decided that here it should be a warning. It could be that there's no row to validate or the Filter or SelectColumn didn't return anything.

The typing isn't quite right then. It says it returns a GridQueryResult, but if GridQueryJobRequest.query_result is None, that is what will be returned. All the examples assume it will not be None as well. I would strongly consider raising an exception here.

linglp · 2026-07-17T15:55:08Z

+    @otel_trace_method(
+        method_to_trace_name=lambda self, **kwargs: f"Grid_Validate_Rows_Session_ID: {self.session_id}"
+    )
+    async def validate_rows_async(


This is also important to review. Examples in the docstring demonstrated how you could use this function with connect.

Copilot

Pull request overview

Adds structured Grid query support to the experimental curator models so callers can query a Grid session and retrieve per-row JSON-schema validation results, including support for creating/binding Grid replicas via a new API endpoint and convenience connect context managers.

Changes:

Added new Grid query/validation dataclasses (SelectItem/Filter hierarchies, QueryRequest, query result models) and a new GridQueryJobRequest async job type.
Added Grid replica support end-to-end: new REST API call, replica/request models, and Grid.create_replica_async plus Grid.connect_async/connect to bind a replica for a session scope.
Added async unit tests and updated experimental reference docs to surface the new APIs.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/unit/synapseclient/models/async/unit_test_curation_async.py	Adds unit tests covering new Grid query/validation dataclasses, replica creation, connect context manager, and row validation flow.
synapseclient/models/mixins/asynchronous_job.py	Registers the async job endpoint mapping for `GridQueryJobRequest`.
synapseclient/models/curation.py	Implements the new Grid query/validation dataclasses, replica workflow, connect context managers, and `validate_rows_async`.
synapseclient/models/init.py	Re-exports newly added curator models/enums for the public `synapseclient.models` namespace.
synapseclient/core/constants/concrete_types.py	Adds concreteType constants for the new Grid query and filter/select item types.
synapseclient/api/curation_services.py	Adds `create_grid_replica` REST API wrapper (POST `/grid/session/{sessionId}/replica`).
synapseclient/api/init.py	Exposes the new `create_grid_replica` API function via the API package.
docs/reference/experimental/sync/curator.md	Updates experimental sync reference to include new Grid methods and types.
docs/reference/experimental/async/curator.md	Updates experimental async reference to include new Grid methods and types.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        grid_replica = await create_grid_replica(
+            session_id=self.session_id,
+            create_replica_request=CreateReplicaRequest(
+                self.session_id
+            ).to_synapse_request(),
+            synapse_client=synapse_client,
+        )
+        replica_data = grid_replica.get("replica")
+        if not replica_data:
+            raise ValueError(
+                f"Replica could not be created for grid session '{self.session_id}': "
+                f"no replica was returned in the Synapse response."
+            )
+        return GridReplica().fill_from_dict(replica_data)


andrewelamb · 2026-07-17T17:33:53Z

+                synapse_client=synapse_client,
+            )
+
+        replica = self.create_replica(synapse_client=synapse_client)


Shouldn't this be checking if a replica already exists for this session before creating a new one?

When I looked at the front-end code here, they always create a brand-new replica regardless of whether the session is new or pre-existing, and they also don't check if the replicas exist for this session or not. I think here we should match the front-end behavior.

linglp · 2026-07-17T18:31:07Z

@andrewelamb Per Nick's comment here, I think maybe there's no harsh enforcement of the replica limit? But based on my understanding of John Hill's comment here, we shouldn't create replica IDs wastefully (i.e. per function call). There's no way from the client side at least to "delete" a replica, and every replica created just becomes permanent part of the document (see his comment here)

To answer your second question: the idea here is that we reuse one replica for the lifetime of one with block — every validate_rows/validate_rows_async call made inside that block shares the same replica ID. When John said that the replica ID should be re-used in a session here, I think he meant within one with block, and not separate scripts or the whole Python process.

…test

BryanFauble

Let's look to add a few end-to-end integration tests, the rest of the logic looks good to me!

andrewelamb · 2026-07-23T20:35:38Z

+            timeout=timeout, synapse_client=synapse_client
+        )
+
+        if not request.query_result or not request.query_result.rows:


The typing isn't quite right then. It says it returns a GridQueryResult, but if GridQueryJobRequest.query_result is None, that is what will be returned. All the examples assume it will not be None as well. I would strongly consider raising an exception here.

andrewelamb · 2026-07-23T20:39:38Z

+
+    Attributes:
+        grid_session_id: The ID of the grid session.
+        replica: Information about a replica. Populated after calling the


This doesn't seem to be a real attribute?

andrewelamb · 2026-07-23T20:42:23Z

+        A grid replica is an in-memory document that represents a 'copy' of the
+        grid. Each replica is identified by a unique replicaId, issued by the
+        'hub'. A user can have more than one replica at a time (i.e. using
+        multiple browser tabs/machines). A user is limited to 10 replicas


I thought we determined this limit didn't exist?

andrewelamb · 2026-07-23T20:42:52Z

+        A grid replica is an in-memory document that represents a 'copy' of the
+        grid. Each replica is identified by a unique replicaId, issued by the
+        'hub'. A user can have more than one replica at a time (i.e. using
+        multiple browser tabs/machines). A user is limited to 10 replicas


see above comment

andrewelamb · 2026-07-23T20:43:33Z

+    https://rest-docs.synapse.org/rest/POST/grid/session/sessionId/replica.html
+
+    Note: Only the user that started the grid session may create a replica.
+    A user is limited to 10 replicas per-hour per-grid-session.


I thought we determined this limit doesn't exist?

BryanFauble

Thanks for the updates! Nothing additional beyond what has already been flagged.

linglp added 10 commits July 16, 2026 21:59

add all dataclasses and tests

8f98d13

fix docstring

78ec656

added create replica endpoint, dataclasses, updated documentation

879a2c8

add test for create replica data classes and add add a unit test of c…

b7d80dd

…reate replica

if nothing returned, then it should raise error

df9d8bd

raise an error if query_request is none when querying the grid

bdf168c

update documentation; export filter and column selection data classes…

9ff1bad

…; add them to documentation; add GridQueryValidationResult data class; add a lot of examples to validate_rows

add the connect method

3f6f584

add connect to documentation

146ac3c

restore thing

050a448

linglp commented Jul 17, 2026

View reviewed changes

linglp added 3 commits July 17, 2026 11:27

when no validation result returned, it should not raise an error

16ac436

added otel trace method

224ddd4

update docstring

a5ea43d

linglp commented Jul 17, 2026

View reviewed changes

add test

8fb46ce

linglp marked this pull request as ready for review July 17, 2026 15:57

linglp requested a review from a team as a code owner July 17, 2026 15:57

Copilot AI review requested due to automatic review settings July 17, 2026 15:57

Copilot started reviewing on behalf of linglp July 17, 2026 15:57 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

linglp commented Jul 17, 2026

View reviewed changes

Comment thread synapseclient/models/curation.py Outdated

make sure that GridQuery does not allow empty column selection

6449b72

BryanFauble reviewed Jul 17, 2026

View reviewed changes

Comment thread synapseclient/models/__init__.py Outdated

andrewelamb reviewed Jul 17, 2026

View reviewed changes

Comment thread synapseclient/models/curation.py

linglp added 8 commits July 20, 2026 10:52

make create replica a private method

d07551b

update docstring

028b57a

remove fill_from_dict in selectItem classes and filter classes; edit …

cf7b2d4

…test

update import; docstring example; and documentation

67a045f

Merge branch 'develop' into SYNPY-1880-clean

efacbfe

update to reference the grid previously created

17cb35e

query result should be returned instead of none when there is no row

3b81365

add to grid class docstring

b853a52

BryanFauble reviewed Jul 21, 2026

View reviewed changes

Comment thread synapseclient/models/curation.py

BryanFauble reviewed Jul 21, 2026

View reviewed changes

Comment thread synapseclient/models/curation.py Outdated

BryanFauble reviewed Jul 21, 2026

View reviewed changes

Comment thread synapseclient/models/curation.py

BryanFauble requested changes Jul 21, 2026

View reviewed changes

linglp added 5 commits July 21, 2026 16:59

update docstring example to avoid recreating another grid instance

2c81990

added integration tests

990fac7

update docstring

740c76d

add import back

056e25c

add comment

8518dce

linglp requested a review from BryanFauble July 22, 2026 18:51

Merge branch 'develop' into SYNPY-1880-clean

ee55485

linglp requested a review from andrewelamb July 23, 2026 17:23

andrewelamb requested changes Jul 23, 2026

View reviewed changes

BryanFauble approved these changes Jul 23, 2026

View reviewed changes

Uh oh!

Conversation

linglp commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem:

Solution:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

linglp Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

linglp commented Jul 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BryanFauble left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BryanFauble left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

linglp commented Jul 17, 2026 •

edited

Loading

linglp Jul 17, 2026 •

edited

Loading