[SYNPY-1851] Add high level function for creating grids#1423
[SYNPY-1851] Add high level function for creating grids#1423andrewelamb wants to merge 22 commits into
Conversation
linglp
left a comment
There was a problem hiding this comment.
LGTM! I have small design questions but overall it looks good. Thanks for the fixes.
| ### Cleaning up a Grid session | ||
|
|
||
| ```python | ||
| # Delete the grid session |
There was a problem hiding this comment.
Nit: I think it's very easy to forget unlinking the grid and the task. What about creating something like CurationTask.delete_active_grid_session_async() that deletes the Grid and clears the link in one coordinated call in the future?
There was a problem hiding this comment.
Alternative is to make get_or_create_curator_grid a context manager that handled this automatically with a boolean flag to not clean up.
That way in the finally you can handle this for the user (The following is not correct code, but just mocking up what it roughly looks like).
try:
yield latest_grid
finally:
if cleanup:
latest_grid.delete()
# Remove the deleted session's reference from the task, so the task no longer
# points at a session that no longer exists
status = CurationTask(task_id).get_status()
status.execution_details = None
curation_task.update_status(curation_task_status=status)
BryanFauble
left a comment
There was a problem hiding this comment.
There are some changes to make before merging, approving now though.
Problem:
A data contributor assigned a
CurationTaskhad no simple, idempotent way to open the Grid for that task. The only option wasCurationTask.create_grid_session(), which always creates a brand-new session and re-links it to the task, replacing any session already linked. This made it easy to accidentally start over:session_idalone — theGridmodel could be created but not retrieved from Synapse.Affected code:
synapseclient/extensions/curator/,synapseclient/models/curation.py(Grid model), andsynapseclient/api/curation_services.py.Solution:
Add a high-level, get-or-create entry point plus the lower-level pieces it needs:
synapseclient.extensions.curator.get_curator_grid(task_id, ...)— the primary addition. It:CurationTask.The first call starts a session; subsequent calls return the same one, so contributors can pick up where they left off without accidentally starting over.
record_set_idand authorization mode are taken from the task properties automatically.Grid.get()/Grid.get_async()— retrieve a Grid session from Synapse by itssession_idand populate the model viafill_from_dict().get_grid_session()in the API layer — wrapsGET /grid/session/{sessionId}.Non-grid execution details on the task raise a
ValueError; a 404 on the linked session is handled by recreating, while any other HTTP error propagates. Docs (metadata_contribution.mdand the async/sync curator reference) are updated to steer contributors toget_curator_grid, and the cleanup section now shows how to clear the task's stale session reference afterdelete().Testing:
tests/unit/synapseclient/extensions/unit_test_get_curator_grid.py) covering every branch ofget_curator_grid: create-when-no-session, get-existing-by-id, recreate-on-404, non-404 error propagation, general error propagation, create-when-execution-details-None, and raise-on-non-grid-execution-details.tests/integration/.../test_grid_async.py::test_get_grid_session_async) creating a Grid from a record set and re-fetching it via a freshGridinstance, asserting the round-tripped fields match.