Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 26 additions & 14 deletions docs/guides/extensions/curator/metadata_contribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This guide shows how to programmatically complete a record-based metadata curati
By following this guide, you will:

- List curation tasks in a Synapse project
- Create a Grid session for a record-based curation task
- Get or create a Grid session for a record-based curation task
- Download metadata from the Grid to a local CSV
- Edit the metadata locally
- Upload the metadata back into the Grid
Expand Down Expand Up @@ -58,9 +58,9 @@ for task in all_tasks:
curation_task = all_tasks[0]
```

#### Option B: Filter the list by assignee, state, or name
#### Option B: Filter the list by assignee, state, or data type

Use this when you want to find tasks assigned to you, tasks in a specific state, or locate a task by name. Each filter still returns a list — pick the one you want from it.
Use this when you want to find tasks assigned to you, tasks in a specific state, or locate a task by its data type. Each filter still returns a list — pick the one you want from it.

```python
from synapseclient.models import CurationTask
Expand All @@ -78,12 +78,12 @@ in_progress_tasks = list(
CurationTask.list(project_id=PROJECT_ID, state_filter=["IN_PROGRESS"])
)

# Find a task by name (list() does not support name filtering directly — filter after listing)
target_name = "AnimalMetadata_Curation"
named_tasks = [
# Find a task by data type (list() does not support data type filtering directly — filter after listing)
target_data_type = "AnimalMetadata"
matching_tasks = [
task
for task in CurationTask.list(project_id=PROJECT_ID)
if task.name == target_name
if task.data_type == target_data_type
]

# Select the task you want from whichever list you built above
Expand All @@ -100,12 +100,16 @@ from synapseclient.models import CurationTask
curation_task = CurationTask(task_id=12345).get()
```

### Step 3: Create a Grid session for the task
### Step 3: Get or create a Grid session for the task

Each option in Step 2 leaves you with a single `curation_task`. Start a new Grid session on it — it picks the `record_set_id` from the task properties automatically and links the session back to the task. If the task already has an active Grid session linked, calling this replaces the link with the new session.
Each option in Step 2 leaves you with a single `curation_task`. Open its Grid with [get_or_create_curator_grid][synapseclient.extensions.curator.get_or_create_curator_grid]. Given the task's ID, it returns the Grid already linked to the task, or — if none is linked yet — creates a new Grid session, links it to the task, and returns it. The first call starts the session; every subsequent call returns that same session, so you can pick up where you left off without accidentally starting over.

The `record_set_id` and the authorization mode are taken from the task properties automatically, so you do not need to specify them here.

```python
latest_grid = curation_task.create_grid_session()
from synapseclient.extensions.curator import get_or_create_curator_grid

latest_grid = get_or_create_curator_grid(task_id=curation_task.task_id)
```

### Step 4: Download record-based metadata as a local CSV
Expand Down Expand Up @@ -230,7 +234,7 @@ Row 2:

#### Fix and re-export

If any rows are invalid, recreate a Grid session against the RecordSet (see Step 3), correct the offending rows, and re-run Steps 4–6 to re-export. The validation report is regenerated on each export, so iterate until the report is clean before letting the administrator know your task is ready.
If any rows are invalid, re-open the Grid session (see Step 3 — `get_or_create_curator_grid` returns the session already linked to the task), correct the offending rows, and re-run Steps 4–6 to re-export. The validation report is regenerated on each export, so iterate until the report is clean before letting the administrator know your task is ready.

> **If get_detailed_validation_results returns None after exporting:** check that record_set.validation_file_handle_id is set after the re-fetch. If it isn't, the export did not complete — re-run export_to_record_set() on an active Grid session against the same RecordSet.

Expand All @@ -249,7 +253,7 @@ File-based tasks follow the same overall flow as record-based tasks (Steps 1–8
**No CSV import.** `import_csv` is not currently supported for file-based grids. Instead, you can either:

- Download the CSV (Step 4) as a local reference, make your edits locally, then copy-paste the values back into the Grid UI
- Make edits directly in the Synapse Grid UI — Step 3 prints the session URL (`https://www.synapse.org/Grid:default?sessionId=...`) after creating the session
- Make edits directly in the Synapse Grid UI — build the session URL from the grid returned in Step 3: `https://www.synapse.org/Grid:default?sessionId={latest_grid.session_id}`

**Use `synchronize()` instead of `export_to_record_set()`.** After editing in the Grid UI, push your changes back to the underlying files:

Expand All @@ -266,10 +270,17 @@ This writes the Grid annotation values back to each file as Synapse annotations.
### Cleaning up a Grid session

```python
# Delete the grid session

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think it's very easy to forget unlinking the grid and the task. What about creating something like CurationTask.delete_active_grid_session_async() that deletes the Grid and clears the link in one coordinated call in the future?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cconrad8 What do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative is to make get_or_create_curator_grid a context manager that handled this automatically with a boolean flag to not clean up.

That way in the finally you can handle this for the user (The following is not correct code, but just mocking up what it roughly looks like).

try:
    yield latest_grid
finally:
    if cleanup:
        latest_grid.delete()
        
        # Remove the deleted session's reference from the task, so the task no longer
        # points at a session that no longer exists
        status = CurationTask(task_id).get_status()
        status.execution_details = None
        curation_task.update_status(curation_task_status=status)

latest_grid.delete()

# Remove the deleted session's reference from the task, so the task no longer
# points at a session that no longer exists
status = curation_task.get_status()
status.execution_details = None
curation_task.update_status(curation_task_status=status)
```

Deleting is permanent — you can no longer re-export from this session. If you spot more issues later, create a new Grid session via Step 3.
Deleting is permanent — you can no longer re-export from this session. If you spot more issues later, get a fresh Grid session via Step 3: `get_or_create_curator_grid` creates a new one and links it to the task.

## References

Expand All @@ -278,7 +289,8 @@ Deleting is permanent — you can no longer re-export from this session. If you
<!-- markdownlint-disable MD052 -->
- [CurationTask.list][synapseclient.models.CurationTask.list] - List curation tasks in a project
- [CurationTask.get][synapseclient.models.CurationTask.get] - Fetch a CurationTask by id
- [CurationTask.create_grid_session][synapseclient.models.CurationTask.create_grid_session] - Create a Grid session for a CurationTask and link it to the task status
- [get_or_create_curator_grid][synapseclient.extensions.curator.get_or_create_curator_grid] - Get the Grid attached to a CurationTask, creating and linking one if needed
- [CurationTask.create_grid_session][synapseclient.models.CurationTask.create_grid_session] - Always create a new Grid session for a CurationTask and link it to the task status
- [CurationTask.set_task_state][synapseclient.models.CurationTask.set_task_state] - Set the state on a CurationTask's status
- [Grid.download_csv][synapseclient.models.Grid.download_csv] - Download Grid contents as a local CSV
- [Grid.import_csv][synapseclient.models.Grid.import_csv] - Upsert CSV edits back into a Grid session (record-based grids only)
Expand Down
1 change: 1 addition & 0 deletions docs/reference/experimental/async/curator.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@
inherited_members: true
members:
- create_async
- get_async
- export_to_record_set_async
- synchronize_async
- download_csv_async
Expand Down
1 change: 1 addition & 0 deletions docs/reference/experimental/sync/curator.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@
inherited_members: true
members:
- create
- get
- export_to_record_set
- synchronize
- download_csv
Expand Down
2 changes: 2 additions & 0 deletions synapseclient/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
delete_grid_session,
get_curation_task,
get_curation_task_status,
get_grid_session,
list_curation_tasks,
list_grid_sessions,
update_curation_task,
Expand Down Expand Up @@ -335,6 +336,7 @@
"delete_grid_session",
"get_curation_task",
"get_curation_task_status",
"get_grid_session",
"list_curation_tasks",
"list_grid_sessions",
"update_curation_task",
Expand Down
27 changes: 27 additions & 0 deletions synapseclient/api/curation_services.py
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,33 @@ async def list_grid_sessions(
yield item


async def get_grid_session(
session_id: str,
*,
synapse_client: Optional["Synapse"] = None,
) -> dict[str, Any]:
"""
Get a single grid session by its session id.

https://rest-docs.synapse.org/rest/GET/grid/session/sessionId.html

Arguments:
session_id: The unique identifier of the grid session to get.
synapse_client: If not passed in and caching was not disabled by
`Synapse.allow_client_caching(False)` this will use the last created
instance from the Synapse class constructor.

Returns:
The GridSession as a dictionary.
"""
from synapseclient import Synapse

client = Synapse.get_client(synapse_client=synapse_client)

response = await client.rest_get_async(uri=f"/grid/session/{session_id}")
return response


async def delete_grid_session(
session_id: str,
*,
Expand Down
6 changes: 6 additions & 0 deletions synapseclient/extensions/curator/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
"""

from .file_based_metadata_task import create_file_based_metadata_task
from .get_or_create_grid import (
get_or_create_curator_grid,
get_or_create_curator_grid_async,
)
from .record_based_metadata_task import create_record_based_metadata_task
from .schema_generation import generate_jsonld, generate_jsonschema
from .schema_management import (
Expand All @@ -16,6 +20,8 @@
from .schema_registry import query_schema_registry

__all__ = [
"get_or_create_curator_grid",
"get_or_create_curator_grid_async",
"create_file_based_metadata_task",
"create_record_based_metadata_task",
"query_schema_registry",
Expand Down
Loading
Loading