Summary
The DataMaker API now exposes a persistent, team/project-scoped Sets resource (/sets CRUD), but the Python SDK has no way to read or write it. A Set is a durable snapshot of generated/fetched rows (a regression dataset, shared fixture, or audit-ready export). The web UI's empty state and the Set model docstring both advertise dm.save_set() as the scenario-facing entry point — but that method does not exist yet. This issue tracks adding a SetsClient route module plus delegated DataMaker methods, mirroring the existing TemplatesClient pattern.
Motivation
Scenarios run inside datamaker-runner and talk to the API through this SDK (DATAMAKER_API_KEY / DATAMAKER_API_URL). Users are told they can persist a regression set from a scenario via dm.save_set(...), but there is no SDK surface for it, so the documented flow dead-ends. Sets need first-class SDK methods like every other entity (templates, connections, projects).
API surface to wrap
The API (datamaker-api) exposes these routes, all authMiddleware + scopeFilter scoped to the caller's team/project:
| Method |
Path |
Notes |
GET |
/sets?projectId= |
List sets for the active project scope |
GET |
/sets/:id |
Fetch one |
POST |
/sets |
Create. Body: { name, description?, data?, rowCount?, projectId? }. rowCount is derived from data (array length) when omitted; teamId/projectId are resolved server-side from the validated scope. |
PATCH |
/sets/:id |
Partial update (name?, description?, data?, rowCount?) |
DELETE |
/sets/:id |
Delete |
data is arbitrary JSON; the common case is an array of row objects.
Proposed implementation (mirror routes/templates.py)
1. New route module src/datamaker/routes/sets.py
class SetsClient(BaseClient):
"""Client for saved Set operations."""
def get_sets(self, project_id: str | None = None) -> List[Dict]: ...
def get_set(self, set_id: str) -> Dict: ...
def create_set(self, name, data=None, description=None, row_count=None, project_id=None) -> Dict: ...
def update_set(self, set_id: str, set_data: Dict) -> Dict: ...
def delete_set(self, set_id: str) -> Dict: ...
2. Register in main.py — instantiate self._sets = SetsClient(...) and add delegated methods on DataMaker:
def get_sets(self, project_id=None): return self._sets.get_sets(project_id)
def get_set(self, set_id): return self._sets.get_set(set_id)
def create_set(self, name, data=None, description=None, row_count=None, project_id=None): ...
def update_set(self, set_id, set_data): return self._sets.update_set(set_id, set_data)
def delete_set(self, set_id): return self._sets.delete_set(set_id)
3. save_set() convenience helper — the documented scenario-facing entry point. Thin wrapper over create_set (or upsert by name if we want re-running a scenario to overwrite rather than duplicate):
def save_set(self, name, data, description=None, project_id=None) -> Dict:
"""Persist rows as a saved Set. Used from scenarios: dm.save_set("orders-regression", rows)."""
return self._sets.create_set(name=name, data=data, description=description, project_id=project_id)
Decide whether save_set should be create-only or upsert-by-name. Upsert matches the "re-run a scenario, refresh the regression set" mental model, but the API has no upsert endpoint yet — if we want it, either add GET /sets?name= filtering + client-side find-then-PATCH, or a server-side upsert. Flag for discussion.
Acceptance criteria
Reference
src/datamaker/routes/templates.py — route-module + CRUD pattern
src/datamaker/main.py — client composition + delegated methods
- API side (in
automators-com/datamaker): apps/datamaker-api/src/routes/sets.ts, packages/database/prisma/schema.prisma (Set model). Tracked there by automators-com/datamaker#2443.
Summary
The DataMaker API now exposes a persistent, team/project-scoped Sets resource (
/setsCRUD), but the Python SDK has no way to read or write it. A Set is a durable snapshot of generated/fetched rows (a regression dataset, shared fixture, or audit-ready export). The web UI's empty state and theSetmodel docstring both advertisedm.save_set()as the scenario-facing entry point — but that method does not exist yet. This issue tracks adding aSetsClientroute module plus delegatedDataMakermethods, mirroring the existingTemplatesClientpattern.Motivation
Scenarios run inside
datamaker-runnerand talk to the API through this SDK (DATAMAKER_API_KEY/DATAMAKER_API_URL). Users are told they can persist a regression set from a scenario viadm.save_set(...), but there is no SDK surface for it, so the documented flow dead-ends. Sets need first-class SDK methods like every other entity (templates, connections, projects).API surface to wrap
The API (
datamaker-api) exposes these routes, allauthMiddleware+scopeFilterscoped to the caller's team/project:GET/sets?projectId=GET/sets/:idPOST/sets{ name, description?, data?, rowCount?, projectId? }.rowCountis derived fromdata(array length) when omitted;teamId/projectIdare resolved server-side from the validated scope.PATCH/sets/:idname?,description?,data?,rowCount?)DELETE/sets/:iddatais arbitrary JSON; the common case is an array of row objects.Proposed implementation (mirror
routes/templates.py)1. New route module
src/datamaker/routes/sets.py2. Register in
main.py— instantiateself._sets = SetsClient(...)and add delegated methods onDataMaker:3.
save_set()convenience helper — the documented scenario-facing entry point. Thin wrapper overcreate_set(or upsert by name if we want re-running a scenario to overwrite rather than duplicate):Acceptance criteria
SetsClientadded undersrc/datamaker/routes/sets.pywith get/get-one/create/update/delete.DataMakerexposes delegatedget_sets/get_set/create_set/update_set/delete_set+save_set.save_set(name, data, ...)persists rows and the result shows on the Sets page.dataarray length auto-populatesrowCount(parity with the API's derivation) when not passed.tests/covering the new methods (follow existing route tests).llms.txtupdated to documentdm.save_set()and the Sets methods.Reference
src/datamaker/routes/templates.py— route-module + CRUD patternsrc/datamaker/main.py— client composition + delegated methodsautomators-com/datamaker):apps/datamaker-api/src/routes/sets.ts,packages/database/prisma/schema.prisma(Setmodel). Tracked there by automators-com/datamaker#2443.