Add scheduled tasks feature and delayed entity signals#160
Conversation
Add a recurring schedule feature (durabletask.scheduled) for parity with durabletask-dotnet, built on durable entities and a helper orchestrator. Enable it on a worker with configure_scheduled_tasks(worker) and manage schedules from the client via ScheduledTaskClient / ScheduleClient (create, describe, list, update, pause, resume, delete). To support the schedule entity's self-rearming, add an optional signal_time parameter to entity, orchestration-context, and client signal_entity methods, and make the in-memory backend honor delayed (scheduled) entity signals. Includes unit tests, in-memory E2E tests, and live DTS E2E tests for both the schedule feature and delayed signals, plus an example and changelog entries.
There was a problem hiding this comment.
Pull request overview
This PR introduces a new scheduled tasks capability (durabletask.scheduled) to the core Durable Task Python SDK, implemented on top of durable entities plus a helper orchestrator. As a prerequisite, it also adds delayed (scheduled) entity signals end-to-end (entity APIs, orchestration context, client APIs, and the in-memory test backend) to support time-based schedule re-arming.
Changes:
- Added the
durabletask.scheduledpackage (schedule entity, client surface, models/DTOs, transition rules, registration helper, and error types). - Added a
signal_timeparameter across entity/orchestration/clientsignal_entityAPIs and mapped it to the protobufscheduledTimefield. - Updated the in-memory backend to defer entity operations scheduled in the future; added unit/E2E coverage plus a runnable example and changelog entry.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/durabletask/scheduled/test_transitions.py | Unit tests for schedule status transition rules. |
| tests/durabletask/scheduled/test_scheduled_e2e.py | In-memory E2E coverage for schedule lifecycle and runs. |
| tests/durabletask/scheduled/test_schedule_entity.py | Unit tests for schedule entity behavior/actions via executor harness. |
| tests/durabletask/scheduled/test_models.py | Unit tests for scheduled-task models, validation, and serialization. |
| tests/durabletask/scheduled/init.py | Test package init. |
| tests/durabletask/entities/test_delayed_signals_e2e.py | In-memory E2E coverage for delayed entity signals (client + orchestrator). |
| tests/durabletask-azuremanaged/scheduled/test_dts_scheduled_e2e.py | Live DTS E2E coverage for scheduled tasks. |
| tests/durabletask-azuremanaged/scheduled/init.py | Azuremanaged test package init. |
| tests/durabletask-azuremanaged/entities/test_dts_delayed_signals_e2e.py | Live DTS E2E coverage for delayed entity signals. |
| examples/scheduled_tasks.py | End-to-end sample showing schedule creation and management. |
| durabletask/worker.py | Extends orchestration-context signal_entity plumbing to accept signal_time. |
| durabletask/testing/in_memory_backend.py | Implements delayed entity operation delivery in the in-memory backend. |
| durabletask/task.py | Updates orchestration-context API contract/docs to include signal_time. |
| durabletask/scheduled/transitions.py | Defines valid schedule state transitions keyed by operation. |
| durabletask/scheduled/schedule_status.py | Introduces ScheduleStatus enum. |
| durabletask/scheduled/schedule_entity.py | Implements the schedule entity state machine and re-arming via delayed signals. |
| durabletask/scheduled/registration.py | Adds configure_scheduled_tasks(worker) registration helper. |
| durabletask/scheduled/orchestrator.py | Adds helper orchestrator for “awaitable” schedule entity operations. |
| durabletask/scheduled/models.py | Adds public DTOs/options plus internal config/state serialization. |
| durabletask/scheduled/exceptions.py | Adds scheduled-task exception types. |
| durabletask/scheduled/client.py | Adds ScheduledTaskClient and per-schedule ScheduleClient. |
| durabletask/scheduled/init.py | Exposes the scheduled tasks public surface. |
| durabletask/internal/helpers.py | Adds scheduled time support to entity-signal orchestrator actions. |
| durabletask/internal/client_helpers.py | Adds scheduled time support to client SignalEntityRequest. |
| durabletask/entities/entity_context.py | Adds signal_time to entity-side signal_entity API and action emission. |
| durabletask/entities/durable_entity.py | Adds signal_time to DurableEntity.signal_entity facade. |
| durabletask/client.py | Adds signal_time to sync + async client signal_entity APIs. |
| CHANGELOG.md | Documents the new scheduled tasks feature and delayed entity signals. |
…uled-tasks # Conflicts: # CHANGELOG.md # durabletask/client.py # durabletask/entities/entity_context.py # durabletask/internal/client_helpers.py
The JSON serializer encoded dataclasses via dataclasses.asdict before ever checking for a to_json hook, so a dataclass could not override its own serialization -- a problem for dataclasses whose fields are not JSON-native (e.g. timedelta/datetime). The read path already consults from_json before its dataclass branch; this makes the write path symmetric by checking the to_json hook first and falling back to asdict/SimpleNamespace.
With the serializer now honoring the to_json hook for dataclasses, the schedule option types expose to_json/from_json instead of bespoke to_dict/from_dict. The schedule entity operations can again annotate their input as the option dataclass and let the worker reconstruct it via the from_json hook, removing the manual _coerce_options shim and the Any-typed parameter workaround.
berndverst
left a comment
There was a problem hiding this comment.
Review summary
Thanks @andystaples — this is a solid, well-tested feature that mirrors the .NET ScheduledTasks design closely: the Schedule entity state machine, the execution-token mechanism that cancels stale timer signals, and the per-operation orchestrator so client writes can await entity completion. Lint is clean and the scheduled + delayed-signal unit/in-memory-E2E tests pass locally (48 passed).
The recent commits are good improvements: making ScheduleState.from_json converter-free means state round-trips under any code path (not just the worker's threaded converter), and signal_time -> scheduledTime is now correctly threaded through both the client request builder and the orchestrator-action builder (these previously hardcoded scheduledTime=None).
A few things I'd like addressed or explicitly deferred (details inline):
Recommend before merge
- Naive/aware
datetimecrash inScheduleClient._matches_filter: comparing an awareschedule_created_atagainst a possibly-naivecreated_from/created_toraisesTypeError. Cheap to fix + test. - Missing
WORKER_CAPABILITY_SCHEDULED_TASKSadvertisement: .NET advertises this capability; Python defines it in the protobuf but never sends it. Either wire it throughconfigure_scheduled_tasks/worker.py, or confirm the backend doesn't gate on it.
Reasonable follow-ups (these match current .NET behavior)
- Typed schedule exceptions flattened to
RuntimeErroracross the orchestration boundary. list_schedulesapplies status/created filters client-side, so pages may be underfilled.- Exclusive
created_from/created_tobounds (consider inclusive). - Cross-SDK entity-name casing (
@schedule@vs@Schedule@). - No async client surface (the SDK already has an async client; .NET offers async).
Overall: mergeable with minor fixes — I'd gate on the datetime fix and a decision on the capability flag; the rest can be tracked as follow-ups. Nice work.
…solidation - Advertise WORKER_CAPABILITY_SCHEDULED_TASKS from configure_scheduled_tasks via a new TaskHubGrpcWorker.add_capability(), mirroring .NET's UseScheduledTasks (currently inert against the DTS backend, which only gates on HistoryStreaming). - Fix naive-vs-aware datetime TypeError in schedule list filtering: ScheduleQuery normalizes its created_from/created_to bounds to aware UTC, and the client filter defensively normalizes the stored timestamp. Bounds remain exclusive to match the .NET ScheduledTasks implementation. - Consolidate the duplicated _ensure_aware helpers into a single public helpers.ensure_aware, removing the cross-module private-usage warning. - Document that list_schedules applies status/created filters client-side, so pages may be underfilled (matches .NET). - Add unit tests for query normalization, exclusive filter bounds, and the scheduled-tasks capability advertisement.
Summary
Adds a scheduled tasks feature (
durabletask.scheduled) to the Python SDK,bringing it to parity with the
Microsoft.DurableTask.ScheduledTaskspackage indurabletask-dotnet. A schedule periodically starts a target orchestration based
on a configurable interval and time window. The feature is built entirely on top
of durable entities plus a small helper orchestrator — no new sidecar/protocol
requirements.
To make schedules work, this PR also adds delayed (scheduled) entity signals
across the entity, orchestration-context, and client
signal_entityAPIs, andteaches the in-memory test backend to honor them.
What's new
durabletask.scheduledpackageconfigure_scheduled_tasks(worker)— registers the schedule entity andoperation orchestrator on a worker (parity with .NET's
UseScheduledTasks).ScheduledTaskClient—create_schedule,get_schedule,get_schedule_client,list_schedules.ScheduleClient— per-schedulecreate,describe,update,pause,resume,delete.ScheduleCreationOptions,ScheduleUpdateOptions,ScheduleDescription,ScheduleQuery,ScheduleStatus.ScheduleNotFoundError,ScheduleInvalidTransitionError,ScheduleClientValidationError.Schedules support
interval(≥ 1s),start_at,end_at, andstart_immediately_if_late, with a state machine (Uninitialized→Active⇄
Paused) and an execution-token mechanism that cancels stale timer signals,mirroring the .NET design.
Delayed entity signals (prerequisite)
signal_timeparameter to:EntityContext.signal_entity/DurableEntity.signal_entityOrchestrationContext.signal_entitysignal_entity(sync and async)scheduledTimefield. The public parametername
signal_timematches the .NET public API (SignalEntityOptions.SignalTime).scheduledTimeis inthe future (via a background timer thread), on both the client-signal and
orchestration-emitted-signal paths.
Tests
and state/options serialization round-trips.
prefix/status filters, delete; plus delayed-signal deferral via client and
orchestrator.
pytest.mark.dts): scheduled-task lifecycle and delayedsignals, verified against the DTS emulator.
Docs & examples
examples/scheduled_tasks.py.CHANGELOG.mdentries under Unreleased.Notes
durabletaskpackage (not a separate distribution as in.NET) for simpler installation and testing.
schedule, so list filtering usesthe
@schedule@<prefix>instance-ID prefix.