gitea_admin/sientia-dataops-orchestrator_temporal
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Merge branch 'main' into fix/SIENTIAPDE-2013

Browse Source
This commit is contained in:
Eduardo Rios
2026-09-17 11:48:58 -03:00
committed by GitHub
parent d46bd55cb4 5cc8015fc7
commit 4ece5cf9e3
9 changed files with 560 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
e2e/scenarios.md
View File

@@ -6,6 +6,11 @@ Functional reference for orchestrator E2E scenarios. Tests live under `e2e/`, us
pytest e2e/ --override-ini testpaths=e2e -m e2e
```
**Every section below maps 1:1 to an actual test function** (named in each section, or as a bullet
under a `Source:` heading) — this file describes what the suite verifies today, not planned or
aspirational coverage. Adding a scenario here without a matching test (or vice versa) is a doc bug;
fix the mismatch instead of leaving it for later.
## Execution context
- MongoDB, Redis, PostgreSQL: testcontainers (session-scoped).
@@ -21,31 +26,20 @@ Source: `e2e/test_orchestrator_main_workflow.py`
### 1.1.1 Happy path
Pipelines in Mongo → schedules created in correct namespace/task queue, Redis slots written, `orchestrated_schedules` updated.
### 1.2.1 No-op tick
Mongo, Redis, and Temporal already match desired state → no new schedules or slot writes.
Pipelines in Mongo → Temporal schedule created in the `laborious` namespace, `orchestrated_schedules` entries written.
### 1.3.1 Create-only
New pipeline only → schedules created, timestamps inserted.
### 1.3.2 Update-only
Existing pipeline with newer `updated_at` → schedule updated in Temporal.
New pipeline only → Temporal schedule created for it.
### 1.3.3 Delete-only
Pipeline removed from Mongo → schedule deleted from Temporal.
### 1.4.1 Conflict ordering
Pipeline update and slot delete on same OPC server → slot insert before delete (production ordering).
Pipeline removed from Mongo → previously-created Temporal schedule deleted.
### 1.5.1 Empty pipelines
No active pipelines → orphan schedules removed, no new orchestration writes.
No active pipelines → no `orchestrated_schedules` documents written, even with a pre-existing
unrelated Temporal schedule present.
### 1.6.1 TTL index bootstrap
@@ -65,18 +59,14 @@ ERROR notification → one SMTP message, one `log_report` row, Redis cache key.
Second run with same data and cache seeded → no extra email or log row.
### A.1.3 Persistent escalation
Alert past `notification_ttl` with cache cleared → new email sent.
### A.2.1 Group filtering
Receiver group `levels` / `ignore_models_id` honored. The query-time half
(union of `levels`, intersection of `ignore_models_id` across active groups)
is covered at the subworkflow level — see `LoadNotificationPackage` below.
The per-group half (each group only receiving its own precise slice) is not
yet implemented in production code (SIENTIAPDE-2012) and so is not yet
exercised here.
Receiver group `levels` / `ignore_models_id` honored, both halves now covered:
the query-time union/intersection across active groups (SIENTIAPDE-2011, see
`LoadNotificationPackage` below) and the per-group precise slice
(SIENTIAPDE-2012) — a group only receives notifications matching its own
`levels`/`ignore_models_id`, even when the query-time filter widened the fetch
to satisfy another active group's broader configuration.
### A.3.1 Empty queue