da8a796197
New, separate model_id-based filter distinct from the existing ignore_models_id deny-list (SIENTIAPDE-2012): a group configuring model_ids only receives notifications from those models. Both axes apply together (AND) when a group sets both. Design decisions made while implementing (matching the levels/ ignore_models_id split from SIENTIAPDE-2011/2012, not yet confirmed with the team beyond what's in the ticket): - Query-time filter: union of every active group's model_ids, applied only when ALL active groups set one -- mirrors levels' exact union-or-nothing logic in _compute_query_time_filter. Combines cleanly with ignore_models_id's intersection $nin on the same model_id field. - Per-group filter: extended _check_levels_and_ignore_models_id (now _check_levels_and_model_filters) in slot_manager.py, and the inline condition in formatters.py's filter_notification_reports. - Empty/missing model_ids means no restriction, same convention as levels and ignore_models_id. Covered by new unit tests (slot_manager, formatters, _compute_query_time_filter) and two new e2e tests against real Mongo/Redis/Temporal: per-group allow-list interacting with another unrestricted group, and the query-time union excluding a model outside every active group's allow-list. Not yet verified on the real cluster -- blocked on the dev cluster's package index not auto-syncing new sientia_do releases (separate issue, being investigated for SIENTIAPDE-2013). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
4.4 KiB
E2E Scenario Documentation — Orchestrator
Functional reference for orchestrator E2E scenarios. Tests live under e2e/, use @pytest.mark.e2e, and run with:
pytest e2e/ --override-ini testpaths=e2e -m e2e
Every section below maps 1:1 to an actual test function (named in each section, or as a bullet
under a Source: heading) — this file describes what the suite verifies today, not planned or
aspirational coverage. Adding a scenario here without a matching test (or vice versa) is a doc bug;
fix the mismatch instead of leaving it for later.
Execution context
- MongoDB, Redis, PostgreSQL: testcontainers (session-scoped).
- SMTP: in-process
aiosmtpd(e2e/smtp_test_server.py). - Temporal:
WorkflowEnvironment.start_local()with stub workers onscouter/laborious. - Production code under
orchestrator/**is not mocked; onlyLoggermay be aMagicMock.
1. Orchestrator workflow
Source: e2e/test_orchestrator_main_workflow.py
1.1.1 Happy path
Pipelines in Mongo → Temporal schedule created in the laborious namespace, orchestrated_schedules entries written.
1.3.1 Create-only
New pipeline only → Temporal schedule created for it.
1.3.3 Delete-only
Pipeline removed from Mongo → previously-created Temporal schedule deleted.
1.5.1 Empty pipelines
No active pipelines → no orchestrated_schedules documents written, even with a pre-existing
unrelated Temporal schedule present.
1.6.1 TTL index bootstrap
First run creates TTL index on notification collection used by scouter pipelines.
2. Alerts workflow
Source: e2e/test_alerts_main_workflow.py
A.1.1 Happy path
ERROR notification → one SMTP message, one log_report row, Redis cache key.
A.1.2 TTL duplicate suppression
Second run with same data and cache seeded → no extra email or log row.
A.2.1 Group filtering
Receiver group levels / ignore_models_id honored, both halves now covered:
the query-time union/intersection across active groups (SIENTIAPDE-2011, see
LoadNotificationPackage below) and the per-group precise slice
(SIENTIAPDE-2012) — a group only receives notifications matching its own
levels/ignore_models_id, even when the query-time filter widened the fetch
to satisfy another active group's broader configuration.
A.2.2 Model_id allow-list
Receiver group model_ids (SIENTIAPDE-2149) honored: a group configuring it only
receives notifications from those models, even when another active group with
no such restriction widens the query-time fetch to include every model.
A.3.1 Empty queue
No notifications → no SMTP, no Postgres row.
3. Reports workflow
Source: e2e/test_reports_main_workflow.py
R.1.1 Happy path
Mixed ERROR/WARNING/INFO → one HTML email with all section headings.
R.1.2 Per-level rendering
Single-level notifications → only matching section in HTML body.
R.2.1 Empty queue
No notifications → no SMTP, no Postgres row.
4. Subworkflows
LoadNotificationPackage — e2e/test_subworkflow_load_notification_package.py
- No prior Redis timestamp → all matching notifications returned, max timestamp stored.
- Prior timestamp → only newer notifications returned.
- Query-time filter (SIENTIAPDE-2011) → union of active groups'
levels, intersection of theirignore_models_id; a model ignored by only one of several active groups is not excluded from the fetch. - Model_id allow-list union at query time (SIENTIAPDE-2149) → when every active group sets
model_ids, the fetch is restricted to the union of every group's allow-list; a model outside that union is never fetched at all, even for a group that would otherwise want it via a different axis. - Empty Mongo → no Redis timestamp write.
- Batch size cap (SIENTIAPDE-2013) → with 5 pending notifications and a batch size of 2, each run returns exactly the next-oldest 2 (sorted ascending by timestamp), and the Redis cursor advances only to what was actually returned, so three successive runs drain the backlog in order instead of skipping anything.
- Batch size 0 means unlimited (SIENTIAPDE-2013) → the configured default (
0) does not cap the query; 12 pending notifications all come back in one run, verified against a real MongoDB instance rather than assumed from pymongo's documented convention.
ProcessNotifications — e2e/test_subworkflow_process_notifications.py
- Full round trip: HTML → SMTP →
log_reportin Postgres. - Empty receiver groups →
{}.