14b854148a
e2e fixtures (default_notification, seed_last_timestamp, default_receiver_group) still assumed the pre-SIENTIAPDE-2007 string-timestamp representation and the old ignore_models field name, so the load_notification_package e2e suite was failing independently of this ticket's changes: - default_notification now seeds `timestamp` as a native datetime (matches production since SIENTIAPDE-2007's Notification.timestamp = now()). - seed_last_timestamp now uses DATETIME_FORMAT_MS_WITH_TZ (matches what put_last_data_timestamp actually writes to Redis today). - default_receiver_group's ignore_models renamed to ignore_models_id, per this ticket's decision. - test_reports_main_workflow.py's explicit timestamp= call sites updated to match. Also fixes a real production bug in put_last_data_timestamp (slot_manager.py) surfaced by running the e2e suite against a real Temporal environment: Temporal's default data converter serializes datetime values to ISO-8601 strings across the activity boundary, and pymongo returns naive datetimes for BSON Date fields, so `data['timestamp'].max().strftime(...)` crashed on a plain str. Unit tests never caught this because they mock the activity boundary and never cross real (de)serialization. Fixed by normalizing through `pandas.to_datetime(..., utc=True)`, which handles native datetime/Timestamp and offset-aware/naive ISO strings uniformly. Adds test_load_package_query_time_filter_union_and_intersection, a new e2e test exercising the actual SIENTIAPDE-2011 behavior end-to-end: union of active groups' levels, intersection of their ignore_models_id, and the requirement that a model ignored by only one of several active groups must not be excluded from the fetch. Note: test_reports_main_workflow.py / test_alerts_main_workflow.py still have a separate, pre-existing failure in process_notifications.py (same family of bug: a notification's timestamp field parsed with a fixed strptime format after crossing the same Temporal serialization boundary) — confirmed present on the unmodified baseline too, unrelated to this ticket, tracked separately. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Orchestrator end-to-end tests
End-to-end tests run every external dependency for real (MongoDB, Redis, PostgreSQL via testcontainers; SMTP via in-process aiosmtpd; Temporal via WorkflowEnvironment.start_local()).
Requirements
- Docker (for testcontainers)
- Python dev dependencies:
pip install -r requirements-dev.txt
Run locally
source ./venv/bin/activate
pytest e2e/ --override-ini testpaths=e2e -m e2e -v
Stop on first failure:
pytest e2e/ --override-ini testpaths=e2e -m e2e -x
Coverage (separate from unit tests)
COVERAGE_FILE=.coverage.e2e pytest e2e/ --override-ini testpaths=e2e -m e2e --cov=orchestrator --cov-branch
coverage combine .coverage .coverage.e2e
coverage report
Unit tests keep the default .coverage file; the E2E run must set COVERAGE_FILE=.coverage.e2e so reports do not overwrite each other.
Scenario catalog
See scenarios.md for numbered scenarios and which test module implements each case.