34dbc886f3
Update project configuration and dependencies - Added .mypy_cache and .cursor to .gitignore. - Changed asyncio_default_fixture_loop_scope and asyncio_default_test_loop_scope to "session" in pyproject.toml. - Updated e2e testing dependencies in requirements-dev.txt, replacing fakeredis and mongomock with pytest-httpserver. - Updated requirements.txt to use sientia_do instead of a specific git commit. - Modified sonar-project.properties to remove a file from coverage exclusions. - Enhanced E2E test fixtures in e2e/conftest.py for better container management. - Cleaned up e2e test files related to CoreScouter and PIWebAPIScouter workflows.
139 lines
4.4 KiB
Markdown
139 lines
4.4 KiB
Markdown
# Scouter E2E scenario catalog
|
|
|
|
## Execution context
|
|
|
|
- **MongoDB**, **Redis**, and **PostgreSQL** run as session-scoped testcontainers with autouse cleanup between tests.
|
|
- **PI Web API** is an in-process HTTP server (`e2e/pi_web_api_test_server.py`) speaking the wire format consumed by `PIWebAPIClient`.
|
|
- **Temporal** uses `WorkflowEnvironment.start_local()` and a single worker on `scouter-test-queue`.
|
|
- **Production code is not mocked** (except `Logger` and optional notification insert spy).
|
|
|
|
---
|
|
|
|
## 0. Harness smoke tests
|
|
|
|
Diagnostic-only checks under `e2e/test_harness_smoke.py`. They are not business scenarios; they exist to fail fast when the harness itself (Docker / containers / Temporal worker wiring) is broken, before the numbered suite runs.
|
|
|
|
### 0.0.1 Postgres schema ready (`test_postgres_schema_ready`)
|
|
|
|
Confirms the autouse fixture executed `db_schema.sql` and `sientia_data.laborious_data` exists in the Postgres testcontainer.
|
|
|
|
### 0.0.2 Activities construct (`test_activities_construct`)
|
|
|
|
Confirms the production `Activities` instance initializes against the Mongo/Redis/Postgres testcontainers without hanging (no `patch(...)` involved).
|
|
|
|
### 0.0.3 Temporal PI happy path (`test_temporal_pi_happy_path`)
|
|
|
|
End-to-end liveness check: `WorkflowEnvironment.start_local()` + worker + in-process PI server + `PIWebAPIScouter` complete without raising. Functional assertions for this flow live in scenario **2.1.1**.
|
|
|
|
---
|
|
|
|
## 1. Scouter workflow
|
|
|
|
### 1.1.1 Happy path
|
|
|
|
Seed `raw_<schedule>` with multiple documents, run `Scouter`, assert Postgres rows and Redis `last_data_timestamp:scouter:<schedule>`.
|
|
|
|
### 1.2.1 Incremental load
|
|
|
|
Pre-seed Redis timestamp; seed older and newer Mongo docs; assert only newer rows export and timestamp advances.
|
|
|
|
### 1.3.1 Empty Mongo early exit
|
|
|
|
Empty `raw_<schedule>`; workflow exits without Postgres rows or Redis timestamp key.
|
|
|
|
### 1.3.2 No Redis timestamp first run
|
|
|
|
No prior Redis key; all seeded Mongo docs load and timestamp is written after success.
|
|
|
|
---
|
|
|
|
## 2. PIWebAPIScouter workflow
|
|
|
|
### 2.1.1 Happy path
|
|
|
|
PI server `success` mode with two tags; assert Postgres rows, Redis hold key, one HTTP request recorded.
|
|
|
|
### 2.1.2 Multiple tags
|
|
|
|
Five tags with `avg` / `mdn` / `max` / `min` / `lts`; assert five distinct `variable` values and exact aggregated numbers in Postgres.
|
|
|
|
### 2.1.3 Debug data package
|
|
|
|
`debug_data_package=True`; assert `data_package_pi_web_api_scouter_*` Redis key with `data` and `held_data`.
|
|
|
|
### 2.2.1 Empty response early exit
|
|
|
|
Server `empty` mode; zero Postgres rows for `model_id`, one request recorded.
|
|
|
|
### 2.3.1 PI Web API connection error
|
|
|
|
Server `error` mode (HTTP 5xx); workflow fails; `PI_WEB_API_REQUEST_ERROR` notification in Mongo.
|
|
|
|
### 2.3.2 PI Web API timeout
|
|
|
|
Server `timeout` mode; workflow fails; `PI_WEB_API_REQUEST_ERROR` notification sent.
|
|
|
|
### 2.3.3 Invalid endpoint
|
|
|
|
Workflow uses `/invalid/endpoint` (404); workflow fails; `PI_WEB_API_REQUEST_ERROR` notification sent.
|
|
|
|
---
|
|
|
|
## 3. CoreScouter subworkflow
|
|
|
|
### 3.1.1 Complete processing success
|
|
|
|
Single tag, no filters; Postgres row and `held_data_*` Redis key; no `data_package_*` key.
|
|
|
|
### 3.1.2 Null values filter discard
|
|
|
|
`NULL_VALUES_FILTER` DISCARD; one WARNING notification; only valid row in Postgres.
|
|
|
|
### 3.1.3 Null values filter warn
|
|
|
|
`NULL_VALUES_FILTER` WARN; notification sent; both rows in Postgres.
|
|
|
|
### 3.1.4 Out of bounds filter discard
|
|
|
|
`OUT_OF_BOUNDS_FILTER` DISCARD; in-range row only; WARNING notification.
|
|
|
|
### 3.1.5 Aggregation avg
|
|
|
|
Three points; Postgres `value` equals arithmetic mean (20.0).
|
|
|
|
### 3.1.6 Aggregation mdn
|
|
|
|
Median equals 5.0 in Postgres.
|
|
|
|
### 3.1.7 Aggregation max
|
|
|
|
Maximum equals 15.0 in Postgres.
|
|
|
|
### 3.1.8 Aggregation min
|
|
|
|
Minimum equals 30.0 in Postgres.
|
|
|
|
### 3.1.9 Aggregation lts
|
|
|
|
Last-by-timestamp value equals 300.0 in Postgres.
|
|
|
|
### 3.1.10 Fill missing tags
|
|
|
|
`fill_missing_tags=True`; `held_data_*` contains missing tag keys with `None`.
|
|
|
|
### 3.1.11 Debug data package
|
|
|
|
`debug_data_package=True`; `data_package_*` Redis key decodes to dict with `data` and `held_data`.
|
|
|
|
### 3.2.1 Empty after grouping early exit
|
|
|
|
Empty column-oriented `data`; no Postgres rows; no populated `held_data_*`.
|
|
|
|
### 3.3.1 Invalid aggregation function
|
|
|
|
`aggr_func= bogus`; `AGGREGATION_ISSUES` ERROR notification; bogus tag absent from Postgres.
|
|
|
|
### 3.3.2 Postgres export failure surfaces
|
|
|
|
Drop `value` column before run; workflow fails; ERROR notification in Mongo.
|