# Scouter E2E scenario catalog

## Execution context

- **MongoDB**, **Redis**, and **PostgreSQL** run as session-scoped testcontainers with autouse cleanup between tests.
- **PI Web API** is an in-process HTTP server (`e2e/pi_web_api_test_server.py`) speaking the wire format consumed by `PIWebAPIClient`.
- **Temporal** uses `WorkflowEnvironment.start_local()` and a single worker on `scouter-test-queue`.
- **Production code is not mocked** (except `Logger` and optional notification insert spy).

---

## 0. Harness smoke tests

Diagnostic-only checks under `e2e/test_harness_smoke.py`. They are not business scenarios; they exist to fail fast when the harness itself (Docker / containers / Temporal worker wiring) is broken, before the numbered suite runs.

### 0.0.1 Postgres schema ready (`test_postgres_schema_ready`)

Confirms the autouse fixture executed `db_schema.sql` and `sientia_data.laborious_data` exists in the Postgres testcontainer.

### 0.0.2 Activities construct (`test_activities_construct`)

Confirms the production `Activities` instance initializes against the Mongo/Redis/Postgres testcontainers without hanging (no `patch(...)` involved).

### 0.0.3 Temporal PI happy path (`test_temporal_pi_happy_path`)

End-to-end liveness check: `WorkflowEnvironment.start_local()` + worker + in-process PI server + `PIWebAPIScouter` complete without raising. Functional assertions for this flow live in scenario **2.1.1**.

---

## 1. Scouter workflow

### 1.1.1 Happy path

Seed `raw_<schedule>` with multiple documents, run `Scouter`, assert Postgres rows and Redis `last_data_timestamp:scouter:<schedule>`.

### 1.2.1 Incremental load

Pre-seed Redis timestamp; seed older and newer Mongo docs; assert only newer rows export and timestamp advances.

### 1.3.1 Empty Mongo early exit

Empty `raw_<schedule>`; workflow exits without Postgres rows or Redis timestamp key.

### 1.3.2 No Redis timestamp first run

No prior Redis key; all seeded Mongo docs load and timestamp is written after success.

---

## 2. PIWebAPIScouter workflow

### 2.1.1 Happy path

PI server `success` mode with two tags; assert Postgres rows, Redis hold key, one HTTP request recorded.

### 2.1.2 Multiple tags

Five tags with `avg` / `mdn` / `max` / `min` / `lts`; assert five distinct `variable` values and exact aggregated numbers in Postgres.

### 2.1.3 Debug data package

`debug_data_package=True`; assert `data_package_pi_web_api_scouter_*` Redis key with `data` and `held_data`.

### 2.2.1 Empty response early exit

Server `empty` mode; zero Postgres rows for `model_id`, one request recorded.

### 2.3.1 PI Web API connection error

Server `error` mode (HTTP 5xx); workflow fails; `PI_WEB_API_REQUEST_ERROR` notification in Mongo.

### 2.3.2 PI Web API timeout

Server `timeout` mode; workflow fails; `PI_WEB_API_REQUEST_ERROR` notification sent.

### 2.3.3 Invalid endpoint

Workflow uses `/invalid/endpoint` (404); workflow fails; `PI_WEB_API_REQUEST_ERROR` notification sent.

---

## 3. CoreScouter subworkflow

### 3.1.1 Complete processing success

Single tag, no filters; Postgres row and `held_data_*` Redis key; no `data_package_*` key.

### 3.1.2 Null values filter discard

`NULL_VALUES_FILTER` DISCARD; one WARNING notification; only valid row in Postgres.

### 3.1.3 Null values filter warn

`NULL_VALUES_FILTER` WARN; notification sent; both rows in Postgres.

### 3.1.4 Out of bounds filter discard

`OUT_OF_BOUNDS_FILTER` DISCARD; in-range row only; WARNING notification.

### 3.1.5 Aggregation avg

Three points; Postgres `value` equals arithmetic mean (20.0).

### 3.1.6 Aggregation mdn

Median equals 5.0 in Postgres.

### 3.1.7 Aggregation max

Maximum equals 15.0 in Postgres.

### 3.1.8 Aggregation min

Minimum equals 30.0 in Postgres.

### 3.1.9 Aggregation lts

Last-by-timestamp value equals 300.0 in Postgres.

### 3.1.10 Fill missing tags

`fill_missing_tags=True`; `held_data_*` contains missing tag keys with `None`.

### 3.1.11 Debug data package

`debug_data_package=True`; `data_package_*` Redis key decodes to dict with `data` and `held_data`.

### 3.2.1 Empty after grouping early exit

Empty column-oriented `data`; no Postgres rows; no populated `held_data_*`.

### 3.3.1 Invalid aggregation function

`aggr_func= bogus`; `AGGREGATION_ISSUES` ERROR notification; bogus tag absent from Postgres.

### 3.3.2 Postgres export failure surfaces

Drop `value` column before run; workflow fails; ERROR notification in Mongo.