SIENTIAPDE-1646
Update E2E test report and enhance drift analysis handling - Updated the E2E test report metrics to reflect the latest test results, showing 47 collected tests with all passing. - Removed outdated sections related to failed tests and their causes, streamlining the report. - Implemented a regression fix in the drift analysis to handle empty merged frames, ensuring workflows skip export when no drift metrics are available. - Enhanced the `insert_sample_data` and `insert_sample_prediction` functions to allow customizable timestamps for better test accuracy. - Refactored E2E tests to improve clarity and maintainability, particularly in handling repeat scenarios with distinct timestamps.
This commit is contained in:
@@ -85,11 +85,15 @@ Source: `e2e/test_predictions_batch_prediction_process.py`
|
||||
- Workflow exits without export.
|
||||
|
||||
#### 2.1.3 REPEAT with history
|
||||
**Summary**: Prior prediction is reused.
|
||||
**Summary**: Prior prediction is reused; outcome depends on batch vs history timestamp.
|
||||
|
||||
**Description**:
|
||||
- Input gate returns `REPEAT`.
|
||||
- `repeat_last_prediction` path is executed using existing historical row.
|
||||
- `repeat_last_prediction` inserts a row using ``last_timestamp`` from the batch payload (max timestamp in `laborious_data` for the query), not the copied row’s timestamp.
|
||||
|
||||
**Tests**:
|
||||
- **Collision**: batch `last_timestamp` equals the historical prediction row’s `timestamp` → Postgres `unique_model_id_timestamp` violation; workflow fails; still one row.
|
||||
- **Distinct batch time**: laborious rows are stamped later than the historical prediction → second row inserted; same prediction fields as the first (see `assert_repeat`).
|
||||
|
||||
#### 2.1.4 REPEAT without history
|
||||
**Summary**: Repeat requested but no previous prediction exists.
|
||||
@@ -112,6 +116,8 @@ Source: `e2e/test_predictions_batch_prediction_process.py`
|
||||
#### 2.2.3 REPEAT on transform response error
|
||||
**Summary**: Transform response error triggers repeat-last-prediction path.
|
||||
|
||||
**Tests**: Same timestamp collision vs distinct batch timestamp as §2.1.3 (`*_fails` / `*_inserts_second_row`).
|
||||
|
||||
#### 2.2.4 STOP on transform content NaN
|
||||
**Summary**: Content gate (`NAN_VALUES`) blocks on all-NaN transform payload.
|
||||
|
||||
@@ -126,6 +132,8 @@ Source: `e2e/test_predictions_batch_prediction_process.py`
|
||||
#### 2.3.3 REPEAT on predict response error
|
||||
**Summary**: Predict response error routes to repeat-last-prediction.
|
||||
|
||||
**Tests**: Same timestamp collision vs distinct batch timestamp as §2.1.3 (`*_fails` / `*_inserts_second_row`).
|
||||
|
||||
### 2.4.1 Priority Conflict Resolution
|
||||
**Summary**: Deterministic selection when multiple filters produce different flags.
|
||||
|
||||
@@ -296,15 +304,33 @@ first 30% of target rows as reference.
|
||||
- The workflow surfaces the `ValueError` ("Invalid chunk period: ...").
|
||||
- No rows are persisted.
|
||||
|
||||
#### D.4.3 `chunk_period='s'` preserves seconds in `chunk_start_date`
|
||||
**Summary**: Target data spans two minutes with samples at second-30
|
||||
boundaries; the activity is configured with `chunk_period='s'`.
|
||||
#### D.4.3a Insufficient drift metrics while target has rows
|
||||
**Summary**: Laborious raises when the merged drift table is empty but the
|
||||
target window is non-empty (`Insufficient drift data:` + notification
|
||||
`MODEL_METRICS_DRIFT_INSUFFICIENT_DATA`).
|
||||
|
||||
**Description**:
|
||||
- The e2e patches `ModelMetrics.get_drift_metrics` to return an empty
|
||||
DataFrame, simulating a ``sientia_model`` path that emits no rows.
|
||||
|
||||
**Expected Outcome**:
|
||||
- Workflow fails; no rows in `sientia_data.drift_metrics`.
|
||||
|
||||
#### D.4.3b `chunk_period='s'` preserves seconds in `chunk_start_date`
|
||||
**Summary**: Target data includes sub-minute spacing across several minutes;
|
||||
the activity uses `chunk_period='s'`.
|
||||
|
||||
**Expected Outcome**:
|
||||
- At least one persisted `chunk_start_date` carries `seconds=30`, proving
|
||||
that the analyzer chunked at sub-minute granularity and the ISO-text
|
||||
serialization preserved the boundary.
|
||||
|
||||
**Note**: The e2e patches `DriftAnalysis._chunk_dataframe` to **skip empty**
|
||||
`pd.Grouper(freq='s')` buckets. The stock implementation iterates every
|
||||
second between min/max timestamps, producing empty chunks and NaT rows that
|
||||
`calculate_drift` filters away entirely. The durable fix belongs in
|
||||
`sientia_model`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Simple Metrics Workflow Scenarios
|
||||
|
||||
Reference in New Issue
Block a user