vitor-aignosi
8d34228d7b
Update E2E test report and enhance drift analysis handling - Updated the E2E test report metrics to reflect the latest test results, showing 47 collected tests with all passing. - Removed outdated sections related to failed tests and their causes, streamlining the report. - Implemented a regression fix in the drift analysis to handle empty merged frames, ensuring workflows skip export when no drift metrics are available. - Enhanced the `insert_sample_data` and `insert_sample_prediction` functions to allow customizable timestamps for better test accuracy. - Refactored E2E tests to improve clarity and maintainability, particularly in handling repeat scenarios with distinct timestamps.
49 lines
2.3 KiB
Markdown
49 lines
2.3 KiB
Markdown
# E2E test run report
|
||
|
||
**Date:** 2026-05-08
|
||
**Command:** `source venv/bin/activate && rtk pytest e2e/ -v --tb=short`
|
||
**Environment:** Linux, Python 3.11.15, pytest 9.0.3
|
||
|
||
## Summary
|
||
|
||
| Metric | Count |
|
||
|--------|------:|
|
||
| Collected | 47 |
|
||
| **Passed** | **47** |
|
||
| **Failed** | **0** |
|
||
|
||
Full pytest output (when using `rtk`) is stored under `~/.local/share/rtk/tee/` as timestamped `*_pytest.log` files.
|
||
|
||
---
|
||
|
||
## Regression fixed during this run (drift)
|
||
|
||
An initial e2e run failed **3** drift tests with:
|
||
|
||
`ValueError: The truth value of a Index is ambiguous`
|
||
|
||
**Cause:** `calculate_drift` passes `reference_data.columns` (a **pandas `Index`**) into `ModelMetrics.get_drift_metrics`, which forwards it to `sientia_model.analytics.drift_analysis.DriftAnalysis`. The analyzer uses patterns such as `if not features:` on the feature list. Boolean evaluation of an `Index` raises in pandas.
|
||
|
||
**Fix (in `laborious/activities/model_metrics.py`):** At the start of `get_drift_metrics`, normalize with `feature_names: list[str] = list(reference_columns)` and use `feature_names` in the `DriftAnalysis` config and in `detect_univariate_drift` / `detect_multivariate_drift`.
|
||
|
||
After this change, the full **`e2e/`** suite was re-run and **all 47 tests passed**.
|
||
|
||
---
|
||
|
||
## Suite coverage (high level)
|
||
|
||
| Area | File(s) | Notes |
|
||
|------|---------|--------|
|
||
| Drift workflow | `e2e/test_drift.py` | Happy path, 30% reference fallback, empty target, bad `chunk_period`, empty merge / no export, sub-minute chunking |
|
||
| Predictions batch | `e2e/test_predictions_batch_*.py` | Main workflow, prediction process gates / repeat, format export |
|
||
| Child workflows | `e2e/test_child_workflows_e2e.py` | Format + export path |
|
||
| Minimal retrain | `e2e/test_minimal_retrain.py` | Success / failure / missing target / no data |
|
||
| MinIO offload | `e2e/test_minio_offload.py` | Load query + batch path |
|
||
| Simple metrics | `e2e/test_simple_metrics.py` | Persistence, subset, edge cases |
|
||
|
||
---
|
||
|
||
## Relation to earlier reports
|
||
|
||
Older failures described in previous versions of this document (e.g. Jensen–Shannon NULLs vs `drift_metrics.value` NOT NULL, sparse `chunk_period='s'` data) are **not** reproduced in this run. If those topics resurface after data or dependency changes, see the dedicated notes under `docs/` (e.g. drift / JS investigations) and `e2e/scenarios.md`.
|