8d34228d7b
Update E2E test report and enhance drift analysis handling - Updated the E2E test report metrics to reflect the latest test results, showing 47 collected tests with all passing. - Removed outdated sections related to failed tests and their causes, streamlining the report. - Implemented a regression fix in the drift analysis to handle empty merged frames, ensuring workflows skip export when no drift metrics are available. - Enhanced the `insert_sample_data` and `insert_sample_prediction` functions to allow customizable timestamps for better test accuracy. - Refactored E2E tests to improve clarity and maintainability, particularly in handling repeat scenarios with distinct timestamps.
2.3 KiB
E2E test run report
Date: 2026-05-08
Command: source venv/bin/activate && rtk pytest e2e/ -v --tb=short
Environment: Linux, Python 3.11.15, pytest 9.0.3
Summary
| Metric | Count |
|---|---|
| Collected | 47 |
| Passed | 47 |
| Failed | 0 |
Full pytest output (when using rtk) is stored under ~/.local/share/rtk/tee/ as timestamped *_pytest.log files.
Regression fixed during this run (drift)
An initial e2e run failed 3 drift tests with:
ValueError: The truth value of a Index is ambiguous
Cause: calculate_drift passes reference_data.columns (a pandas Index) into ModelMetrics.get_drift_metrics, which forwards it to sientia_model.analytics.drift_analysis.DriftAnalysis. The analyzer uses patterns such as if not features: on the feature list. Boolean evaluation of an Index raises in pandas.
Fix (in laborious/activities/model_metrics.py): At the start of get_drift_metrics, normalize with feature_names: list[str] = list(reference_columns) and use feature_names in the DriftAnalysis config and in detect_univariate_drift / detect_multivariate_drift.
After this change, the full e2e/ suite was re-run and all 47 tests passed.
Suite coverage (high level)
| Area | File(s) | Notes |
|---|---|---|
| Drift workflow | e2e/test_drift.py |
Happy path, 30% reference fallback, empty target, bad chunk_period, empty merge / no export, sub-minute chunking |
| Predictions batch | e2e/test_predictions_batch_*.py |
Main workflow, prediction process gates / repeat, format export |
| Child workflows | e2e/test_child_workflows_e2e.py |
Format + export path |
| Minimal retrain | e2e/test_minimal_retrain.py |
Success / failure / missing target / no data |
| MinIO offload | e2e/test_minio_offload.py |
Load query + batch path |
| Simple metrics | e2e/test_simple_metrics.py |
Persistence, subset, edge cases |
Relation to earlier reports
Older failures described in previous versions of this document (e.g. Jensen–Shannon NULLs vs drift_metrics.value NOT NULL, sparse chunk_period='s' data) are not reproduced in this run. If those topics resurface after data or dependency changes, see the dedicated notes under docs/ (e.g. drift / JS investigations) and e2e/scenarios.md.