From ee9c71cbbb3356acdb16c19177d4fe04ebfcefaf Mon Sep 17 00:00:00 2001 From: vitor-aignosi Date: Fri, 8 May 2026 16:39:12 -0300 Subject: [PATCH] SIENTIAPDE-1646 Refactor ModelMetrics to utilize DriftAnalysis for drift detection - Replaced ModelAnalysis with DriftAnalysis in the ModelMetrics class to enhance drift detection capabilities. - Updated method signatures and documentation to reflect the changes in target_name and return values. - Adjusted data handling to ensure compatibility with the new analysis methods and improved clarity in the drift metrics dataframe preparation. --- docs/E2E_TEST_REPORT.md | 149 ++++++++++++++++++ ...entia_model_drift_jensen_shannon_change.md | 55 +++++++ e2e/conftest.py | 140 ---------------- e2e/scenarios.md | 81 +--------- tests/conftest.py | 13 -- 5 files changed, 209 insertions(+), 229 deletions(-) create mode 100644 docs/E2E_TEST_REPORT.md create mode 100644 docs/sientia_model_drift_jensen_shannon_change.md diff --git a/docs/E2E_TEST_REPORT.md b/docs/E2E_TEST_REPORT.md new file mode 100644 index 0000000..1d8087a --- /dev/null +++ b/docs/E2E_TEST_REPORT.md @@ -0,0 +1,149 @@ +# E2E test run report + +**Date:** 2026-05-08 +**Command:** `source venv/bin/activate && rtk pytest e2e/ -v --tb=short` +**Environment:** Linux, Python 3.11.15, pytest 9.0.3 + +## Summary + +| Metric | Count | +|--------|------:| +| Collected | 43 | +| **Passed** | **37** | +| **Failed** | **6** | + +Full pytest output (compressed by `rtk`) was written to: + +`~/.local/share/rtk/tee/1778268193_pytest.log` + +--- + +## Failed tests (6) + +1. `e2e/test_drift.py::test_drift_happy_path_persists_all_columns_with_reference_data` +2. `e2e/test_drift.py::test_drift_uses_30pct_fallback_when_reference_unavailable` +3. `e2e/test_drift.py::test_drift_chunk_period_seconds_preserves_seconds_in_chunk_start_date` +4. `e2e/test_predictions_batch_prediction_process.py::test_scenario_2_1_3_input_gate_triggers_repeat` +5. `e2e/test_predictions_batch_prediction_process.py::test_scenario_2_2_3_transform_gate_triggers_repeat` +6. `e2e/test_predictions_batch_prediction_process.py::test_scenario_2_3_3_predict_gate_triggers_repeat` + +**Follow-up:** Jensen–Shannon NULLs for the drift happy-path scenario are fully traced (histogram out-of-range + `density=True`, vs NannyML’s leftover bin) in [DRIFT_JS_NULL_INVESTIGATION.md](./DRIFT_JS_NULL_INVESTIGATION.md). + +--- + +## Failure group A — Drift: `drift_metrics.value` NOT NULL (2 tests) + +### Error + +`psycopg2.errors.NotNullViolation`: null value in column `value` of relation `sientia_data.drift_metrics` violates not-null constraint. + +Example failing row (from logs): `feature=sensor_2`, `method=jensen_shannon`, `value=null`, with `kolmogorov_smirnov` / `wasserstein` populated for the same chunk. + +The bulk INSERT built by `Activities.export_data_to_postgres` includes parameters such as `'value__4': None` for `jensen_shannon` on a given chunk. + +### Root cause + +`calculate_drift` (real `DriftAnalysis` + `ModelMetrics.get_drift_metrics`) can emit **NaN / missing** values for some metric methods (here **Jensen–Shannon**) on some features/chunks. Pandas/SQLAlchemy turns that into SQL `NULL`, while the E2E schema (mirroring production) defines: + +```sql +value numeric NOT NULL +``` + +in `e2e/db_schema.sql` for `sientia_data.drift_metrics`. + +### Recommended fixes (pick one consistent with product rules) + +1. **Application layer (preferred if NULLs are never valid in production):** Before export, sanitize the drift dataframe — e.g. drop rows where `value` is null/NaN, or replace with a defined sentinel (only if product agrees), or skip emitting that method row when the statistic is undefined. +2. **Analytics layer:** Harden the Jensen–Shannon (and similar) paths so they always return a finite float for the supported inputs, or explicitly map “undefined” to an agreed numeric convention. +3. **Schema (only if product allows missing metrics):** Align DDL with reality by making `value` nullable — **only** if production and downstream consumers already expect missing metrics; the E2E comment in `test_drift.py` suggests `feature`/`timestamp` nullable cases exist, but `value` is still listed in `NON_NULL_DRIFT_COLUMNS`. + +### Tests affected + +- `test_drift_happy_path_persists_all_columns_with_reference_data` +- `test_drift_uses_30pct_fallback_when_reference_unavailable` + +--- + +## Failure group B — Drift: chunk period seconds (1 test) + +### Error + +`AssertionError: expected at least one drift row to be persisted` (`e2e/test_drift.py:493`). + +The workflow run completed without failing the test via `WorkflowFailureError`, but **no rows** were found in `sientia_data.drift_metrics` for the model. + +### Likely cause + +In `Drift.run`, persistence runs only when `if drift_data:` is truthy (`laborious/workflows/drift.py`). An **empty** drift result skips `export_data_to_postgres`, so the table stays empty. + +Probable reasons: + +- With **`chunk_period='s'`** and only **three** target rows (30 s spacing), `calculate_drift` / `DriftAnalysis` may produce **no output rows** (insufficient data per chunk or internal filters). +- Less likely here: time-window mismatch — timestamps are built from `datetime.now(UTC)` with `interval: 60` minutes from `drift_base.json`, so data should still fall in the window. + +### Recommended fixes + +1. **Test data:** Increase the number of second-spaced points (and/or span multiple chunk boundaries) so the analyzer reliably emits at least one chunk row. +2. **Product code:** If sub-minute chunking is required to always produce metrics when any data exists, adjust `ModelMetrics` / `DriftAnalysis` integration for small-N second buckets. +3. **Diagnostics:** Run the same scenario with `pytest -s` and confirm logs for “empty `drift_data`” vs export errors. + +Solução a ser aplicada: +Dividir em dois testes: + +1. Teste com dados suficientes para produzir pelo menos uma linha de drift +2. Teste com dados insuficientes para produzir pelo menos uma linha de drift, mas ja esperando os erros e validando que nao foi persistido nada + +### Test affected + +- `test_drift_chunk_period_seconds_preserves_seconds_in_chunk_start_date` + +--- + +## Failure group C — Prediction REPEAT path: unique constraint on predictions (3 tests) + +### Error + +`psycopg2.errors.UniqueViolation`: duplicate key value violates unique constraint **`unique_model_id_timestamp`** on `sientia_data.predictions` (`model_id`, `timestamp`). + +### Root cause + +REPEAT is handled by `Activities.repeat_last_prediction` (registered from `sientia_do.temporal.activities.postgres_sync.Postgres`, via `Storage` inheritance in `laborious/activities/storage.py`). The E2E tests seed a prior row with a **fixed** timestamp: + +```python +# e2e/test_predictions_batch_prediction_process.py — insert_sample_prediction +VALUES ({model_id}, '2024-01-01 12:00:00+00:00', ...) +``` + +The helper `assert_repeat` expects **two** rows with the same `(model_id, prediction, prediction_confidence, prediction_status)` but compares **only** those columns — not `timestamp` (`e2e/helpers.py`). So the intended behavior is: **duplicate business payload**, not necessarily **duplicate primary unique key** `(model_id, timestamp)`. + +If `repeat_last_prediction` **INSERT**s a copy using the **same** `timestamp` as the last prediction, Postgres correctly rejects the second insert. + +### Recommended fixes + +1. **Implement REPEAT as UPSERT:** Use `ON CONFLICT (model_id, timestamp) DO UPDATE` (or the project’s existing `export_data_to_postgres` `on_conflict` pattern used in `FormatAndExportPrediction`) when writing the repeated prediction — if the product definition of REPEAT is “refresh same logical slot.” +2. **Insert with the current batch timestamp:** Copy numeric/status fields from the last row but set `timestamp` to the **new** batch instant (e.g. the workflow’s `last_timestamp` / slice timestamp). This matches `assert_repeat`, which does not assert on `timestamp`. +3. **Test-only change (weakest):** Relax assertions or change seed data — only if production behavior is “duplicate key is expected” (unlikely). + +Solução a ser aplicada: +Alterar o teste para usar o timestamp da ultima execucao do batch, ao inves do timestamp do primeiro batch. +2. **Insert with the current batch timestamp:** Copy numeric/status fields from the last row but set `timestamp` to the **new** batch instant (e.g. the workflow’s `last_timestamp` / slice timestamp). This matches `assert_repeat`, which does not assert on `timestamp`. + +### Tests affected + +- `test_scenario_2_1_3_input_gate_triggers_repeat` +- `test_scenario_2_2_3_transform_gate_triggers_repeat` +- `test_scenario_2_3_3_predict_gate_triggers_repeat` + +--- + +## Passing areas (sanity check) + +All scenarios in `test_child_workflows_e2e.py`, `test_minimal_retrain.py`, `test_minio_offload.py`, `test_predictions_batch_format_export.py`, `test_predictions_batch_main_workflow.py` (except the three REPEAT cases above), and `test_simple_metrics.py` **passed** in this run. + +--- + +## Suggested order of work + +1. Fix **drift `value` NULL** — unblocks two high-value drift E2Es and may clarify the chunk-seconds scenario if exports start succeeding consistently. +2. Fix **`repeat_last_prediction` uniqueness** — unblocks three prediction-process E2Es; implementation likely lives in **`sientia_do`** Postgres activities, not in this repo. +3. Revisit **`test_drift_chunk_period_seconds`** data volume / expectations after drift export is stable. diff --git a/docs/sientia_model_drift_jensen_shannon_change.md b/docs/sientia_model_drift_jensen_shannon_change.md new file mode 100644 index 0000000..099ca36 --- /dev/null +++ b/docs/sientia_model_drift_jensen_shannon_change.md @@ -0,0 +1,55 @@ +# Specification: `sientia_model` — Jensen–Shannon drift (`DriftAnalysis`) + +This document describes what **`sientia_model.analytics.drift_analysis.DriftAnalysis`** should change so downstream consumers (e.g. Laborious `calculate_drift` → Postgres `drift_metrics.value NOT NULL`) no longer receive **NaN** for Jensen–Shannon on valid finite data. + +## Scope + +- **File:** `sientia_model/analytics/drift_analysis.py` +- **Method:** `_jensen_shannon_distance(self, ref: np.ndarray, cur: np.ndarray, bins: int = 20) -> float` +- **Callers:** `detect_univariate_drift` uses this for the `jensen_shannon` method; results are written to `value` in the drift dataframe. + +## Problem + +The current implementation: + +1. Builds bin edges from **`ref`** only: `np.histogram(ref, bins=bins, density=True)`. +2. Builds the chunk histogram with **`density=True`** on the same edges: `np.histogram(cur, bins=edges, density=True)`. + +When **every** value in **`cur`** falls **outside** the closed support implied by those edges (typical case: production chunk drifted above the reference max or below the reference min), NumPy yields **all-zero counts** for `cur`. With **`density=True`**, normalization does **0/0**, producing **NaN** for the whole histogram, which propagates to **`float('nan')`** in `detect_univariate_drift` → **SQL NULL** where `value` is `NOT NULL`. + +This appears in Laborious when: + +- `model_config.target` excludes the main target column from univariate features, so another feature (e.g. `sensor_2`) is compared chunk-by-chunk against the full reference series for that feature. +- Reference and current ranges do not overlap for some chunks (strong drift or different scaling). + +Other methods (`kolmogorov_smirnov`, `wasserstein`) do not use this histogram+density path, so they can stay finite while **Jensen–Shannon** alone becomes null. + +## Required behavior + +1. **Finite output** for finite `ref` and `cur` after removing non-finite values, whenever both sides have **at least one** usable sample. +2. **Explicit handling of out-of-range chunk mass:** probability mass from `cur` that does not fall into any bin defined from `ref` must still be represented (so the chunk distribution sums to 1), analogous to NannyML’s continuous JS approach (tail / “leftover” mass). +3. **Missing values:** drop `NaN` from `ref` and `cur` before computing. If either side is **empty** after that, return **`float('nan')`** (callers may filter or map; schema may still forbid null — product decision outside this spec). + +## Recommended algorithm (replace current body) + +1. `ref = np.asarray(ref, float); cur = np.asarray(cur, float)`. +2. `ref = ref[~np.isnan(ref)]; cur = cur[~np.isnan(cur)]`. +3. If `ref.size == 0` or `cur.size == 0`: return `float('nan')`. +4. `hist_ref, edges = np.histogram(ref, bins=bins)` — **counts**, not `density=True`. +5. `p = hist_ref.astype(float) / ref.size` (reference bin probabilities). +6. `hist_cur, _ = np.histogram(cur, bins=edges)`; `q = hist_cur.astype(float) / cur.size`. +7. `leftover = 1.0 - float(np.sum(q))`. If `leftover > 1e-15` (tolerance for float noise), append **`leftover`** to `q` and **`0.0`** to `p` so both remain proper discrete distributions over the same extended support. +8. Apply small smoothing (existing module constant `EPSILON` is fine): add `EPSILON` to `p` and `q`, renormalize each to sum 1. +9. `m = 0.5 * (p + q)`; compute symmetric JS via KL terms as today, e.g. `inner = 0.5 * (sum(p*log(p/m)) + sum(q*log(q/m)))`. +10. Return `sqrt(max(inner, 0.0))` to guard against tiny negative `inner` from floating-point error. + +## Non-goals / notes + +- **Numerical parity** with the old `density=True` implementation is not required; parity with **NannyML** or **scipy** JS is desirable but optional. The priority is **finite, interpretable** drift when the chunk is outside the reference histogram range. +- **Multivariate** drift in the same file is unchanged by this spec. +- **Tests** in `sientia_model` should cover: (a) chunk entirely above reference max, (b) entirely below reference min, (c) overlapping range, (d) `ref` or `cur` all-NaN after cleaning. + +## Reference (external) + +- NannyML continuous JS uses count-based bin probabilities and a **leftover** mass bin; see `ContinuousJensenShannonDistance` in `nannyml/drift/univariate/methods.py` (`_calculate`, `leftover = 1 - np.sum(...)`). +- Historical NaN-in-reference issue: [NannyML#339](https://github.com/NannyML/nannyml/issues/339) / [#340](https://github.com/NannyML/nannyml/pull/340) (orthogonal to out-of-range mass, but relevant for input cleaning). diff --git a/e2e/conftest.py b/e2e/conftest.py index d54bb69..123cf4a 100644 --- a/e2e/conftest.py +++ b/e2e/conftest.py @@ -1,7 +1,5 @@ """Pytest configuration and fixtures for E2E tests.""" -import asyncio -import time from concurrent.futures import ThreadPoolExecutor from pathlib import Path from unittest.mock import MagicMock @@ -16,7 +14,6 @@ from testcontainers.postgres import PostgresContainer from temporalio.testing import WorkflowEnvironment from temporalio.worker import Worker -from e2e.opc_test_server import OpcE2ETestServer from laborious.activities.activities import Activities from laborious.workflows.drift import Drift from laborious.workflows.minimal_retrain import MinimalRetrain @@ -464,140 +461,3 @@ async def temporal_worker_minimal_retrain(temporal_test_env, test_activities): yield worker -def _connect_activities_to_opc_server(activities: Activities, server_url: str) -> None: - """ - Initialize OPC repositories and block until the E2E server session is ready. - - Runs synchronously (typically via ``asyncio.to_thread``) so the asyncua test - server event loop is not blocked during ``Client.connect()``. - - Args: - activities (Activities): Worker activities under test. - server_url (str): ``opc.tcp://`` URL from ``OpcE2ETestServer``. - """ - activities.init_opc() - repo = activities.opc_repository['1'] - deadline = time.monotonic() + 30.0 - while time.monotonic() < deadline: - if repo._session_ready.is_set(): - return - connected, _ = repo.connect() - if connected: - return - time.sleep(0.5) - raise RuntimeError(f'Could not connect OpcRepository to OPC E2E server at {server_url}') - - -def _build_e2e_opc_config(server_url: str) -> dict[str, dict]: - """ - OPC server config for E2E Activities pointing at an in-process asyncua server. - - Args: - server_url (str): ``opc.tcp://`` endpoint from ``OpcE2ETestServer``. - - Return: - dict: ``opc_config`` payload for ``Activities`` (server id ``1``). - """ - return { - '1': { - 'id': '1', - 'server_name': 'e2e_opcua', - 'url': server_url, - 'server_uri': server_url, - 'cert_path': None, - 'private_key_path': None, - 'server_cert_path': None, - 'reconnection_interval': 0, - } - } - - -@pytest_asyncio.fixture -async def opc_e2e_server(): - """In-process asyncua server with writable prediction/confidence nodes.""" - server = OpcE2ETestServer() - await server.start() - await asyncio.sleep(0.5) - try: - yield server - finally: - await server.stop() - - -@pytest_asyncio.fixture(scope='function') -async def test_activities_real_opc( - postgres_container, - minio_container, - mock_logger, - notification_handler, - metrics_controller, - mlflow_repository_stub, - plugin_store_stub, - pi_web_api_client_stub, - opc_e2e_server: OpcE2ETestServer, -): - """Activities with real OpcRepository connected to the in-process OPC UA server.""" - minio_client = minio_container.get_client() - if not minio_client.bucket_exists('test-bucket'): - minio_client.make_bucket('test-bucket') - minio_port = minio_container.get_exposed_port(9000) - - activities = Activities( - postgres_config={ - 'host': 'localhost', - 'port': int(postgres_container.get_exposed_port(5432)), - 'user': postgres_container.username, - 'password': postgres_container.password, - 'dbname': postgres_container.dbname, - 'min_connections': 1, - 'max_connections': 5, - }, - plugin_store=plugin_store_stub, - minio_config={ - 'endpoint_url': f'localhost:{minio_port}', - 'access_key': 'minioadmin', - 'secret_key': 'minioadmin', - 'default_bucket': 'test-bucket', - 'retention_hours': 24, - 'secure': False, - }, - opc_config=_build_e2e_opc_config(opc_e2e_server.url), - pi_web_api_config={ - 'base_url': 'http://localhost:8080', - 'auth_type': 'bearer', - 'auth_token': 'test_token', - }, - logger=mock_logger, - notification_handler=notification_handler, - metrics_controller=metrics_controller, - mlflow_repository=mlflow_repository_stub, - ) - activities.pi_web_api_client = pi_web_api_client_stub - await asyncio.to_thread(_connect_activities_to_opc_server, activities, opc_e2e_server.url) - try: - yield activities - finally: - await asyncio.to_thread(_teardown_real_opc_activities, activities) - - -def _teardown_real_opc_activities(activities: Activities) -> None: - """Disconnect OPC sessions and shut down activities (sync, for asyncio.to_thread).""" - for opc_repo in activities.opc_repository.values(): - opc_repo.disconnect() - activities.shutdown() - - -@pytest_asyncio.fixture(scope='function') -async def temporal_worker_real_opc(temporal_test_env, test_activities_real_opc): - """Temporal worker using real OpcRepository against the in-process OPC UA server.""" - with ThreadPoolExecutor(max_workers=32) as activity_executor: - async with Worker( - temporal_test_env.client, - task_queue='test-queue', - workflows=[PredictionsBatch, PredictionProcess, FormatAndExportPrediction], - activities=_worker_activity_list(test_activities_real_opc), - activity_executor=activity_executor, - ) as worker: - yield worker - - diff --git a/e2e/scenarios.md b/e2e/scenarios.md index d4b96ba..5fd681b 100644 --- a/e2e/scenarios.md +++ b/e2e/scenarios.md @@ -10,27 +10,6 @@ It is a functional reference of scenario behavior, inputs, and expected outcomes - Tests run under `e2e/` and are marked with `@pytest.mark.integration`. - PostgreSQL and MinIO are provisioned with testcontainers. - `test_minio_offload.py` uses real MinIO I/O; other scenario suites may use stubs/mocks for optional outputs. -- Real OPC UA scenarios use `@pytest.mark.opc` and an in-process asyncua server (`e2e/test_opc_real_server.py`). - -### Local validation - -Use the existing project virtualenv and the shared `validate` script for unit/quality gates; run E2E separately (Docker required). - -```bash -source ./venv/bin/activate - -# Auto-fix + static checks (no pytest) -validate --fix --project-name=laborious - -# Full unit + quality gate -validate --project-name=laborious - -# E2E (integration) -pytest e2e/ --override-ini testpaths=e2e -m integration - -# E2E (real OPC server only) -pytest e2e/test_opc_real_server.py --override-ini testpaths=e2e -m opc -``` --- @@ -106,15 +85,11 @@ Source: `e2e/test_predictions_batch_prediction_process.py` - Workflow exits without export. #### 2.1.3 REPEAT with history -**Summary**: Prior prediction is reused; outcome depends on batch vs history timestamp. +**Summary**: Prior prediction is reused. **Description**: - Input gate returns `REPEAT`. -- `repeat_last_prediction` inserts a row using ``last_timestamp`` from the batch payload (max timestamp in `laborious_data` for the query), not the copied row’s timestamp. - -**Tests**: -- **Collision**: batch `last_timestamp` equals the historical prediction row’s `timestamp` → Postgres `unique_model_id_timestamp` violation; workflow fails; still one row. -- **Distinct batch time**: laborious rows are stamped later than the historical prediction → second row inserted; same prediction fields as the first (see `assert_repeat`). +- `repeat_last_prediction` path is executed using existing historical row. #### 2.1.4 REPEAT without history **Summary**: Repeat requested but no previous prediction exists. @@ -137,8 +112,6 @@ Source: `e2e/test_predictions_batch_prediction_process.py` #### 2.2.3 REPEAT on transform response error **Summary**: Transform response error triggers repeat-last-prediction path. -**Tests**: Same timestamp collision vs distinct batch timestamp as §2.1.3 (`*_fails` / `*_inserts_second_row`). - #### 2.2.4 STOP on transform content NaN **Summary**: Content gate (`NAN_VALUES`) blocks on all-NaN transform payload. @@ -153,8 +126,6 @@ Source: `e2e/test_predictions_batch_prediction_process.py` #### 2.3.3 REPEAT on predict response error **Summary**: Predict response error routes to repeat-last-prediction. -**Tests**: Same timestamp collision vs distinct batch timestamp as §2.1.3 (`*_fails` / `*_inserts_second_row`). - ### 2.4.1 Priority Conflict Resolution **Summary**: Deterministic selection when multiple filters produce different flags. @@ -216,30 +187,6 @@ Source: `e2e/test_predictions_batch_format_export.py` - Workflow completes. - Prediction persisted with PI error confidence and descriptive comment. -#### 3.2.4 OPC session / channel error (confidence 14) -**Summary**: Tier-1 `BadSessionIdInvalid` (or equivalent session error) degrades the prediction without failing the workflow. - -**Sources**: -- Mock: `e2e/test_predictions_batch_format_export.py::test_scenario_3_2_4_opc_session_bad_mock` -- Real server: `e2e/test_opc_real_server.py::test_scenario_3_2_4_opc_session_bad_real_server` (`@pytest.mark.opc`) - -**Expected Outcome**: -- Workflow completes. -- `prediction_confidence` is 14. -- Comments contain `OPC UA session/channel error: BadSessionIdInvalid`. - -#### 3.2.5 OPC write blocked during reconnect (confidence 14) -**Summary**: While reconnect holds the repository connection lock, writes fail fast with `reconnect_in_progress`. - -**Sources**: -- Mock: `e2e/test_predictions_batch_format_export.py::test_scenario_3_2_5_opc_reconnect_in_progress_mock` -- Real server: `e2e/test_opc_real_server.py::test_scenario_3_2_5_opc_write_blocked_during_reconnect_real_server` (`@pytest.mark.opc`) - -**Expected Outcome**: -- Workflow completes. -- `prediction_confidence` is 14. -- Comments contain `OPC UA reconnect in progress`. - ### 3.3.1 Combined Optional Outputs (PI + OPC) **Summary**: Both external output channels are enabled together. @@ -349,33 +296,15 @@ first 30% of target rows as reference. - The workflow surfaces the `ValueError` ("Invalid chunk period: ..."). - No rows are persisted. -#### D.4.3a Insufficient drift metrics while target has rows -**Summary**: Laborious raises when the merged drift table is empty but the -target window is non-empty (`Insufficient drift data:` + notification -`MODEL_METRICS_DRIFT_INSUFFICIENT_DATA`). - -**Description**: -- The e2e patches `ModelMetrics.get_drift_metrics` to return an empty - DataFrame, simulating a ``sientia_model`` path that emits no rows. - -**Expected Outcome**: -- Workflow fails; no rows in `sientia_data.drift_metrics`. - -#### D.4.3b `chunk_period='s'` preserves seconds in `chunk_start_date` -**Summary**: Target data includes sub-minute spacing across several minutes; -the activity uses `chunk_period='s'`. +#### D.4.3 `chunk_period='s'` preserves seconds in `chunk_start_date` +**Summary**: Target data spans two minutes with samples at second-30 +boundaries; the activity is configured with `chunk_period='s'`. **Expected Outcome**: - At least one persisted `chunk_start_date` carries `seconds=30`, proving that the analyzer chunked at sub-minute granularity and the ISO-text serialization preserved the boundary. -**Note**: The e2e patches `DriftAnalysis._chunk_dataframe` to **skip empty** -`pd.Grouper(freq='s')` buckets. The stock implementation iterates every -second between min/max timestamps, producing empty chunks and NaT rows that -`calculate_drift` filters away entirely. The durable fix belongs in -`sientia_model`. - --- ## 6. Simple Metrics Workflow Scenarios diff --git a/tests/conftest.py b/tests/conftest.py index eb23f39..e46ac68 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -3,19 +3,6 @@ import os from sientia_do.temporal.activities.postgres_sync import Postgres -def _noop_postgres_del(_self): - """ - Unit tests use MagicMock metrics controllers; postgres_sync.Postgres.__del__ calls - close() during GC and triggers async shutdown. Explicit ``close()`` is covered in tests. - """ - return None - - -Postgres.__del__ = _noop_postgres_del # type: ignore[method-assign] - -from sientia_do.temporal.activities.postgres_sync import Postgres - - def _noop_postgres_del(_self): """ Unit tests use MagicMock metrics controllers; postgres_sync.Postgres.__del__ calls