Commit Graph

619 Commits

Author SHA1 Message Date
e887372a76 Merge pull request #58 from Aignosi/feature/SIENTIAPDE-2189-min-samples-guard
feat(simple_metrics): min samples guard + model-level, window-aware breach alert (SIENTIAPDE-2189)
2026-10-09 14:31:36 -03:00
ci-dependencies-fetcher[bot]
a9a2d7af80 Revert "chore: bump version to 0.9.0"
This reverts commit 694915e94f.
2026-10-09 17:26:58 +00:00
github-actions[bot]
694915e94f chore: bump version to 0.9.0 2026-10-09 17:26:21 +00:00
6fce847fce Merge pull request #59 from Aignosi/feature/SIENTIAPDE-2191-adaptive-state
feat(simple_metrics): adaptive state on the pipelines doc (SIENTIAPDE-2191)
2026-10-09 14:25:56 -03:00
PedroHMCosme
6cc69b0f93 test(simple_metrics): build the input outside pytest.raises (Sonar python:S5778) (SIENTIAPDE-2191) 2026-10-09 10:36:32 -03:00
PedroHMCosme
e1b94b1da4 fix(simple_metrics): skip invalid interval_minutes and restart a malformed healthy_streak (SIENTIAPDE-2191)
- interval_minutes is untyped Mongo config: a string or null now warns and
  skips the adaptive state instead of failing every run (TypeError).
- A hand-edited healthy_streak (null, text, inf) restarts the count and keeps
  escalated, instead of raising on every run.
- README: step 4 runs on every schedule, so a MongoDB failure fails the run.
2026-10-09 10:06:05 -03:00
PedroHMCosme
a812acd4e9 refactor(activities): move Mongo activities into MongoActivities (SIENTIAPDE-2191)
Rename PredictionMetadata (prediction_metadata.py) to MongoActivities (mongo.py) and move update_simple_metrics_adaptive_state there from ModelMetrics, so both Mongo-backed activities share one class and repository (PR #59 review, agreed with the class author). write_prediction_metadata is unchanged; the pure hysteresis helpers stay in model_metrics.py. References in activities.py, docstrings and tests updated.
2026-10-09 09:50:37 -03:00
PedroHMCosme
d265e3328c fix(simple_metrics): fail the run explicitly when the adaptive state update fails (SIENTIAPDE-2191)
Drop the try/except around update_simple_metrics_adaptive_state in the workflow and the broad except in the activity, so Mongo failures fail the run and show up in the Temporal metrics (PR #59 review). Invalid frequency_alert stays a warning + skip (the orchestrator ignores it too) and a malformed adaptive_state is read as not escalated.
2026-10-08 17:45:44 -03:00
PedroHMCosme
877a0c7543 fix(simple_metrics): warn when the pipelines doc is missing; document Mongo requirements (SIENTIAPDE-2191) 2026-10-08 08:48:08 -03:00
PedroHMCosme
ac30f4c464 fix(simple_metrics): update adaptive state even without thresholds so removed thresholds recover (SIENTIAPDE-2191) 2026-10-08 08:44:35 -03:00
PedroHMCosme
44116f3f3f docs(simple_metrics): document min samples guard and adaptive state step (SIENTIAPDE-2191) 2026-10-08 08:44:35 -03:00
PedroHMCosme
844a29ed27 test(e2e): adaptive state is escalated in pipelines on threshold breach (SIENTIAPDE-2191) 2026-10-08 08:44:35 -03:00
PedroHMCosme
e05ec1c0a9 feat(simple_metrics): update adaptive state after each run, empty windows included (SIENTIAPDE-2191) 2026-10-08 08:44:34 -03:00
PedroHMCosme
34f56c7416 feat(simple_metrics): write adaptive_state to the pipelines doc (SIENTIAPDE-2191) 2026-10-08 08:43:09 -03:00
PedroHMCosme
cb97f7344f feat(simple_metrics): add adaptive-state hysteresis and recovery-run rules (SIENTIAPDE-2191) 2026-10-08 08:43:09 -03:00
PedroHMCosme
f503fd0610 refactor(simple_metrics): extract threshold_breaches helper (SIENTIAPDE-2191) 2026-10-08 08:43:09 -03:00
PedroHMCosme
9f8237a46e fix(simple_metrics): ignore invalid simple_metrics_min_samples; tighten tests and docs (SIENTIAPDE-2189) 2026-10-08 08:43:01 -03:00
PedroHMCosme
ccf4dc86e2 feat(simple_metrics): skip threshold alerting below simple_metrics_min_samples (SIENTIAPDE-2189) 2026-10-07 15:26:57 -03:00
PedroHMCosme
9c89cfbf8e feat(simple_metrics): model-level, window-aware threshold breach alert (SIENTIAPDE-2189) 2026-10-07 15:18:35 -03:00
PedroHMCosme
196b1aa708 Merge release/SIENTIAPDE-1646 into feature/SIENTIAPDE-1986-simple-metrics-rewrite
Requirements follow the release; only sientia_model>=0.12.0 (RegressionMetrics),
model-library@0.13.1 and scikit-learn==1.8.0 (local) are kept from this branch.
2026-10-06 08:19:37 -03:00
github-actions[bot]
79605db260 chore: bump version to 0.9.0-rc2 2026-10-02 17:23:19 +00:00
4ae76cfd83 Merge pull request #55 from Aignosi/rc/SIENTIAPDE-2173
Rc/sientiapde 2173 - Mover repeat_last_prediction do sientia-dataops-library para o laborious, com suporte a metadata_id
2026-10-02 14:22:56 -03:00
Carla Pinto
14b27e4c30 rc/SIENTIAPDE-2173- fix: add missing metadata_id column to e2e/db_schema.sql 2026-10-02 18:57:56 +02:00
Carla Pinto
09349b6274 Merge branch 'release/SIENTIAPDE-1646' into rc/SIENTIAPDE-2173 2026-10-02 18:43:27 +02:00
Carla Pinto
2ace55c241 rc/SIENTIAPDE-2173- refactor: move repeat_last_prediction from sientia-dataops-library into Storage, add metadata_id 2026-10-02 18:41:18 +02:00
github-actions[bot]
6981d09ac6 chore: bump version to 0.9.0-rc1 2026-10-02 13:24:13 +00:00
860bce1ba3 Merge pull request #54 from Aignosi/feature-rc/SIENTIAPDE-2164
Feature rc/sientiapde 2164 - Prediction Metadata — Metadado rico no input_gate
2026-10-02 10:23:50 -03:00
Carla Pinto
c190267292 fetaure-rc/SIENTIAPDE-2164- fix: drop star-unpack guard on input_gate result, back to fixed 4-tuple 2026-10-02 15:12:03 +02:00
Carla Pinto
206b47dfae feature-rc/SIENTIAPDE-2164- fix: never let a metadata write failure block the prediction 2026-10-01 19:14:24 +02:00
Carla Pinto
9c628e70a6 feature-rc/SIENTIAPDE-2164 - refactor: centralize prediction metadata write in a dedicated activity; REPEAT now writes metadata too 2026-10-01 17:46:20 +02:00
Carla Pinto
db32450c68 feature-rc/SIENTIAPDE-2164 -fix: apply Temporal replay-safe unpacking and guard against missing filter counters (review findings) 2026-09-30 19:40:03 +02:00
Carla Pinto
a01763b69c feature-rc/SIENTIAPDE-2164 - apply ruff format to test_gates.py (CI quality-gate) 2026-09-30 18:03:28 +02:00
Carla Pinto
e5ebf243bd Merge branch 'release/SIENTIAPDE-1646' into feature-rc/SIENTIAPDE-2164 2026-09-30 17:52:12 +02:00
Carla Pinto
cff81e3594 feature-rc/SIENTIAPDE-2164 -Enrich input_gate metadata (filter/policy/counter) and write to Mongo on CONTINUE 2026-09-30 17:49:16 +02:00
github-actions[bot]
b5a54147f5 chore: bump version to 0.9.0
Some checks failed
Quality gate / quality-gate (push) Has been cancelled
2026-09-29 15:33:26 +00:00
ci-dependencies-fetcher[bot]
150e2f7259 Revert "chore: bump version to 0.8.0"
This reverts commit e5cea1ef96.
2026-09-29 14:22:49 +00:00
github-actions[bot]
e5cea1ef96 chore: bump version to 0.8.0 2026-09-29 14:22:15 +00:00
Carla Fernandes
08b2986f81 Merge pull request #53 from Aignosi/feature/SIENTIAPDE-2162
feature/SIENTIAPDE-2162 Prediction Metadata — Collection prediction_metadata no MongoDB
2026-09-29 16:21:48 +02:00
Carla Pinto
8b69b2cb29 feature/SIENTIAPDE-2162 - Inject mongo_repository into gates and add best-effort prediction metadata write 2026-09-28 19:33:24 +02:00
9920cd61c2 SIENTIAPDE-1646
Add tests for retrain, drift, storage, OPC repo, worker paths

- covers csv lookup fallback, drift insufficient-data passthrough,
  timestamp tz normalization, storage close without engine, OPC
  watchdog/write_value fallbacks and reconnect reporting, and
  legacy runtime install in worker main
2026-09-28 11:53:01 -03:00
d27f6573fe SIENTIAPDE-1646
Simplify merge in retrain: expose timestamp explicitly for join

Refactor evaluation_data merge to reset both frames' index into a
"timestamp" column before joining, instead of merging on index then
copying it out afterward. Same validate='one_to_one' safety check,
fewer intermediate steps.
2026-09-25 12:45:19 -03:00
f323da7b56 SIENTIAPDE-1646
Harden retrain merge and OPC write against silent failures

- pandas merge now uses validate='one_to_one' so duplicated timestamps
  from wrapper.retrain fail loudly instead of silently multiplying rows
- OPC write_data raises OpcClientNotInitializedError instead of a bare
  assert when client is torn down between validation and write, so
  callers get a reported error instead of an AssertionError
- add regression test covering the torn-down client path

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-25 12:27:17 -03:00
5c3c9292fc SIENTIAPDE-1646
Fix flaky shutdown assertions in test suite

GC-collected instances during tests also invoke patched shutdown,
making assert_called_once_with flaky. Count calls for this instance
specifically instead of asserting total call count.
2026-09-25 12:10:18 -03:00
9fb81ef556 SIENTIAPDE-1646
Pin sqlalchemy<2.1 to fix psycopg driver break; use tmp_path in retrain

- sqlalchemy 2.1 defaults postgresql:// URLs to psycopg (v3), but sientia_do
  only ships psycopg2-binary, breaking with "No module named psycopg"
- Replace hardcoded 'tmp' mkdtemp mock with pytest tmp_path fixture in
  retrain_model tests to avoid writing to a fake path
2026-09-25 11:58:55 -03:00
26e37e0a0a SIENTIAPDE-1646
Fix quality-gate repo list: remove space, swap mlops-library for model-l

Correct repo list breaks scan otherwise: space in name mismatched dep, wrong package name (mlops-library doesn't exist / renamed to model-library).
2026-09-25 11:03:53 -03:00
116330fc3f SIENTIAPDE-1646
Use requirements.txt for quality-gate scan instead of light variant

Switch dependency scan input from requirements-light.txt to requirements.txt
so quality gate checks full dependency set, not partial one.
2026-09-25 11:00:18 -03:00
00b6382a88 SIENTIAPDE-1646
Simplify bandit nosec comments and mock metrics_controller with real spe

- drift.py/simple_metrics.py: move nosec rationale to a code comment above
  the query instead of an inline suppression comment, keeping bandit happy
  with a plain `# nosec`
- tests: replace bare `MagicMock()` with `MagicMock(spec=MetricsController)`
  for metrics_controller across activities/repository tests so mocks fail
  fast on attribute typos instead of silently accepting any call
2026-09-25 10:57:46 -03:00
4c51e7c2b3 SIENTIAPDE-1646
SIENTIAPDE-1646 Merge origin/main into release/SIENTIAPDE-1646

Bring main up to date while keeping this branch's sync execution, PluginStore
runtime and sientia_model (SientiaMLflowRepository/SientiaModel) structure.

Resolution decisions:
- OPC: keep the sync asyncua.sync implementation; port main's server_uri
  validation, e2e race fix (lock held until released) and the new unit tests
  converted to sync.
- Model import workflow (.sientia bundles), joblib/transform flavors and the
  local MLFlowRepository (model_repository.py) are dropped, with their tests,
  docs, metrics, env vars and connectors config.
- Worker: drift/simple_metrics stay pinned to the 'core' runtime; port main's
  HOSTNAME pod id, logger-based prometheus startup and sys.exit after finally.
- values.yaml removed, following main (SIENTIAPDE-1945).
- Deps: keep sientia_model; take main's fsspec/s3fs/botocore/boto3 pins and
  sientia_do>=1.12.2; drop import-only deps (pynacl, cryptography, pymongo,
  python-dotenv, pycaret, joblib, gcsfs).
- e2e: keep sientia_data schema and scenario input files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 10:45:43 -03:00
8ef7ed800a SIENTIAPDE-2071: chore: remove dead Makefile
Some checks failed
Quality gate / quality-gate (push) Has been cancelled
Card 16 of the SIENTIAPDE-2052 epic. This repo is a Flow B dataops
worker, it does not build its own image at all anymore, it runs off
a shared runtime image with code cloned from Gitea at deploy time. A
per repo image build Makefile pointing at aignosi.azurecr.io is dead
weight, no workflow calls make. Safe to remove.
2026-09-15 00:08:37 -03:00
github-actions[bot]
ac67d5b447 chore: bump version to 0.7.2 2026-09-11 15:19:06 +00:00