Add e2e scenario for drift window and chunk_period
- Add scenario 1.8.1: two drift pipelines on frequency 1m. One has no
interval_minutes and an invalid chunk_period='hour', so it expects
interval=2, chunk_period='min' and one DRIFT_CHUNK_PERIOD_INVALID
notification. The other sets interval_minutes=60 and chunk_period='s',
and both are forwarded unchanged with no warning (SIENTIAPDE-2212).
- Add a _schedule_input helper that decodes the workflow input of a
laborious-namespace schedule.
- Reformat the ensure_namespace_search_attributes signature in
conftest.py to a single line.
SIENTIAPDE-2212-drift-janela-e-chunk-period: Add configurable chunk period and dynamic window calculation for drift, with warnings for invalid periods.
reusable-deploy-dataops.yml encadeia release-tarball -> dataops-module-release.yml,
que pede pull-requests:read; GitHub valida a interseção de permissions em todo o
workflow_call encadeado, independente do if: que gate o job. Removido por engano
em 586a5b9, achando que só existia por causa do trigger pull_request antigo.
[skip ci]
Caller workflow had no permissions block, so GitHub's permission-
intersection rule capped the nested reusable-version-bump.yml call
to read-only, breaking dispatch ("requesting 'contents: write', but
is only allowed 'contents: read'").
[skip ci]
New workflow bumps pyproject.toml's version on PR merge, calling the
shared reusable-version-bump.yml (SIENTIAPDE-2217 version-bump reform) —
zero build/image/chart/deploy side effects. deploy.yml (workflow_dispatch-
only) will read this committed value instead of calculating one.
[skip ci]
Removes auto-deploy-on-PR-merge trigger and the old PR-merge gate. Deploy now
triggers only via workflow_dispatch, with chart_ref required (no default) as
the literal sientia-helm-chart branch bumped and deployed — hard error if it
doesn't exist, no silent fallback to development/main.
- .github/workflows/deploy.yml: drop pull_request trigger, add chart_ref
input, branch_name now github.ref_name, drop pull-requests:read permission
- Delete .github/workflows/require-chart-ref-label.yml
[skip ci]
sientia_do 1.12.5 pins the psycopg2 driver in the Postgres engine URL.
Without it, rebuilding the image pulls SQLAlchemy 2.1.x, whose default
'postgresql://' driver is psycopg (v3), and the worker crashes at
startup with "ModuleNotFoundError: No module named 'psycopg'".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the review-feedback fixes (batch-size default, requirements-local,
docstring trim) from PR #43 so #44 stays consistent with its base branch.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
# Conflicts:
# CLAUDE.md
- NOTIFICATION_QUEUE_BATCH_SIZE default reverted to 0 (unlimited) -- the
existing behavior before this ticket, so deployments must opt in to the
cap rather than having it silently applied at 5000
- requirements-local.txt: bump sientia_do pin to 1.12.3 to match requirements.txt
- Trim the load_latest_data docstring addition so it reads as documentation,
not a changelog entry
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
levels and model_ids (SIENTIAPDE-2149) both computed a per-group union with
identical "union only if every active group restricts, else unrestricted"
logic. Extracting _union_when_all_restrict removes the duplicated for/if
block for each axis, addressing the SonarQube quality gate failure on this
PR (cognitive complexity).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New, separate model_id-based filter distinct from the existing
ignore_models_id deny-list (SIENTIAPDE-2012): a group configuring
model_ids only receives notifications from those models. Both axes
apply together (AND) when a group sets both.
Design decisions made while implementing (matching the levels/
ignore_models_id split from SIENTIAPDE-2011/2012, not yet confirmed
with the team beyond what's in the ticket):
- Query-time filter: union of every active group's model_ids, applied
only when ALL active groups set one -- mirrors levels' exact
union-or-nothing logic in _compute_query_time_filter. Combines
cleanly with ignore_models_id's intersection $nin on the same
model_id field.
- Per-group filter: extended _check_levels_and_ignore_models_id (now
_check_levels_and_model_filters) in slot_manager.py, and the inline
condition in formatters.py's filter_notification_reports.
- Empty/missing model_ids means no restriction, same convention as
levels and ignore_models_id.
Covered by new unit tests (slot_manager, formatters,
_compute_query_time_filter) and two new e2e tests against real
Mongo/Redis/Temporal: per-group allow-list interacting with another
unrestricted group, and the query-time union excluding a model outside
every active group's allow-list.
Not yet verified on the real cluster -- blocked on the dev cluster's
package index not auto-syncing new sientia_do releases (separate issue,
being investigated for SIENTIAPDE-2013).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The library's release pipeline bumped a patch version (1.12.3), not the
minor bump (1.13.0) the feat: commit prefixes would normally produce --
correcting the pin here to match what was actually published.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Vitor asked whether limit=0 really means "no limit" on the Mongo side
and whether that had actually been tested. It's pymongo's own
documented convention, but nothing in the suite exercised it against a
real MongoDB with enough documents to mean anything -- the existing
tests only ever seed 2-5. Seeds 12 (more than any other test in this
file) with the batch size left at its config default (0) and asserts
all 12 come back in one run, against real testcontainers Mongo.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Removed scenarios documented with no backing test (1.2.1, 1.3.2, 1.4.1,
A.1.3) and corrected three descriptions that claimed assertions the
tests don't actually make (1.1.1's "Redis slots written", 1.3.1's
"timestamps inserted", 1.5.1's "orphan schedules removed") to match
what's really verified. A.2.1 stays as-is here -- it's already accurate
for this branch's real per-group filtering test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Also fixes the test_activities fixture, which built mongodb_config by
hand and was missing the new required notification_queue_batch_size key
-- every e2e test would have hit a KeyError without this.
New test seeds 5 notifications with a batch size of 2 and confirms three
successive runs return them oldest-first in order (n1,n2 / n3,n4 / n5),
proving the cursor only advances to what was actually returned rather
than skipping documents a capped query didn't fetch. Verified against
real testcontainers Mongo/Redis/Temporal, not mocks.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A cold start or notification storm had no backstop: load_latest_data
fetched the entire matching backlog in one query. Confirmed live during
SIENTIAPDE-2012's cluster verification -- an empty Redis cursor plus a
real recurring error let a single run's query exceed Temporal's 4MB
message limit and get stuck retrying forever.
Threads a new notification_queue_batch_size config (env var
NOTIFICATION_QUEUE_BATCH_SIZE, default 5000) through to the repository's
find(), hard cap per run, no drain loop -- the incremental
last_data_timestamp cursor picks up any remainder next run, per the
parent ticket's design decision. Sorts by timestamp ascending so a
capped run always returns the oldest pending documents first; without
that, a capped batch has no guarantee of hitting the oldest documents,
and the cursor would advance past whatever it skipped.
Requires sientia_do>=1.13.0 (adds limit/sort params to
MongoDBRepository.find in both the sync and async client) -- bumped the
pin here per the parent ticket's decision to do it in the same PR.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
filter_notification_alerts tripped Sonar's cognitive-complexity gate
(16 > 15 allowed) from its nested loop/if structure. Split into
_filter_group_notifications (per-group notification selection) and
_determine_alert_type (Redis TTL classification), leaving the public
method as a simple loop over receiver groups. Behavior and call order
are unchanged -- verified against the existing filter_notification_alerts
unit tests by inspection (sientia_do isn't installable in this local
environment, so the real suite can't run here; CI will confirm).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The parameter-level annotation on _check_keys_and_notif_id alone
wasn't enough -- mypy needs the type at the empty list's declaration
site inside filter_notification_alerts. Also reformatted the helper's
now-longer signature to satisfy ruff.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes CI formatting failure introduced by the manual Sonar cognitive-
complexity fix (extracting the ignore-list/dedup check into its own
helper).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes CI formatting failure introduced by the manual Sonar cognitive-
complexity fix (extracting the levels/ignore_models_id check into its
own helper).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
