SIENTIAPDE-2149: model_ids allow-list filter for receiver groups
New, separate model_id-based filter distinct from the existing ignore_models_id deny-list (SIENTIAPDE-2012): a group configuring model_ids only receives notifications from those models. Both axes apply together (AND) when a group sets both. Design decisions made while implementing (matching the levels/ ignore_models_id split from SIENTIAPDE-2011/2012, not yet confirmed with the team beyond what's in the ticket): - Query-time filter: union of every active group's model_ids, applied only when ALL active groups set one -- mirrors levels' exact union-or-nothing logic in _compute_query_time_filter. Combines cleanly with ignore_models_id's intersection $nin on the same model_id field. - Per-group filter: extended _check_levels_and_ignore_models_id (now _check_levels_and_model_filters) in slot_manager.py, and the inline condition in formatters.py's filter_notification_reports. - Empty/missing model_ids means no restriction, same convention as levels and ignore_models_id. Covered by new unit tests (slot_manager, formatters, _compute_query_time_filter) and two new e2e tests against real Mongo/Redis/Temporal: per-group allow-list interacting with another unrestricted group, and the query-time union excluding a model outside every active group's allow-list. Not yet verified on the real cluster -- blocked on the dev cluster's package index not auto-syncing new sientia_do releases (separate issue, being investigated for SIENTIAPDE-2013). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -347,6 +347,7 @@ def default_receiver_group(
|
||||
levels: list[str] | None = None,
|
||||
contents: list[str] | None = None,
|
||||
ignore_models_id: list[str] | None = None,
|
||||
model_ids: list[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Minimal active receiver_groups document."""
|
||||
return {
|
||||
@@ -356,6 +357,7 @@ def default_receiver_group(
|
||||
'levels': levels or ['ERROR', 'WARNING', 'INFO'],
|
||||
'contents': contents or ['core_alerts', 'persistent_alerts', 'reports'],
|
||||
'ignore_models_id': ignore_models_id or [],
|
||||
'model_ids': model_ids or [],
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -68,6 +68,12 @@ the query-time union/intersection across active groups (SIENTIAPDE-2011, see
|
||||
`levels`/`ignore_models_id`, even when the query-time filter widened the fetch
|
||||
to satisfy another active group's broader configuration.
|
||||
|
||||
### A.2.2 Model_id allow-list
|
||||
|
||||
Receiver group `model_ids` (SIENTIAPDE-2149) honored: a group configuring it only
|
||||
receives notifications from those models, even when another active group with
|
||||
no such restriction widens the query-time fetch to include every model.
|
||||
|
||||
### A.3.1 Empty queue
|
||||
|
||||
No notifications → no SMTP, no Postgres row.
|
||||
@@ -99,6 +105,7 @@ No notifications → no SMTP, no Postgres row.
|
||||
- No prior Redis timestamp → all matching notifications returned, max timestamp stored.
|
||||
- Prior timestamp → only newer notifications returned.
|
||||
- Query-time filter (SIENTIAPDE-2011) → union of active groups' `levels`, intersection of their `ignore_models_id`; a model ignored by only one of several active groups is not excluded from the fetch.
|
||||
- Model_id allow-list union at query time (SIENTIAPDE-2149) → when every active group sets `model_ids`, the fetch is restricted to the union of every group's allow-list; a model outside that union is never fetched at all, even for a group that would otherwise want it via a different axis.
|
||||
- Empty Mongo → no Redis timestamp write.
|
||||
- Batch size cap (SIENTIAPDE-2013) → with 5 pending notifications and a batch size of 2, each run returns exactly the next-oldest 2 (sorted ascending by timestamp), and the Redis cursor advances only to what was actually returned, so three successive runs drain the backlog in order instead of skipping anything.
|
||||
- Batch size 0 means unlimited (SIENTIAPDE-2013) → the configured default (`0`) does not cap the query; 12 pending notifications all come back in one run, verified against a real MongoDB instance rather than assumed from pymongo's documented convention.
|
||||
|
||||
@@ -182,6 +182,73 @@ async def test_scenario_a_2_1_per_group_levels_and_ignore_models_id(
|
||||
assert 'model-x' not in combined
|
||||
|
||||
|
||||
@pytest.mark.e2e
|
||||
@pytest.mark.asyncio
|
||||
async def test_scenario_a_2_2_model_ids_allow_list(
|
||||
temporal_env: WorkflowEnvironment,
|
||||
orchestrator_worker,
|
||||
stub_workers,
|
||||
mongo_uri: str,
|
||||
smtp_server,
|
||||
):
|
||||
"""
|
||||
A.2.2 (SIENTIAPDE-2149): a group configuring `model_ids` only receives
|
||||
notifications from those models, on top of the existing query-time/per-group
|
||||
filtering (SIENTIAPDE-2011/2012).
|
||||
|
||||
group-allow (model_ids=['model-a']) coexists with group-everything (no
|
||||
model_ids restriction). Because the query-time filter only restricts to an
|
||||
allow-list union when EVERY active group has one, group-everything's lack
|
||||
of a model_ids restriction means the query itself fetches both models --
|
||||
the per-group filter added by this ticket is what narrows group-allow back
|
||||
down to model-a only.
|
||||
"""
|
||||
seed_receiver_groups(
|
||||
mongo_uri,
|
||||
E2E_DATABASE,
|
||||
[
|
||||
default_receiver_group(
|
||||
group_name='group-allow', contents=['core_alerts'], model_ids=['model-a']
|
||||
),
|
||||
default_receiver_group(group_name='group-everything', contents=['core_alerts']),
|
||||
],
|
||||
)
|
||||
seed_notifications(
|
||||
mongo_uri,
|
||||
E2E_DATABASE,
|
||||
[
|
||||
default_notification(
|
||||
notification_id='err-a', level='ERROR', model_name='model-a', model_id='model-a'
|
||||
),
|
||||
default_notification(
|
||||
notification_id='err-b',
|
||||
level='ERROR',
|
||||
model_name='model-b',
|
||||
model_id='model-b',
|
||||
timestamp=datetime(2024, 6, 1, 12, 1, 0, tzinfo=UTC),
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
await start_and_await_workflow(
|
||||
temporal_env.client,
|
||||
Alerts.run,
|
||||
load_scenario_input('alerts_happy_path.json'),
|
||||
make_workflow_id('alerts-2149'),
|
||||
)
|
||||
|
||||
# One email per group -- group-allow got only err-a, group-everything got both.
|
||||
assert len(smtp_server.messages) == 2
|
||||
bodies = [extract_html_body(message) for message in smtp_server.messages]
|
||||
|
||||
# Distinguish the two emails by content rather than order: only
|
||||
# group-everything's email mentions model-b.
|
||||
allow_body = next(b for b in bodies if 'model-b' not in b)
|
||||
everything_body = next(b for b in bodies if 'model-b' in b)
|
||||
assert 'model-a' in allow_body
|
||||
assert 'model-a' in everything_body
|
||||
|
||||
|
||||
@pytest.mark.e2e
|
||||
@pytest.mark.asyncio
|
||||
async def test_scenario_a_3_1_empty_queue(
|
||||
|
||||
@@ -205,6 +205,57 @@ async def test_load_package_query_time_filter_union_and_intersection(
|
||||
assert ids == {'error-included', 'warning-included', 'modely-included'}
|
||||
|
||||
|
||||
@pytest.mark.e2e
|
||||
@pytest.mark.asyncio
|
||||
async def test_load_package_query_time_filter_model_ids_union(
|
||||
temporal_env: WorkflowEnvironment,
|
||||
orchestrator_worker,
|
||||
stub_workers,
|
||||
mongo_uri: str,
|
||||
redis_client: Redis,
|
||||
):
|
||||
"""
|
||||
SIENTIAPDE-2149: query-time filter unions active groups' `model_ids`
|
||||
allow-list when every active group sets one, excluding any model outside
|
||||
that union from the fetch entirely -- not just from a single group's
|
||||
per-group slice.
|
||||
|
||||
group_a wants only model-a, group_b wants only model-b -- both restrict
|
||||
levels to ERROR so only the model_ids axis is under test. model-c is
|
||||
outside the union of both allow-lists and must never be fetched.
|
||||
"""
|
||||
seed_receiver_groups(
|
||||
mongo_uri,
|
||||
E2E_DATABASE,
|
||||
[
|
||||
default_receiver_group(group_name='group_a', levels=['ERROR'], model_ids=['model-a']),
|
||||
default_receiver_group(group_name='group_b', levels=['ERROR'], model_ids=['model-b']),
|
||||
],
|
||||
)
|
||||
seed_notifications(
|
||||
mongo_uri,
|
||||
E2E_DATABASE,
|
||||
[
|
||||
default_notification(notification_id='a-included', level='ERROR', model_id='model-a'),
|
||||
default_notification(notification_id='b-included', level='ERROR', model_id='model-b'),
|
||||
default_notification(notification_id='c-excluded', level='ERROR', model_id='model-c'),
|
||||
],
|
||||
)
|
||||
|
||||
input_data = load_scenario_input('subworkflow_load_notification_package.json')
|
||||
_build_metadata(input_data)
|
||||
|
||||
result = await start_and_await_workflow(
|
||||
temporal_env.client,
|
||||
LoadNotificationPackage.run,
|
||||
input_data,
|
||||
make_workflow_id('load-pkg-model-ids-union'),
|
||||
)
|
||||
|
||||
ids = {n['notification_id'] for n in result['notification_package']}
|
||||
assert ids == {'a-included', 'b-included'}
|
||||
|
||||
|
||||
@pytest.mark.e2e
|
||||
@pytest.mark.asyncio
|
||||
async def test_load_package_empty_mongo_no_redis_write(
|
||||
|
||||
Reference in New Issue
Block a user