gitea_admin/sientia-dataops-orchestrator_temporal
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Merge fix/SIENTIAPDE-2013 into fix/SIENTIAPDE-2149

Browse Source 
Brings in the review-feedback fixes (batch-size default, requirements-local,
docstring trim) from PR #43 so #44 stays consistent with its base branch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

# Conflicts:
#	CLAUDE.md
This commit is contained in:
Eduardo Rios
2026-09-22 16:52:30 -03:00
parent 0edb2b5be0 d3104d66ab
commit c85c3440ed
5 changed files with 7 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
orchestrator/activities/mongo_db.py
View File

@@ -451,15 +451,9 @@ class MongoDB(SientiaMonitoring):
filtering by timestamp to enable incremental data processing. It
handles connection management and provides comprehensive error reporting.
SIENTIAPDE-2013: the query is capped at ``notification_queue_batch_size``
documents (hard cap per run, no drain loop) and sorted by ``timestamp``
ascending, so a capped run always returns the *oldest* pending documents
first. That matters because the caller advances its incremental cursor to
the max timestamp of whatever this returns -- without the ascending sort,
a capped batch could skip over older documents that never get picked up
on a later run. This backstops a cold start, backlog buildup, or
notification storm from fetching an unbounded (and potentially
multi-megabyte) payload in one query.
The query is capped at ``notification_queue_batch_size`` documents and sorted by
``timestamp`` ascending, so a capped run returns the oldest pending documents first --
the caller advances its cursor only to what was actually returned, so nothing is skipped.
Args:
input_data (dict[str, Any]): Activity input parameters.

2
orchestrator/utils/connectors_config.py
View File

@@ -33,7 +33,7 @@ def build_mongodb_config():
'connection_string': connection_string,
'database_name': getenv('MONGODB_DATABASE_NAME', 'sientia'),
'ttl_index_seconds': int(getenv('MONGODB_TTL_INDEX_HOURS', '1')) * 3600,
'notification_queue_batch_size': int(getenv('NOTIFICATION_QUEUE_BATCH_SIZE', '5000')),
'notification_queue_batch_size': int(getenv('NOTIFICATION_QUEUE_BATCH_SIZE', '0')),
}