SIENTIAPDE-1222
Enhance README and activities with advanced notification filtering features - Updated README to include new intelligent notification filtering capabilities, including TTL-based duplicate prevention and ignore lists. - Enhanced `Formatters` activity to support notification filtering for scheduled reports and added detailed filtering logic. - Improved `SlotManager` to implement advanced notification filtering with persistent alert detection and group-based filtering. - Updated `Alerts` workflow to incorporate intelligent filtering for real-time error notifications, ensuring efficient alert delivery.
This commit is contained in:
vitor-aignosi
@@ -19,12 +19,20 @@ topic_separator = "\n ========== \n"
|
||||
|
||||
class Formatters(BaseActivity):
|
||||
"""
|
||||
Schedule and slot configuration formatting activity.
|
||||
Schedule and slot configuration formatting and notification filtering activity.
|
||||
|
||||
This class provides formatting operations for schedules and OPC slots,
|
||||
converting pipeline configurations into Temporal-compatible formats
|
||||
and managing slot distribution across active ingestors for optimal
|
||||
resource utilization.
|
||||
This class provides comprehensive formatting operations for schedules and OPC slots,
|
||||
converting pipeline configurations into Temporal-compatible formats, managing slot
|
||||
distribution across active ingestors, and implementing notification filtering for
|
||||
scheduled reports.
|
||||
|
||||
Key Features:
|
||||
- Pipeline schedule configuration formatting (scouter, predictions_batch, minimal_retrain)
|
||||
- OPC slot distribution across active ingestors
|
||||
- Notification filtering for comprehensive reports
|
||||
- Group-based report filtering with ignore list support
|
||||
- Resource optimization algorithms
|
||||
- Configuration validation and transformation
|
||||
|
||||
Args:
|
||||
scouter_namespace (str): Scouter workflow namespace
|
||||
@@ -45,8 +53,16 @@ class Formatters(BaseActivity):
|
||||
@activity.defn(name="process_schedules")
|
||||
async def process_schedules(self, input_data: dict[str, Any]) -> dict[str, Any]:
|
||||
"""
|
||||
Process schedules. Generates a schedule config dictionary
|
||||
based on the input data workflow type.
|
||||
Process pipeline configurations into Temporal-compatible schedule configurations.
|
||||
|
||||
This method transforms pipeline configurations from MongoDB into properly formatted
|
||||
Temporal schedule configurations, organizing them by workflow type (scouter and
|
||||
laborious) and applying the appropriate configuration builders for each pipeline type.
|
||||
|
||||
Pipeline Types Supported:
|
||||
- scouter: Data collection workflows with OPC tag configurations
|
||||
- predictions_batch: ML prediction workflows with OPC write configurations
|
||||
- minimal_retrain: Model retraining workflows with SQL query configurations
|
||||
|
||||
Args:
|
||||
- input_data (dict[str, Any]): The input data containing
|
||||
@@ -629,17 +645,34 @@ class Formatters(BaseActivity):
|
||||
@activity.defn(name="filter_notification_reports")
|
||||
async def filter_notification_reports(self, input_data: dict[str, Any]) -> dict[str, Any]:
|
||||
"""
|
||||
Filters notification reports based on sending configurations and notification package.
|
||||
Filter notifications for comprehensive scheduled reports.
|
||||
|
||||
This method filters notifications of all levels (ERROR, WARNING, INFO, DEBUG)
|
||||
for scheduled report generation. Unlike alert filtering, this method does not
|
||||
implement TTL-based duplicate prevention since reports are meant to provide
|
||||
comprehensive coverage of system activity within a time window.
|
||||
|
||||
Filtering Logic:
|
||||
- Processes all notification levels (not just ERROR)
|
||||
- Applies group-specific content filtering for "reports" type
|
||||
- Respects ignore lists for each receiver group
|
||||
- Prevents duplicate notifications within the same report
|
||||
- Groups notifications by receiver group configurations
|
||||
|
||||
Args:
|
||||
input_data (dict[str, Any]): The input data containing:
|
||||
- metadata (dict): Metadata for logging purposes.
|
||||
- notification_package (list): The package of notifications to filter.
|
||||
- sending_configs (list): The configurations for sending notifications.
|
||||
Each config should have 'group_name', 'contents', and optionally 'ignore' fields.
|
||||
- metadata (dict): Workflow execution metadata for logging
|
||||
- notification_package (list): All notifications to filter (any level)
|
||||
- sending_configs (list): Receiver group configurations with:
|
||||
- group_name (str): Name of the receiver group
|
||||
- contents (list): Content types to include (must contain "reports")
|
||||
- ignore (list, optional): Notification IDs to exclude
|
||||
|
||||
Returns:
|
||||
dict[str, Any]: The filtered receiver groups with their notifications.
|
||||
dict[str, Any]: Filtered receiver groups with their notifications, keyed by group_name.
|
||||
Each group contains:
|
||||
- All receiver group configuration fields
|
||||
- notifications (list): Filtered notifications for this group
|
||||
"""
|
||||
metadata = input_data['metadata']
|
||||
notification_package = input_data['notification_package']
|
||||
|
||||
@@ -16,12 +16,20 @@ with workflow.unsafe.imports_passed_through():
|
||||
|
||||
class SlotManager(Redis):
|
||||
"""
|
||||
Redis-based OPC slot management activity.
|
||||
Redis-based OPC slot management and notification filtering activity.
|
||||
|
||||
This class manages OPC server slots and notification processing through
|
||||
Redis operations. It provides functionality for loading, updating, and
|
||||
deleting OPC slots, managing active ingestors, and handling notification
|
||||
caching with timestamp management.
|
||||
This class manages OPC server slots and provides advanced notification
|
||||
filtering capabilities through Redis operations. It handles OPC slot
|
||||
lifecycle management, active ingestor tracking, and implements intelligent
|
||||
notification filtering with TTL-based duplicate prevention.
|
||||
|
||||
Key Features:
|
||||
- OPC slot loading, updating, and deletion
|
||||
- Active ingestor management
|
||||
- Notification filtering for alerts with TTL management
|
||||
- Persistent alert detection for ongoing issues
|
||||
- Notification caching with configurable expiration
|
||||
- Group-based filtering with ignore list support
|
||||
|
||||
Args:
|
||||
host (str): Redis server hostname
|
||||
@@ -42,10 +50,23 @@ class SlotManager(Redis):
|
||||
@activity.defn(name="load_opc_slots")
|
||||
async def load_opc_slots(self, input_data: dict[str, Any]) -> dict[str, Any]:
|
||||
"""
|
||||
Load all OPC slots from Redis
|
||||
Load all OPC slots from Redis for current system state assessment.
|
||||
|
||||
This method retrieves all OPC server slot configurations from Redis,
|
||||
which are used to determine current resource allocation and identify
|
||||
changes needed for pipeline orchestration.
|
||||
|
||||
Args:
|
||||
input_data (dict[str, Any]): Activity input containing metadata
|
||||
|
||||
Returns:
|
||||
dict[str, Any]: A dictionary of OPC slots
|
||||
dict[str, Any]: Dictionary of OPC slots keyed by server ID, where each slot contains:
|
||||
- Configuration parameters for OPC server connections
|
||||
- Active pipeline assignments
|
||||
- Resource allocation details
|
||||
|
||||
Raises:
|
||||
Exception: If Redis connection fails or data retrieval errors occur
|
||||
"""
|
||||
|
||||
metadata = input_data.get("metadata", {})
|
||||
@@ -313,18 +334,35 @@ class SlotManager(Redis):
|
||||
@activity.defn(name="filter_notification_alerts")
|
||||
async def filter_notification_alerts(self, input_data: dict[str, Any]) -> dict[str, Any]:
|
||||
"""
|
||||
Filter notification alerts based on sending configurations and notification package.
|
||||
Filter notification alerts with intelligent TTL-based duplicate prevention.
|
||||
|
||||
This method implements advanced notification filtering for ERROR-level alerts,
|
||||
preventing spam through TTL management and detecting persistent issues that
|
||||
require escalation. It applies user group-based filtering with configurable
|
||||
ignore lists and content policies.
|
||||
|
||||
Filtering Logic:
|
||||
- Checks Redis cache for recently sent notifications
|
||||
- Identifies "core_alerts" for new notifications
|
||||
- Detects "persistent_alerts" for ongoing issues beyond TTL
|
||||
- Applies group-specific content filtering and ignore lists
|
||||
- Prevents duplicate notifications within the same TTL window
|
||||
|
||||
Args:
|
||||
input_data (dict[str, Any]): The input data containing:
|
||||
- metadata (dict): Metadata for logging purposes.
|
||||
- notification_package (list): The package of notifications to filter.
|
||||
- sending_configs (list): The configurations for sending notifications.
|
||||
Each config should have 'group_name', 'contents', and optionally 'ignore' fields.
|
||||
- notification_ttl (int): Time to live for notifications in seconds.
|
||||
- metadata (dict): Workflow execution metadata for logging
|
||||
- notification_package (list): ERROR-level notifications to filter
|
||||
- sending_configs (list): Receiver group configurations with:
|
||||
- group_name (str): Name of the receiver group
|
||||
- contents (list): Alert types to include (core_alerts, persistent_alerts)
|
||||
- ignore (list, optional): Notification IDs to exclude
|
||||
- notification_ttl (int): Seconds before considering notification persistent
|
||||
|
||||
Returns:
|
||||
dict[str, Any]: The filtered receiver groups with their notifications.
|
||||
dict[str, Any]: Filtered receiver groups with their notifications, keyed by group_name.
|
||||
Each group contains:
|
||||
- All receiver group configuration fields
|
||||
- notifications (list): Filtered notifications for this group
|
||||
"""
|
||||
metadata = input_data['metadata']
|
||||
notification_package = input_data['notification_package']
|
||||
|
||||
@@ -9,22 +9,42 @@ with workflow.unsafe.imports_passed_through():
|
||||
|
||||
@workflow.defn(name="alerts")
|
||||
class Alerts:
|
||||
"""
|
||||
Alerts workflow for real-time error notification delivery.
|
||||
|
||||
This workflow processes ERROR-level notifications from the notification queue
|
||||
and sends immediate alerts to configured user groups. It implements intelligent
|
||||
filtering with TTL-based duplicate prevention and persistent alert detection
|
||||
for ongoing issues.
|
||||
"""
|
||||
|
||||
@workflow.run
|
||||
async def run(self, input_data: dict[str, Any]):
|
||||
"""
|
||||
Workflow to send alerts to the users
|
||||
Execute the alerts workflow for real-time error notification delivery.
|
||||
|
||||
This workflow loads ERROR-level notifications from MongoDB, applies
|
||||
intelligent filtering with TTL management to prevent alert spam,
|
||||
and sends immediate email alerts to configured receiver groups.
|
||||
|
||||
The workflow implements:
|
||||
- TTL-based duplicate prevention for notifications
|
||||
- Persistent alert detection for ongoing issues
|
||||
- User group-based filtering with ignore lists
|
||||
- Audit logging of alert delivery status
|
||||
|
||||
Args:
|
||||
input_data (dict[str, Any]): Input data. It contains the following keys:
|
||||
- schedule_name: str - Name of the schedule
|
||||
- notification_ttl: int - Period before consider some notification persistent
|
||||
- sent_ttl: int - Time to live for the sent notification
|
||||
input_data (dict[str, Any]): Workflow input parameters.
|
||||
Required fields:
|
||||
- schedule_name (str): Name of the alert schedule
|
||||
- notification_ttl (int): Seconds before considering notification persistent
|
||||
- sent_ttl (int): Time-to-live for sent notification cache
|
||||
|
||||
Returns:
|
||||
None
|
||||
None: Workflow completes without return value
|
||||
|
||||
Raises:
|
||||
Exception: If the workflow fails
|
||||
Exception: If alert processing or delivery fails
|
||||
"""
|
||||
metadata = {
|
||||
'metadata': {
|
||||
|
||||
Reference in New Issue
Block a user