SIENTIAPDE-988
Enhance ingestor and manager classes with notification handling - Integrated NotificationHandler into Ingestor, DataManager, IngestorManager, and OpcManager for improved error reporting and monitoring. - Updated methods to send notifications on critical events such as Kafka publishing errors, OPC connection issues, and cycle count warnings. - Refactored related tests to ensure coverage of new notification functionalities and validate integration with existing components. - Improved logging and error handling across the system to enhance traceability and operational insights.
This commit is contained in:
@@ -2,6 +2,7 @@ from logging import Formatter, StreamHandler, getLogger
|
||||
from os import getenv
|
||||
|
||||
from ingestor.managers.ingestor_manager import IngestorManager
|
||||
from sientia_do.notifications.handlers import NotificationHandler
|
||||
|
||||
|
||||
class Ingestor:
|
||||
@@ -37,14 +38,27 @@ class Ingestor:
|
||||
self.poll_interval = int(getenv("POLL_INTERVAL", 5))
|
||||
|
||||
self.kafka_servers = kafka_servers.split(",")
|
||||
self.logger = None
|
||||
self.init_logger()
|
||||
self.notification_handler = NotificationHandler(
|
||||
servers=self.kafka_servers,
|
||||
logger=self.logger,
|
||||
project_name="OPC_INGESTOR",
|
||||
pipeline_name="-",
|
||||
trigger_name="-",
|
||||
model_name="-",
|
||||
model="-"
|
||||
)
|
||||
# build args for build notificarions components
|
||||
|
||||
# call build notifications components
|
||||
|
||||
def init_logger(self):
|
||||
"""
|
||||
Initializes a logger instance for the class.
|
||||
This method sets up a logger with a specified log level, a stream handler,
|
||||
and a formatter. The log level is determined by the environment variable
|
||||
"LOG_LEVEL", defaulting to "INFO" if not set. The logger is then attached
|
||||
This method sets up a logger with a specified log level, a stream handler,
|
||||
and a formatter. The log level is determined by the environment variable
|
||||
"LOG_LEVEL", defaulting to "INFO" if not set. The logger is then attached
|
||||
to the instance for use throughout the class.
|
||||
Attributes:
|
||||
self.logger (logging.Logger): The configured logger instance.
|
||||
@@ -105,7 +119,8 @@ class Ingestor:
|
||||
|
||||
self.ingestor_manager = IngestorManager(
|
||||
self.kafka_servers, self.redis_host, self.redis_port, self.lease_ttl,
|
||||
self.heartbeat_ttl, self.pod_id, self.poll_interval, self.logger, self.redis_username, self.redis_password
|
||||
self.heartbeat_ttl, self.pod_id, self.poll_interval, self.logger,
|
||||
self.notification_handler, self.redis_username, self.redis_password
|
||||
)
|
||||
|
||||
# Declare ingestor ative
|
||||
@@ -120,9 +135,9 @@ class Ingestor:
|
||||
def manage_no_slots(self, number_of_slots: int):
|
||||
"""
|
||||
Manages the scenario where there are no slots assigned to the ingestor.
|
||||
This method checks if the ingestor is active (i.e., has no managed tags)
|
||||
and if the number of available slots is greater than zero. If both
|
||||
conditions are met, it attempts to acquire a slot lease and handles
|
||||
This method checks if the ingestor is active (i.e., has no managed tags)
|
||||
and if the number of available slots is greater than zero. If both
|
||||
conditions are met, it attempts to acquire a slot lease and handles
|
||||
the acquired tags accordingly.
|
||||
Args:
|
||||
number_of_slots (int): The number of available slots.
|
||||
@@ -138,16 +153,16 @@ class Ingestor:
|
||||
|
||||
def manage_leases(self, available_slots: int, lacking_ingestors: int, slot_diff: int):
|
||||
"""
|
||||
Manages the allocation and deallocation of slot leases for ingestors based on
|
||||
Manages the allocation and deallocation of slot leases for ingestors based on
|
||||
the number of available slots, lacking ingestors, and slot differences.
|
||||
Args:
|
||||
available_slots (int): The number of slots currently available for allocation.
|
||||
lacking_ingestors (int): The number of ingestors that are active and without slots.
|
||||
slot_diff (int): The difference between the total slots and the required slots.
|
||||
Behavior:
|
||||
- If there are available slots and lacking ingestors, attempts to acquire slot leases
|
||||
- If there are available slots and lacking ingestors, attempts to acquire slot leases
|
||||
for the available slots and processes the acquired tags.
|
||||
- If there are no lacking ingestors but there are extra slots (slot_diff > 0),
|
||||
- If there are no lacking ingestors but there are extra slots (slot_diff > 0),
|
||||
releases the extra slot leases to ensure proper allocation.
|
||||
Logs:
|
||||
- Logs the number of available slots when attempting to acquire leases.
|
||||
@@ -223,3 +238,6 @@ class Ingestor:
|
||||
|
||||
# Update opc servers
|
||||
self.ingestor_manager.update_slot_config()
|
||||
|
||||
# Check OPC cycles
|
||||
self.ingestor_manager.check_opc_servers_integrity()
|
||||
|
||||
Reference in New Issue
Block a user