SIENTIAPDE-1084
Remove deprecated files and enhance documentation - Deleted `coverage.sh`, `docker-compose.yaml`, `Dockerfile`, and simulator-related files to streamline the project structure. - Updated `README.md` to provide a comprehensive overview of the OPC Ingestor, including features, architecture, installation, usage, and troubleshooting. - Enhanced docstrings across various classes and methods in the `ingestor` module for better clarity and maintainability. - Improved Prometheus metrics documentation in `metrics.py` to ensure proper monitoring and observability of the system.
This commit is contained in:
@@ -13,6 +13,50 @@ import ingestor.metrics as metrics
|
||||
|
||||
|
||||
class IngestorManager(BaseActivity):
|
||||
"""
|
||||
Central coordinator for managing OPC data ingestion operations.
|
||||
|
||||
The IngestorManager orchestrates the interaction between different components:
|
||||
- DataManager: Handles data persistence and Kafka export
|
||||
- OPC Managers: Manage individual OPC UA server connections
|
||||
- ResourceManager: Coordinates slot leasing and load balancing
|
||||
|
||||
This class implements a slot-based architecture where:
|
||||
- Each slot represents a collection of OPC tags from one or more servers
|
||||
- Slots are distributed across multiple ingestor instances for load balancing
|
||||
- Dynamic slot allocation ensures optimal resource utilization
|
||||
|
||||
Key Responsibilities:
|
||||
- Slot lease management and distribution
|
||||
- OPC server connection lifecycle management
|
||||
- Tag subscription coordination
|
||||
- System health monitoring and integrity checks
|
||||
- Load balancing across multiple ingestor instances
|
||||
|
||||
Args:
|
||||
kafka_servers (str): Comma-separated list of Kafka server addresses
|
||||
redis_data (dict): Redis connection parameters (host, port, username, password)
|
||||
lease_ttl (int): Time-to-live for slot leases in seconds
|
||||
heartbeat_ttl (int): Time-to-live for heartbeat signals in seconds
|
||||
poll_interval (int): Main loop polling interval in seconds
|
||||
mongo_connection_string (str): MongoDB connection string
|
||||
mongo_database (str): MongoDB database name
|
||||
metadata (dict): Application metadata for notifications and tracking
|
||||
logger (Logger): Logger instance for application logging
|
||||
notification_handler (NotificationHandler): Handler for sending notifications
|
||||
export_to_kafka (bool): Whether to export data to Kafka
|
||||
|
||||
Attributes:
|
||||
data_manager (DataManager): Manages data persistence and Kafka export
|
||||
opc_managers (dict): Dictionary of OPC managers keyed by server name
|
||||
resource_manager (ResourceManager): Manages Redis-based resource coordination
|
||||
number_of_slots (int): Total number of slots configured in the system
|
||||
poll_interval (int): Main loop polling interval
|
||||
managed_tags (dict): Currently managed tags organized by slot
|
||||
opc_servers (dict): OPC server configurations
|
||||
metadata (dict): Application metadata
|
||||
"""
|
||||
|
||||
def __init__(self,
|
||||
kafka_servers: str, redis_data: dict,
|
||||
lease_ttl: int, heartbeat_ttl: int,
|
||||
@@ -61,6 +105,10 @@ class IngestorManager(BaseActivity):
|
||||
async def initialize_opc_from_config(self, server_config: dict) -> OpcManager | None:
|
||||
"""
|
||||
Initializes an OPC Manager instance using the provided server configuration.
|
||||
|
||||
This method creates and configures an OPC Manager for a specific OPC UA server,
|
||||
establishing the connection and preparing it for tag subscriptions.
|
||||
|
||||
Args:
|
||||
server_config (dict): A dictionary containing the OPC server configuration.
|
||||
Expected keys include:
|
||||
@@ -70,11 +118,15 @@ class IngestorManager(BaseActivity):
|
||||
- 'cert_path' (str, optional): Path to the client certificate file.
|
||||
- 'private_key_path' (str, optional): Path to the private key file.
|
||||
- 'server_cert_path' (str, optional): Path to the server certificate file.
|
||||
data_manager (DataManager): An instance of the DataManager to handle data operations.
|
||||
logger (Logger): A logger instance for logging messages.
|
||||
|
||||
Returns:
|
||||
OpcManager | None: An initialized OpcManager instance if successful,
|
||||
otherwise None if an error occurs during initialization.
|
||||
|
||||
Raises:
|
||||
Exception: If OPC manager initialization fails, the error is logged and
|
||||
a notification is sent, but the method returns None to allow
|
||||
the system to continue operating with other servers.
|
||||
"""
|
||||
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user