SIENTIAPDE-1084

Remove deprecated files and enhance documentation

- Deleted `coverage.sh`, `docker-compose.yaml`, `Dockerfile`, and simulator-related files to streamline the project structure.
- Updated `README.md` to provide a comprehensive overview of the OPC Ingestor, including features, architecture, installation, usage, and troubleshooting.
- Enhanced docstrings across various classes and methods in the `ingestor` module for better clarity and maintainability.
- Improved Prometheus metrics documentation in `metrics.py` to ensure proper monitoring and observability of the system.

ingestor/managers/data_manager.py

@@ -14,6 +14,38 @@ import os
 
 
 class DataManager(BaseActivity):
+    """
+    Manages data persistence and export operations for the OPC Ingestor.
+
+    The DataManager is responsible for:
+    - Storing OPC data in MongoDB for historical analysis and persistence
+    - Exporting data to Kafka for real-time streaming and downstream processing
+    - Managing database connections and ensuring data integrity
+    - Providing data access interfaces for other components
+
+    The manager supports both MongoDB and Kafka operations, with Kafka export
+    being optional and configurable. It implements retry logic for connection
+    failures and provides comprehensive error handling and notification.
+
+    Args:
+        kafka_servers (str): Comma-separated string of Kafka server addresses
+        mongo_connection_string (str): MongoDB connection string
+        mongo_database (str): MongoDB database name
+        export_to_kafka (bool): Whether to enable Kafka export functionality
+        metadata (dict): Application metadata for notifications and tracking
+        logger (Logger): Logger instance for application logging
+        notification_handler (NotificationHandler): Handler for sending notifications
+
+    Attributes:
+        pod_id (str): Pod identifier for metrics labeling
+        kafka_producer (KafkaProducer): Kafka producer instance for data export
+        export_to_kafka (bool): Whether Kafka export is enabled
+        connection_string (str): MongoDB connection string
+        database (str): MongoDB database name
+        mongo_client (MongoClient): MongoDB client instance
+        metadata (dict): Application metadata
+    """
+
     def __init__(
         self,
         kafka_servers: str,
@@ -25,15 +57,32 @@ class DataManager(BaseActivity):
         notification_handler: NotificationHandler,
     ) -> None:
         """
-        Initializes the DataManager instance with a Kafka producer.
-        This constructor attempts to establish a connection to the specified Kafka servers
-        and initializes a Kafka producer for sending messages. It retries the connection
-        up to 3 times if the Kafka servers are unavailable.
+        Initializes the DataManager instance with Kafka and MongoDB connections.
+
+        This constructor attempts to establish connections to the specified services:
+        1. Kafka: Initializes producer with retry logic (up to 3 attempts)
+        2. MongoDB: Establishes connection and verifies server availability
+
+        The initialization process includes:
+        - Kafka producer setup with JSON serialization
+        - MongoDB client initialization and connection testing
+        - Metrics recording for connection status
+        - Error handling with notifications
+
         Args:
-            kafka_servers (str): A comma-separated string of Kafka server addresses.
-            logger (Logger): A logger instance for logging messages.
+            kafka_servers (str): Comma-separated string of Kafka server addresses
+            mongo_connection_string (str): MongoDB connection string
+            mongo_database (str): MongoDB database name
+            export_to_kafka (bool): Whether to enable Kafka export
+            metadata (dict): Application metadata
+            logger (Logger): Logger instance
+            notification_handler (NotificationHandler): Notification handler
+
         Raises:
             NoBrokersAvailable: If the connection to Kafka servers fails after 3 attempts.
+
+        Metrics:
+            - KAFKA_CONNECTION_STATUS: Set to 1 on successful connection, 0 on failure
         """
 
         self.pod_id = os.getenv("HOSTNAME", "localhost")