Update BaseActivity initialization in manager classes to disable error counter metric
- Modified the initialization of DataManager, IngestorManager, OpcManager, and ResourceManager to include 'set_error_counter_metric=False' for improved error handling.
SIENTIAPDE-1174 Add pod_id to Ingestor metadata for improved tracking
- Included 'pod_id' in the metadata dictionary of the Ingestor class to enhance tracking capabilities.
Update requirements and environment configurations
- Changed the sientia-dataops-library dependency to a relative path in requirements.txt.
- Updated image tag to 0.3.0 in values.yaml.
- Renamed HTTP_SERVER_PORT to HTTP_METRICS_PORT in app.py and updated its default value to 9090.
- Added TAG_WRITTEN_COUNT metric in metrics.py to track writing processes.
- Incremented TAG_WRITTEN_COUNT in DataManager upon successful data insertion.
SIENTIAPDE-1163
Refactor Ingestor and DataManager classes to enhance metadata handling
- Updated Ingestor class to include additional metadata fields: 'workflow_name' and 'schema_name'.
- Modified DataManager to accept metadata during initialization.
- Adjusted unit tests to validate the new metadata structure and ensure proper functionality across various managers.
- Removed unused logger initialization in tests for cleaner code.
Refactor logging statements in Ingestor and app modules to use f-strings for improved readability and consistency. Updated error and info logs to enhance clarity in the Ingestor class.
Refactor Ingestor and Manager classes to use CoreNotificationHandler and improve metadata handling
- Updated Ingestor class to initialize logger using get_logger and replaced notification handler initialization with metadata dictionary.
- Refactored DataManager, IngestorManager, OpcManager, and ResourceManager classes to inherit from BaseActivity, allowing for consistent logger and notification handler usage.
- Enhanced notification handling by integrating metadata into notification methods across various managers.
- Added .vscode/ to .gitignore to exclude VSCode configuration files.
Update values.yaml for image tag and GITHUB_BRANCH
- Updated image tag to "0.2.4" for the latest version.
- Changed GITHUB_BRANCH value to "SIENTIAPDE-1163-alterar-dinamica-de-notificacoes-para-usar-o-mongodb-ao-inves-do-kafka".
- Removed .vscode/settings.json as it is no longer needed.
Update requirements and refactor Ingestor class for MongoDB integration
- Updated sientia-dataops-library dependency version to 1.3.0 in requirements.txt.
- Refactored Ingestor class to use CoreNotificationHandler and added MongoDB connection parameters to improve configuration management.
Refactor OpcManager to improve node subscription handling
- Simplified the address node assignment in the OpcManager class by directly using a local variable instead of an instance variable.
- Enhanced debug logging to provide clearer insights into the nodes being subscribed to.
Refactor OPC Manager metrics logging for improved readability
- Reformatted metrics logging statements in the OpcManager class for better readability and consistency.
- Enhanced debug logging to provide clearer insights into subscription management and connection status.
Enhance debug logging in Ingestor class to track changes in managed tags
- Added debug logging to capture changes between new and old managed tags in the Ingestor class, improving traceability of subscription management.
Update GITHUB_BRANCH in values.yaml and remove debug log in DataManager
- Changed GITHUB_BRANCH value to "SIENTIAPDE-1110-criar-testes-e-2-e" in values.yaml.
- Removed unnecessary debug log for skipping messages in DataManager class.
Enhance logging in Ingestor class for subscription management
- Added debug logging for subscribing to new slots, resubscribing to modified slots, and unsubscribing from removed slots in the Ingestor class.
- Updated DataManager to simplify MongoDB connection logic by removing retry loop and adding logging for connection attempts.
- Enhanced IngestorManager to accept MongoDB connection parameters and export_to_kafka flag as part of its initialization.
- Updated requirements.txt to use the latest version of the sientia-dataops-library.
- Changed log level from info to debug for skipping messages to Kafka topic.
- Added debug log for successful message insertion into MongoDB collection.
- Introduced EXPORT_TO_KAFKA variable in values.yaml to control Kafka export behavior.
- Updated Ingestor class to parse and set the export_to_kafka attribute based on the environment variable.
- Introduced EXPORT_TO_KAFKA environment variable in Ingestor class.
- Refactored DataManager to conditionally initialize Kafka producer based on export setting.
- Updated IngestorManager to pass Redis configuration as a dictionary.
- Enhanced error handling and logging for Kafka message publishing.
Update ingestor and data manager for improved shutdown handling and logging
- Incremented project version in quality gate configuration.
- Commented out ingestor service in docker-compose for clarity.
- Enhanced main loop in app.py to handle exit signals and exceptions.
- Added shutdown methods in Ingestor and DataManager classes for graceful resource cleanup.
- Updated unit tests to validate shutdown behavior and exception handling.
- Introduced coverage configuration to omit specific files.
- Changed logging statements in Ingestor and OpcManager from info to debug level to reduce log verbosity and improve performance.
- This adjustment aids in better log management while maintaining necessary traceability for debugging purposes.
- Added logging statements to OpcManager to track data change notifications and cycle count resets for monitored tags.
- Improved traceability of events related to OPC cycles, aiding in debugging and monitoring.
Enhance IngestorManager and related tests with notification handling integration
- Added notification_handler parameter to IngestorManager and OpcManager initialization for improved event handling.
- Updated unit tests to include notification_handler in the Ingestor and IngestorManager instantiation, ensuring proper coverage of new functionality.
- Refactored test cases to validate the integration of notification handling across components.
Enhance ingestor and manager classes with notification handling
- Integrated NotificationHandler into Ingestor, DataManager, IngestorManager, and OpcManager for improved error reporting and monitoring.
- Updated methods to send notifications on critical events such as Kafka publishing errors, OPC connection issues, and cycle count warnings.
- Refactored related tests to ensure coverage of new notification functionalities and validate integration with existing components.
- Improved logging and error handling across the system to enhance traceability and operational insights.
