SIENTIAPDE-1174 Add pod_id to Ingestor metadata for improved tracking
- Included 'pod_id' in the metadata dictionary of the Ingestor class to enhance tracking capabilities.
Update values.yaml to modify replica count and GITHUB_BRANCH environment variable
- Increased replicaCount from 1 to 3 for improved scalability.
- Updated GITHUB_BRANCH environment variable to SIENTIAPDE-1174-mapear-e-implementar-metricas-a-serem-criadas.
Update requirements and environment configurations
- Changed the sientia-dataops-library dependency to a relative path in requirements.txt.
- Updated image tag to 0.3.0 in values.yaml.
- Renamed HTTP_SERVER_PORT to HTTP_METRICS_PORT in app.py and updated its default value to 9090.
- Added TAG_WRITTEN_COUNT metric in metrics.py to track writing processes.
- Incremented TAG_WRITTEN_COUNT in DataManager upon successful data insertion.
Update dependencies and environment variables
- Updated sientia-dataops-library dependency version to 1.3.5 in requirements.txt.
- Changed image tag to 0.2.7 in values.yaml.
- Updated GITHUB_BRANCH environment variable to SIENTIAPDE-1172-criar-pipeline-de-alertas-orquestrador in values.yaml.
Update values.yaml to add metrics service configuration
- Introduced a new 'metrics' service with ClusterIP type, port 9090, and targetPort 9090.
- Updated the serviceMonitor port to reference the new 'metrics' service.
- Changed HTTP_SERVER_PORT environment variable value to 9090 to align with the new metrics service.
SIENTIAPDE-1163
Refactor Ingestor and DataManager classes to enhance metadata handling
- Updated Ingestor class to include additional metadata fields: 'workflow_name' and 'schema_name'.
- Modified DataManager to accept metadata during initialization.
- Adjusted unit tests to validate the new metadata structure and ensure proper functionality across various managers.
- Removed unused logger initialization in tests for cleaner code.
Update values.yaml to restructure service definitions
- Changed the service configuration to a nested structure under 'services'.
- Introduced 'api' and 'opc' services with individual settings, both initially disabled.
- Updated ports for the 'api' service to 4841 while keeping the 'opc' service at 4840.
Refactor logging statements in Ingestor and app modules to use f-strings for improved readability and consistency. Updated error and info logs to enhance clarity in the Ingestor class.
Refactor Ingestor and Manager classes to use CoreNotificationHandler and improve metadata handling
- Updated Ingestor class to initialize logger using get_logger and replaced notification handler initialization with metadata dictionary.
- Refactored DataManager, IngestorManager, OpcManager, and ResourceManager classes to inherit from BaseActivity, allowing for consistent logger and notification handler usage.
- Enhanced notification handling by integrating metadata into notification methods across various managers.
- Added .vscode/ to .gitignore to exclude VSCode configuration files.
Update values.yaml for image tag and GITHUB_BRANCH
- Updated image tag to "0.2.4" for the latest version.
- Changed GITHUB_BRANCH value to "SIENTIAPDE-1163-alterar-dinamica-de-notificacoes-para-usar-o-mongodb-ao-inves-do-kafka".
- Removed .vscode/settings.json as it is no longer needed.
Update requirements and refactor Ingestor class for MongoDB integration
- Updated sientia-dataops-library dependency version to 1.3.0 in requirements.txt.
- Refactored Ingestor class to use CoreNotificationHandler and added MongoDB connection parameters to improve configuration management.
Refactor test assertions for improved readability and consistency
- Updated assertions in test cases for DataManager and OpcManager to enhance readability by formatting long lines.
- Ensured that the expected behavior of metrics logging and message publishing is clearly defined in the tests.
Refactor OpcManager to improve node subscription handling
- Simplified the address node assignment in the OpcManager class by directly using a local variable instead of an instance variable.
- Enhanced debug logging to provide clearer insights into the nodes being subscribed to.
Refactor OPC Manager metrics logging for improved readability
- Reformatted metrics logging statements in the OpcManager class for better readability and consistency.
- Enhanced debug logging to provide clearer insights into subscription management and connection status.
Enhance debug logging in Ingestor class to track changes in managed tags
- Added debug logging to capture changes between new and old managed tags in the Ingestor class, improving traceability of subscription management.
Update GITHUB_BRANCH in values.yaml and remove debug log in DataManager
- Changed GITHUB_BRANCH value to "SIENTIAPDE-1110-criar-testes-e-2-e" in values.yaml.
- Removed unnecessary debug log for skipping messages in DataManager class.
Enhance logging in Ingestor class for subscription management
- Added debug logging for subscribing to new slots, resubscribing to modified slots, and unsubscribing from removed slots in the Ingestor class.
- Updated DataManager to simplify MongoDB connection logic by removing retry loop and adding logging for connection attempts.
- Enhanced IngestorManager to accept MongoDB connection parameters and export_to_kafka flag as part of its initialization.
- Updated requirements.txt to use the latest version of the sientia-dataops-library.
- Changed log level from info to debug for skipping messages to Kafka topic.
- Added debug log for successful message insertion into MongoDB collection.
- Introduced EXPORT_TO_KAFKA variable in values.yaml to control Kafka export behavior.
- Updated Ingestor class to parse and set the export_to_kafka attribute based on the environment variable.
- Introduced EXPORT_TO_KAFKA environment variable in Ingestor class.
- Refactored DataManager to conditionally initialize Kafka producer based on export setting.
- Updated IngestorManager to pass Redis configuration as a dictionary.
- Enhanced error handling and logging for Kafka message publishing.
