SIENTIAPDE-1193 Update values.yaml to include new environment variables for enhanced configuration
- Added new environment variables for improved deployment flexibility.
- Updated existing configurations to align with recent changes in the application structure.
Update requirements and refactor timestamp handling in DataManager
- Updated sientia-dataops-library version to 1.4.3 in requirements.txt.
- Refactored timestamp insertion in DataManager to use the now() function for improved consistency.
Update requirements and refactor datetime handling in OpcManager for timezone support
- Commented out the specific version of sientia-dataops-library in requirements.txt and added a local path.
- Updated GITHUB_BRANCH in values.yaml for consistency with current development.
- Refactored source_timestamp handling in OpcManager to include timezone information.
- Adjusted timestamp formatting in test cases to reflect the new datetime format with timezone.
Refactor logging levels in Ingestor and IngestorManager for improved clarity
- Changed debug log statements to info level in Ingestor class for better visibility of subscription and management actions.
- Updated log level for initializing OPC manager in IngestorManager to info for consistent logging practices.
Update Ingestor project name and metadata keys to lowercase for consistency
- Changed project_name, workflow_name, and schema_name from uppercase to lowercase in Ingestor class and corresponding unit tests.
- Enabled metrics service with updated configuration for Prometheus ServiceMonitor.
- Removed disabled API and OPC service configurations to streamline values.yaml.
- Updated helm upgrade command version to 0.5.0.
- Updated sientia-dataops-library version to 1.4.1 in requirements.txt.
- Changed image tag from 0.3.2 to 0.4.2 in values.yaml.
- Refactored logger imports from sientia_do.temporal.utils.logger to sientia_do.observability.logger in multiple manager files for improved observability.
Refactor test assertions in test_opc_manager.py for improved readability
- Updated the assertion syntax for notification calls in the test_check_opc_listenning_error_notification_and_retry function to use dictionary literals for clarity.
Update requirements and refactor manager classes to remove pod_id
- Updated the sientia-dataops-library dependency version to 1.3.8 in requirements.txt.
- Refactored IngestorManager, OpcManager, and ResourceManager classes to remove pod_id from their initialization and internal handling, enhancing code clarity and consistency.
- Adjusted unit tests to reflect the removal of pod_id, ensuring they remain functional and accurate.
Update BaseActivity initialization in manager classes to enable error counter metric
- Modified the initialization of DataManager, IngestorManager, OpcManager, and ResourceManager to include 'set_error_counter=True' for enhanced error tracking.
Update BaseActivity initialization in manager classes to disable error counter metric
- Modified the initialization of DataManager, IngestorManager, OpcManager, and ResourceManager to include 'set_error_counter_metric=False' for improved error handling.
Refactor unit tests to improve readability and maintainability
- Updated test assertions and mock setups in test_app.py for clarity.
- Added 'pod_id' to the metadata in test_ingestor.py for consistency.
- Introduced a new test for TAG_WRITTEN_COUNT in test_metrics.py to verify its definition.
- Enhanced test data structure in test_data_manager.py to include additional fields.
SIENTIAPDE-1174 Add pod_id to Ingestor metadata for improved tracking
- Included 'pod_id' in the metadata dictionary of the Ingestor class to enhance tracking capabilities.
Update values.yaml to modify replica count and GITHUB_BRANCH environment variable
- Increased replicaCount from 1 to 3 for improved scalability.
- Updated GITHUB_BRANCH environment variable to SIENTIAPDE-1174-mapear-e-implementar-metricas-a-serem-criadas.
Update requirements and environment configurations
- Changed the sientia-dataops-library dependency to a relative path in requirements.txt.
- Updated image tag to 0.3.0 in values.yaml.
- Renamed HTTP_SERVER_PORT to HTTP_METRICS_PORT in app.py and updated its default value to 9090.
- Added TAG_WRITTEN_COUNT metric in metrics.py to track writing processes.
- Incremented TAG_WRITTEN_COUNT in DataManager upon successful data insertion.
Update dependencies and environment variables
- Updated sientia-dataops-library dependency version to 1.3.5 in requirements.txt.
- Changed image tag to 0.2.7 in values.yaml.
- Updated GITHUB_BRANCH environment variable to SIENTIAPDE-1172-criar-pipeline-de-alertas-orquestrador in values.yaml.
Update values.yaml to add metrics service configuration
- Introduced a new 'metrics' service with ClusterIP type, port 9090, and targetPort 9090.
- Updated the serviceMonitor port to reference the new 'metrics' service.
- Changed HTTP_SERVER_PORT environment variable value to 9090 to align with the new metrics service.
SIENTIAPDE-1163
Refactor Ingestor and DataManager classes to enhance metadata handling
- Updated Ingestor class to include additional metadata fields: 'workflow_name' and 'schema_name'.
- Modified DataManager to accept metadata during initialization.
- Adjusted unit tests to validate the new metadata structure and ensure proper functionality across various managers.
- Removed unused logger initialization in tests for cleaner code.
Update values.yaml to restructure service definitions
- Changed the service configuration to a nested structure under 'services'.
- Introduced 'api' and 'opc' services with individual settings, both initially disabled.
- Updated ports for the 'api' service to 4841 while keeping the 'opc' service at 4840.
Refactor logging statements in Ingestor and app modules to use f-strings for improved readability and consistency. Updated error and info logs to enhance clarity in the Ingestor class.
Refactor Ingestor and Manager classes to use CoreNotificationHandler and improve metadata handling
- Updated Ingestor class to initialize logger using get_logger and replaced notification handler initialization with metadata dictionary.
- Refactored DataManager, IngestorManager, OpcManager, and ResourceManager classes to inherit from BaseActivity, allowing for consistent logger and notification handler usage.
- Enhanced notification handling by integrating metadata into notification methods across various managers.
- Added .vscode/ to .gitignore to exclude VSCode configuration files.
Update values.yaml for image tag and GITHUB_BRANCH
- Updated image tag to "0.2.4" for the latest version.
- Changed GITHUB_BRANCH value to "SIENTIAPDE-1163-alterar-dinamica-de-notificacoes-para-usar-o-mongodb-ao-inves-do-kafka".
- Removed .vscode/settings.json as it is no longer needed.
Update requirements and refactor Ingestor class for MongoDB integration
- Updated sientia-dataops-library dependency version to 1.3.0 in requirements.txt.
- Refactored Ingestor class to use CoreNotificationHandler and added MongoDB connection parameters to improve configuration management.
Refactor test assertions for improved readability and consistency
- Updated assertions in test cases for DataManager and OpcManager to enhance readability by formatting long lines.
- Ensured that the expected behavior of metrics logging and message publishing is clearly defined in the tests.
Refactor OpcManager to improve node subscription handling
- Simplified the address node assignment in the OpcManager class by directly using a local variable instead of an instance variable.
- Enhanced debug logging to provide clearer insights into the nodes being subscribed to.
Refactor OPC Manager metrics logging for improved readability
- Reformatted metrics logging statements in the OpcManager class for better readability and consistency.
- Enhanced debug logging to provide clearer insights into subscription management and connection status.
