Enhance BaseActivity initialization across multiple activities to include error counter
- Updated the initialization of the BaseActivity in Gates, MLFlow, and OPC classes to set the error counter to True, improving error tracking and handling capabilities.
Update dependencies and configuration for improved functionality
- Updated sientia-dataops-library version from 1.3.7 to 1.3.8 in requirements.txt.
- Changed GITHUB_BRANCH in values.yaml to reflect new testing focus: SIENTIAPDE-1169.
- Modified notification level in gates.py from WARNING to ERROR for better error handling.
Add async test for write_metrics method in gates_activity
- Implemented a new test to validate the write_metrics functionality, ensuring metrics are correctly recorded for predictions, confidence, and response time.
- Utilized mocking to verify interactions with the metrics tracking system.
SIENTIAPDE-1174 Update tests to enhance MLFlow and workflow activity assertions
- Added logger initialization in MLFlowRepository for improved logging.
- Updated prediction test assertions to reflect changes in output structure.
- Increased activity method call count assertions in workflow tests for accuracy.
pdate metrics tracking to include response time histogram
- Changed PREDICTION_RESPONSE_TIME_MONITOR from Gauge to Histogram for better response time analysis.
- Updated response time observation method in gates.py to utilize the new Histogram functionality.
SIENTIAPDE-1174 Update replica count and image tag in values.yaml
- Increased replicaCount from 1 to 3 for improved availability.
- Updated image tag from 0.3.1 to 0.3.2 for the latest features and fixes.
SIENTIAPDE-1174 Update tests.ipynb to normalize counter and rollout values, and fix MLFlowRepository initialization error
- Changed execution_count for a cell to null for consistency.
- Updated normalization logic for counter and rollout values to scale between -100 and 100.
- Fixed TypeError in MLFlowRepository initialization by ensuring the logger argument is provided.
SIENTIAPDE-1174 Update replica count and enhance logging in MLFlow
- Changed replicaCount in values.yaml from 3 to 1 for reduced resource usage.
- Added debug logging for prediction response data in MLFlow to improve traceability.
SIENTIAPDE-1174 Add write_metrics activity to main workflow for enhanced metrics tracking
- Included the write_metrics activity in the main workflow to support Prometheus metrics tracking.
Update dependencies, modify replica count, and implement metrics tracking
- Updated sientia-dataops-library version from 1.3.5 to 1.3.7 in requirements.txt.
- Changed replicaCount in values.yaml from 5 to 3 and incremented image tag from 0.2.7 to 0.3.1.
- Added Prometheus metrics tracking in gates.py and worker.py, including a new write_metrics method.
- Configured Prometheus service and ServiceMonitor in values.yaml for metrics collection.
Update dependencies in requirements.txt and increment image tag in values.yaml
- Updated sientia-dataops-library version from 1.3.4 to 1.3.5 in requirements.txt.
- Incremented image tag in values.yaml from 0.2.5 to 0.2.7.
Update GITHUB_BRANCH in values.yaml and enhance error notification handling in MLFlow
- Changed GITHUB_BRANCH in values.yaml to reflect the new pipeline for alerts orchestration.
- Added NotificationLevel.ERROR to error notifications in MLFlow for retraining and production model updates.
- Updated tests to verify error notification levels for model management workflows.
Refactor data merging in MLFlowRepository to simplify merge operation in fit method. Removed unnecessary parameters for improved readability and maintainability.
Enhance data merging in MLFlowRepository by specifying the 'on' parameter for improved join accuracy. This change ensures that the merge operation correctly aligns data based on the 'id' field.
Enhance data merging in MLFlowRepository by specifying merge parameters
- Updated the merge operation in the fit method to include 'how' and 'validate' parameters for improved data integrity and flexibility during model training.
Refactor model_repository and enhance test coverage for MLFlow functionalities
- Updated model_repository to ensure the 'temp' directory is created if it doesn't exist using `exist_ok=True`.
- Added new tests for retraining and updating production models, including error handling scenarios.
- Improved existing tests for model management workflows to ensure robustness and reliability.
Update GITHUB_BRANCH in values.yaml and refactor MLFlowRepository for model retraining
- Changed GITHUB_BRANCH in values.yaml to reflect the new pipeline for retraining.
- Refactored MLFlowRepository by introducing create_model_experiment and perform_model_retrain methods to streamline model retraining processes.
- Updated test cases to cover new functionalities and ensure proper error handling in model management workflows.
Update dependencies and enhance ML model retraining functionality
- Updated sientia-dataops-library version in requirements.txt from 1.3.3 to 1.3.4.
- Incremented image tag in values.yaml from 0.2.4 to 0.2.5 and added a new environment variable MONGODB_TTL_INDEX_HOURS.
- Introduced new methods in MLFlowRepository for model retraining and production model updates, including error handling and logging.
- Added retrain_model and update_production_model activities in mlflow.py to support model management workflows.
- Modified MongoDB connection settings in connectors_config.py for improved security and configuration flexibility.
Update GITHUB_BRANCH in values.yaml to reflect new stress testing branch and modify OPC error handling to set success flag instead of continuing on error.
Enhance OPC error handling by adding detailed notifications for missing servers during write operations. The notification includes the list of available OPC servers for better debugging.
SIENTIAPDE-1163 Enhance OPC tests to improve error handling and notifications. Added tests for MongoDB configuration and refined connection error handling in OPC repository.
Refactor OPC connection handling to improve error notifications and update GITHUB_BRANCH in values.yaml for MongoDB integration. Change requirements.txt to point to local dataops library path.
Update notification handler references to use CoreNotificationHandler and add MongoDB configuration to connectors. Update GITHUB_BRANCH in values.yaml for optimization tasks.
Update values.yaml to increase replicaCount to 5, change GITHUB_BRANCH to SIENTIAPDE-1141-criar-testes-de-carga, and adjust POSTGRES_MAX_CONNECTIONS to 30 for improved performance and testing.
Refactor notification handling in Gates and OPC activities to use send_notification method with metadata integration, enhancing error reporting and traceability.
Refactor Gates and OpcRepository for improved logging and error handling. Removed debug statement in Gates and enhanced disconnect method in OpcRepository to log disconnection status and handle exceptions.
