Update values.yaml to change image tag to 1.0.0 and update GITHUB_BRANCH to main for improved versioning and alignment with the main development branch.
Refactor values.yaml to comment out email configuration settings for improved security and clarity. Update orchestrator_functions.py to enhance tag processing by iterating over items in read_tags, improving code readability.
Update GITHUB_BRANCH to feature/SIENTIAPDE-1445 and enhance formatters with a new registry for workflow types, improving configuration management. Refactor process_schedules to utilize the new registry and add error handling for unsupported workflow types. Update tests to validate new functionality and ensure proper integration.
Update GITHUB_BRANCH to feature/SIENTIAPDE-1273 and increment Helm chart version to 0.6.0. Enhance TemporalManager to include offset in schedule intervals for improved scheduling flexibility.
Update image tag to 0.5.0 in values.yaml and change GITHUB_BRANCH to feature/SIENTIAPDE-1325 for specific external operation metrics. Remove unused timestamp formatting in mongo_db.py to streamline data handling.
Remove unused Couchbase configurations and related code. Refactor Activities class to eliminate Couchbase dependency, updating initialization and shutdown methods. Update README and tests to reflect these changes. Upgrade sientia-dataops-library dependency version in requirements.txt.
Refactor orchestrator activities and configuration files
- Removed unused Temporal timeout environment variables from values.yaml.
- Reorganized import statements in various activity files for better readability.
- Updated logging messages to use consistent formatting across activities.
- Enhanced test cases to ensure proper initialization and shutdown of orchestrator activities.
- Improved overall code structure and readability by applying consistent formatting and style adjustments.
SIENTIAPDE-1231: Add timeout configuration for Temporal tasks
- Introduced new timeout parameters in values.yaml for task execution.
- Updated TemporalManager to utilize execution and task timeout settings in seconds.
- Enhanced orchestrator_functions.py to include default timeout values for better orchestration control.
Update orchestration configuration and execution parameters
- Adjusted execution counts in Jupyter notebook cells for better tracking.
- Updated schedule name in the orchestration configuration to align with new requirements.
- Modified `values.yaml` to change the GitHub branch for improved versioning.
- Enhanced `Activities` and `TemporalManager` classes to include timeout parameters for task execution.
- Updated connector configuration to include new timeout settings for better orchestration control.
Update environment variables in values.yaml and enhance predictions_batch function in orchestrator_functions.py
- Changed GITHUB_BRANCH to reflect the new task for adjusting laborious processes to capture timestamps from MLflow.
- Added a new filter configuration for handling empty data in the predictions_batch function to improve error management.
SIENTIAPDE-1205: Update GITHUB_BRANCH and standardize datetime handling in MongoDB activities
- Changed GITHUB_BRANCH in values.yaml from "main" to "SIENTIAPDE-1205-alterar-opc-para-assincrono".
- Updated datetime parsing in mongo_db.py to use DATETIME_FORMAT_MS_WITH_TZ for consistent timestamp handling.
Update orchestration notebook execution count, modify image tag in values.yaml, and standardize datetime handling in MongoDB activities
- Changed execution count in init_orchestration.ipynb from 2 to 1.
- Updated image tag in values.yaml from 0.4.4 to 0.4.5.
- Replaced DATETIME_FORMAT_MS_WITH_TZ with DATETIME_FORMAT_MS in mongo_db.py for consistent timestamp handling.
chore: update image tag and enhance datetime column handling in orchestrator functions
- Updated the image tag in values.yaml from 0.4.2 to 0.4.4 for the latest version.
- Added support for datetime_columns in minimal_retrain and predictions_batch functions to improve timestamp handling in orchestrator functions.
chore: update sientia-dataops-library version and modify environment variables
- Updated the sientia-dataops-library dependency version from 1.4.1 to 1.4.2 in requirements.txt.
- Changed the GITHUB_BRANCH value in values.yaml to reflect the new task SIENTIAPDE-1193.
- Enhanced the process_notifications workflow to include timestamp conversion metadata for improved data handling.
chore: update dependencies and refactor logger imports
- Updated the sientia-dataops-library dependency version from 1.3.8 to 1.4.1 in requirements.txt.
- Changed image tag in values.yaml from 0.4.0 to 0.4.2.
- Refactored logger imports in multiple files to use the new observability module instead of the temporal module.
- Updated retry policy imports in various workflow files to reflect the new structure.
chore: update helm chart version in values.yaml
- Updated the version number for the sientia-orchestrator-worker helm chart from 0.4.0 to 0.5.0 in values.yaml to reflect the latest release.
feat: add SDK metrics service and update telemetry configuration
- Introduced a new service for SDK metrics in values.yaml, enabling metrics collection on port 9091.
- Updated the worker.py to initialize a new runtime with Prometheus metrics configuration for the SDK metrics server.
- Adjusted environment variable for GITHUB_BRANCH to reflect the current focus on testing and performance design.
chore: update helm upgrade command in values.yaml to reflect version change
- Modified the helm upgrade command in values.yaml to use version 0.4.0 instead of 0.4.0-uat for consistency in deployment.
feat: update dependencies and enhance metrics integration
- Updated sientia-dataops-library dependency version from 1.3.5 to 1.3.7 in requirements.txt.
- Incremented image tag in values.yaml from 0.2.7 to 0.3.1.
- Added Prometheus metrics service configuration in values.yaml, enabling metrics collection.
- Implemented Prometheus client in worker.py to start a metrics server and track application health.
SIENTIAPDE-1172
feat: update email configuration and enhance email sending functionality
- Changed EMAIL_SENDER in values.yaml to a new email address.
- Updated EMAIL_SENDER_PASSWORD key reference in values.yaml for improved security.
- Added PostgreSQL configuration parameters to values.yaml for database connectivity.
- Refactored Email class to include a new send_email method for better error handling during email transmission.
- Implemented reconnection logic for SMTP server disconnections to ensure reliable email delivery.
Update dependencies in requirements.txt and increment image tag in values.yaml
- Updated sientia-dataops-library dependency version from 1.3.4 to 1.3.5.
- Changed image tag in values.yaml from 0.2.5 to 0.2.7 for the latest image version.
Update values.yaml and tests to incorporate mail_type parameter for notification handling
- Changed GITHUB_BRANCH in values.yaml to reflect the new alert pipeline.
- Added 'mail_type' parameter to various test cases in SlotManager and notification workflows to enhance notification specificity and tracking.
- Ensured consistency in test data structures across multiple test functions.
feat: integrate email configuration and enhance orchestrator activities
- Added email configuration parameters to values.yaml for sender details and SMTP settings.
- Refactored Email class to correct parameter names and improve logging during initialization.
- Introduced build_email_config function to streamline email configuration retrieval.
- Updated worker.py to include email and Postgres configurations in the main orchestration process.
feat: add TTL index functionality to MongoDB activities and update requirements
- Updated MongoDB class to accept TTL index duration in seconds.
- Implemented create_collection_with_ttl_index method to create collections with TTL indexes.
- Added environment variable for TTL index duration in connectors_config.
- Updated tests to cover new TTL index functionality and ensure proper behavior.
- Bumped sientia-dataops-library version to 1.3.4 in requirements.txt.
refactor: update notification handler imports to use CoreNotificationHandler across activities and worker, enhancing consistency in notification management
refactor: restructure services in values.yaml to separate API and OPC configurations; update mongo_db.py to import datetime correctly and enhance readability
SIENTIAPDE-1150: enhance MongoDB activities to filter successful pipelines; update orchestration workflow to include timestamp management for updated, created, and deleted pipelines; adjust tests to reflect new success criteria
chore: update image tag to 0.2.4 in values.yaml; refactor logging in slot_manager and temporal_manager for improved readability; modify MongoDB and Redis configurations for better defaults; enhance worker.py with new activities and error handling
Refactor predictions_batch to use server_id instead of server_name; update values.yaml to set GITHUB_BRANCH to main; add new laborious pipeline configuration in samples.json
Add unit tests for connectors configuration, logger, workflows, and predictions batch
- Implement tests for MLflow, OPC, and Postgres configuration builders to validate environment variable handling and default values.
- Create tests for the logger to ensure default settings and handler configurations are correct.
- Add comprehensive tests for the FormatAndExportPrediction and PredictionProcess workflows, covering various scenarios including path flags and activity execution.
- Introduce tests for the PredictionsBatch workflow to verify the execution of local activities and child workflows.
- Include a values.yaml file for Kubernetes deployment configuration, specifying image details, service account settings, environment variables, and resource limits.
