chore: update image tag and enhance datetime column handling in orchestrator functions
- Updated the image tag in values.yaml from 0.4.2 to 0.4.4 for the latest version.
- Added support for datetime_columns in minimal_retrain and predictions_batch functions to improve timestamp handling in orchestrator functions.
chore: update sientia-dataops-library version and modify environment variables
- Updated the sientia-dataops-library dependency version from 1.4.1 to 1.4.2 in requirements.txt.
- Changed the GITHUB_BRANCH value in values.yaml to reflect the new task SIENTIAPDE-1193.
- Enhanced the process_notifications workflow to include timestamp conversion metadata for improved data handling.
chore: update dependencies and refactor logger imports
- Updated the sientia-dataops-library dependency version from 1.3.8 to 1.4.1 in requirements.txt.
- Changed image tag in values.yaml from 0.4.0 to 0.4.2.
- Refactored logger imports in multiple files to use the new observability module instead of the temporal module.
- Updated retry policy imports in various workflow files to reflect the new structure.
chore: update helm chart version in values.yaml
- Updated the version number for the sientia-orchestrator-worker helm chart from 0.4.0 to 0.5.0 in values.yaml to reflect the latest release.
feat: add SDK metrics service and update telemetry configuration
- Introduced a new service for SDK metrics in values.yaml, enabling metrics collection on port 9091.
- Updated the worker.py to initialize a new runtime with Prometheus metrics configuration for the SDK metrics server.
- Adjusted environment variable for GITHUB_BRANCH to reflect the current focus on testing and performance design.
chore: update helm upgrade command in values.yaml to reflect version change
- Modified the helm upgrade command in values.yaml to use version 0.4.0 instead of 0.4.0-uat for consistency in deployment.
feat: update dependencies and enhance metrics integration
- Updated sientia-dataops-library dependency version from 1.3.5 to 1.3.7 in requirements.txt.
- Incremented image tag in values.yaml from 0.2.7 to 0.3.1.
- Added Prometheus metrics service configuration in values.yaml, enabling metrics collection.
- Implemented Prometheus client in worker.py to start a metrics server and track application health.
SIENTIAPDE-1172
feat: update email configuration and enhance email sending functionality
- Changed EMAIL_SENDER in values.yaml to a new email address.
- Updated EMAIL_SENDER_PASSWORD key reference in values.yaml for improved security.
- Added PostgreSQL configuration parameters to values.yaml for database connectivity.
- Refactored Email class to include a new send_email method for better error handling during email transmission.
- Implemented reconnection logic for SMTP server disconnections to ensure reliable email delivery.
Update dependencies in requirements.txt and increment image tag in values.yaml
- Updated sientia-dataops-library dependency version from 1.3.4 to 1.3.5.
- Changed image tag in values.yaml from 0.2.5 to 0.2.7 for the latest image version.
Update values.yaml and tests to incorporate mail_type parameter for notification handling
- Changed GITHUB_BRANCH in values.yaml to reflect the new alert pipeline.
- Added 'mail_type' parameter to various test cases in SlotManager and notification workflows to enhance notification specificity and tracking.
- Ensured consistency in test data structures across multiple test functions.
feat: integrate email configuration and enhance orchestrator activities
- Added email configuration parameters to values.yaml for sender details and SMTP settings.
- Refactored Email class to correct parameter names and improve logging during initialization.
- Introduced build_email_config function to streamline email configuration retrieval.
- Updated worker.py to include email and Postgres configurations in the main orchestration process.
feat: add TTL index functionality to MongoDB activities and update requirements
- Updated MongoDB class to accept TTL index duration in seconds.
- Implemented create_collection_with_ttl_index method to create collections with TTL indexes.
- Added environment variable for TTL index duration in connectors_config.
- Updated tests to cover new TTL index functionality and ensure proper behavior.
- Bumped sientia-dataops-library version to 1.3.4 in requirements.txt.
refactor: update notification handler imports to use CoreNotificationHandler across activities and worker, enhancing consistency in notification management
refactor: restructure services in values.yaml to separate API and OPC configurations; update mongo_db.py to import datetime correctly and enhance readability
SIENTIAPDE-1150: enhance MongoDB activities to filter successful pipelines; update orchestration workflow to include timestamp management for updated, created, and deleted pipelines; adjust tests to reflect new success criteria
chore: update image tag to 0.2.4 in values.yaml; refactor logging in slot_manager and temporal_manager for improved readability; modify MongoDB and Redis configurations for better defaults; enhance worker.py with new activities and error handling
Refactor predictions_batch to use server_id instead of server_name; update values.yaml to set GITHUB_BRANCH to main; add new laborious pipeline configuration in samples.json
Add unit tests for connectors configuration, logger, workflows, and predictions batch
- Implement tests for MLflow, OPC, and Postgres configuration builders to validate environment variable handling and default values.
- Create tests for the logger to ensure default settings and handler configurations are correct.
- Add comprehensive tests for the FormatAndExportPrediction and PredictionProcess workflows, covering various scenarios including path flags and activity execution.
- Introduce tests for the PredictionsBatch workflow to verify the execution of local activities and child workflows.
- Include a values.yaml file for Kubernetes deployment configuration, specifying image details, service account settings, environment variables, and resource limits.
