SIENTIAPDE-1150: enhance MongoDB activities to filter successful pipelines; update orchestration workflow to include timestamp management for updated, created, and deleted pipelines; adjust tests to reflect new success criteria
chore: update image tag to 0.2.4 in values.yaml; refactor logging in slot_manager and temporal_manager for improved readability; modify MongoDB and Redis configurations for better defaults; enhance worker.py with new activities and error handling
fix: implement schedule normalization and timestamp management in MongoDB activities, enhance orchestration workflow with new formatting and normalization methods
fix: update schedule handling in temporal_manager and formatters to support multiple namespaces, enhance test coverage, and improve configuration handling
- Changed search attribute key from "Orchestrated" to "orchestrated" in temporal_manager.py and test cases.
- Enhanced logging for schedule creation and updates in temporal_manager.py.
- Updated schedule creation to use workflow_type directly instead of a hardcoded string.
- Modified gather_read_tags function to use server_id instead of server_name for tag identification.
- Adjusted test cases to reflect changes in server_id usage and ensure consistency across tests.
- Fixed model_name retrieval in common_config to access nested models dictionary.
- Updated test cases to align with new data structures and ensure accurate assertions.
Add comprehensive tests and utility functions for orchestrator activities
- Introduced tests for the new Formatters class, covering methods for processing schedules and slots.
- Enhanced SlotManager tests with update and delete slot functionalities.
- Added TemporalManager tests for creating, updating, and deleting schedules, including frequency parsing.
- Implemented utility functions for orchestrator operations, including frequency parsing and tag configuration building.
- Created tests for utility functions to ensure correct behavior and integration with orchestrator activities.
- Established a new converters module for parsing frequency strings into seconds.
Add unit tests for orchestrator activities and workflows
- Implement tests for Activities class, covering initialization and prepare_activity method.
- Create tests for Couchbase class, including successful and failed query loading.
- Add tests for SlotManager class, verifying OPC slot loading and active ingestor retrieval.
- Develop tests for TemporalManager class, focusing on schedule loading functionality.
- Introduce tests for Orchestrator class, ensuring proper execution of workflow activities.
- Establish a new test suite for orchestrator activities and workflows in the tests directory.
Add unit tests for connectors configuration, logger, workflows, and predictions batch
- Implement tests for MLflow, OPC, and Postgres configuration builders to validate environment variable handling and default values.
- Create tests for the logger to ensure default settings and handler configurations are correct.
- Add comprehensive tests for the FormatAndExportPrediction and PredictionProcess workflows, covering various scenarios including path flags and activity execution.
- Introduce tests for the PredictionsBatch workflow to verify the execution of local activities and child workflows.
- Include a values.yaml file for Kubernetes deployment configuration, specifying image details, service account settings, environment variables, and resource limits.
