Notification.timestamp is now stored as a native BSON Date instead of a
string, so load_latest_data must parse the ISO string coming from Redis
back to datetime before building the $gt filter, or the comparison
silently breaks (BSON String vs Date always evaluates False). Also fixes
put_last_data_timestamp, which would otherwise raise TypeError trying to
json.dumps a pandas Timestamp when persisting the new watermark to Redis.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
SIENTIAPDE-1646 Update RUNTIME_WORKFLOWS in TemporalManager to include 'drift' and 'simple_metrics', expanding workflow capabilities and enhancing orchestration functionality.
SIENTIAPDE-1646 Add support for runtime workflows in TemporalManager by introducing RUNTIME_WORKFLOWS constant and updating schedule creation logic to handle runtime names appropriately.
Update GITHUB_BRANCH in values.yaml to feature/SIENTIAPDE-1646, refactor mongo_db.py for cleaner code, and enhance worker.py by adding missing commas for improved syntax consistency.
Enhance orchestration configuration and documentation. Added `RUNTIME` variable to `.env.example`, updated `.gitignore` to exclude `openspec/` and `.cursor/`, and modified `README.md` to clarify queue naming conventions and runtime handling. Refactored activities to use synchronous database and email handling, improving performance and consistency. Updated test cases to reflect these changes and ensure compatibility with new activity definitions.
Fix error model retrieval in notification_generator.ipynb and add on_conflict parameter to common_config in orchestrator_functions.py for improved error handling.
Update init_orchestration.ipynb to reset execution counts and handle ScheduleAlreadyRunningError, ensuring proper error management during schedule creation. Modify values.yaml to change GITHUB_BRANCH to feature/SIENTIAPDE-1478 for alignment with ongoing feature development.
Reintroduce prepare_worker import in worker.py to maintain consistency in worker setup, ensuring clarity and proper initialization for orchestrator workflows.
Refactor worker initialization by introducing prepare_worker function to streamline worker setup for orchestrator, alerts, and reports workflows, enhancing code clarity and maintainability.
Update version in init_orchestration.ipynb to 3.11.14 and enhance README.md with detailed architecture principles, execution flows, and improved filtering mechanisms in workflows. Refactor orchestrator_functions.py for clarity in filter configuration and add support for PI Web API output configuration in predictions_batch. Update tests to reflect new configurations.
SIENTIAPDE-1461 Update values.yaml to configure resource limits and requests, enhance liveness and readiness probes, and set PYPI_SERVER environment variable for improved application performance and monitoring.
Update values.yaml to change image tag to 1.0.0 and update GITHUB_BRANCH to main for improved versioning and alignment with the main development branch.
Refactor test cases in test_orchestrator_functions.py to consolidate read_tags structure and enhance clarity in API error handling, improving test robustness and readability.
Enhance MongoDB activity to handle missing topic configurations gracefully by using get() method and skipping empty collections, improving robustness in pipeline processing.
Refactor tag processing in pi_web_api_scouter function to improve clarity by renaming variables and enhancing readability of the configuration handling.
Refactor values.yaml to comment out email configuration settings for improved security and clarity. Update orchestrator_functions.py to enhance tag processing by iterating over items in read_tags, improving code readability.
Refactor gather_read_tags function to remove logger parameter and associated logging statements, streamlining the function for improved clarity and focus on processing pipelines.
