SIENTIAPDE-1442: Enhance Temporal worker configuration by adding environment variables for concurrency limits and poller behavior settings in values.yaml and worker.py for optimized performance.
Update GITHUB_BRANCH in values.yaml to feature/SIENTIAPDE-1273, increment Helm chart version to 0.6.0, and modify metrics registration in Gates class to skip None values. Update test cases to reflect changes in tag values.
Enhance Redis activity by formatting set and get operations for improved readability and consistency. Update tests to include metadata in assertions for better verification of Redis interactions.
Refactor notification methods in Gates, MongoDB, and Redis classes to use asynchronous send_notification_async. Update related tests to ensure proper async handling and verification of notifications.
Refactor initialization of Activities, Gates, and MongoDB classes for improved readability by using multi-line argument formatting. Update related tests to match the new initialization style.
Update environment configuration and refactor activities to include metrics controller. Remove Kafka settings and adjust Redis and MongoDB initialization. Update tests to reflect changes in initialization and metrics tracking.
Update GITHUB_BRANCH in values.yaml to feature/SIENTIAPDE-1325 for specific external operation metrics. Refactor shutdown methods in Activities class to ensure proper closure of MongoDB, Redis, and Gates connections. Adjust tests to reflect these changes and enhance repository pattern implementation for MongoDB and Redis activities.
Refactor MongoDB, Redis, and Gates classes to extend SientiaMonitoring instead of BaseActivity. Update requirements to point to local dataops library path. Implement repository pattern for MongoDB and Redis operations, enhancing code organization and maintainability.
SIENTIAPDE-1314: Update workflow name from 'sub_core_scouter' to 'subworkflow.core_scouter' in scouter.py and core_scouter.py, and adjust test references accordingly in test_scouter.py for consistency.
Implement tag value registration in Gates activity and update related tests. Enhanced the write_metrics method to process and register tag values, improving metrics tracking. Adjusted tests to validate new functionality.
Update Redis and CoreScouter to incorporate 'fill_missing_tags' parameter in data processing. Adjusted related tests to ensure proper handling of missing tags, enhancing overall functionality and consistency across workflows.
SIENTIAPDE-1318: Add 'fill_missing_tags' parameter to CoreScouter for enhanced tag handling in data processing. This update allows for the specification of missing tags during workflow execution.
SIENTIAPDE-1318: Add support for filling missing tags in Redis data processing. Enhanced the Redis class to include a new parameter for handling missing tags, ensuring that absent tags are filled with None in the data package.
Update .gitignore and refactor metrics.py, activities.py, and gates.py for improved clarity and consistency. Added coverage.xml and cache directories to .gitignore. Standardized string formatting and parameter handling in metrics and activities classes, enhancing code readability. Removed the deprecated faker.py file and adjusted related tests accordingly.
SIENTIAPDE-1084: Refactor apply_aggregation method in Gates class for improved performance and clarity. Changed parameter name from 'group' to 'values', optimized aggregation logic, and enhanced error handling for invalid functions. Streamlined data processing by reducing unnecessary operations and added batch debug logging.
Remove deprecated files and configurations, including .env, Dockerfile, docker-compose.yml, and client-schedule.py. Update README.md to reflect new architecture and features, enhancing clarity on system capabilities and workflows. Adjust values.yaml for image tag and replica count, and improve code documentation across various modules for better maintainability.
Refactor timestamp handling in CoreScouter workflow by updating to DATETIME_FORMAT_WITH_TZ, ensuring consistency with recent changes in datetime management.
Enhance timestamp handling in Redis and CoreScouter workflows by updating to DATETIME_FORMAT_MS_WITH_TZ. Modify tests to reflect new timestamp format, ensuring consistency across data structures and improving overall datetime management.
Refactor Redis activity to utilize the new 'now' function for timestamp generation, ensuring consistency with updated datetime handling. This change replaces the direct use of 'datetime.now()' with 'now()' from the temporal constants.
Update sientia-dataops-library version to 1.4.3 and refactor timestamp handling in MongoDB and Redis activities to use DATETIME_FORMAT constants for improved consistency. Enhance unit tests to reflect these changes.
Update requirements and core scouter workflow: comment out old dataops library dependency, change default namespace in worker.py, and enhance timestamp handling in core_scouter.py with DATETIME_FORMAT.
Refactor logging in worker.py to use custom_info and custom_error methods, enhancing log metadata with pod and workflow details for improved observability.
Enhance logging in Gates, MongoDB, and Redis activities by replacing debug statements with info level logs, improving observability of data processing steps. Update worker configuration to adjust concurrency settings and enable autoscaling for task polling.
Update dependencies and improve observability by changing logger imports. Bump sientia-dataops-library version to 1.4.1 and update image tag to 0.4.2 in values.yaml.
Enable SDK metrics service in values.yaml and update worker.py to initialize SDK metrics server with Prometheus configuration, enhancing observability of the application.
Update image tag to 0.4.0 in values.yaml, modify GITHUB_BRANCH for SIENTIAPDE-1169, and enhance Redis activity to handle model tags for improved data retention.
Update requirements for sientia-dataops-library to version 1.3.7, add ServiceMonitor configuration for Prometheus in values.yaml, and refactor pod_id usage in Gates and Redis activities for improved consistency in metric tracking.
SIENTIAPDE-1174 Add pod_id initialization in Activities class and update Scouter to reference it for improved consistency in environment variable handling.
Update replica count in values.yaml, add application status metrics in metrics.py, and implement Prometheus server in worker.py for enhanced observability.
