Update GITHUB_BRANCH in values.yaml to feature/SIENTIAPDE-1273, increment Helm chart version to 0.6.0, and modify metrics registration in Gates class to skip None values. Update test cases to reflect changes in tag values.
Enhance Redis activity by formatting set and get operations for improved readability and consistency. Update tests to include metadata in assertions for better verification of Redis interactions.
Update requirements to use the remote sientia-dataops-library repository and increment image tag in values.yaml from 0.4.9 to 0.5.0 for version consistency.
Refactor notification methods in Gates, MongoDB, and Redis classes to use asynchronous send_notification_async. Update related tests to ensure proper async handling and verification of notifications.
Refactor initialization of Activities, Gates, and MongoDB classes for improved readability by using multi-line argument formatting. Update related tests to match the new initialization style.
Update environment configuration and refactor activities to include metrics controller. Remove Kafka settings and adjust Redis and MongoDB initialization. Update tests to reflect changes in initialization and metrics tracking.
Update GITHUB_BRANCH in values.yaml to feature/SIENTIAPDE-1325 for specific external operation metrics. Refactor shutdown methods in Activities class to ensure proper closure of MongoDB, Redis, and Gates connections. Adjust tests to reflect these changes and enhance repository pattern implementation for MongoDB and Redis activities.
Refactor MongoDB, Redis, and Gates classes to extend SientiaMonitoring instead of BaseActivity. Update requirements to point to local dataops library path. Implement repository pattern for MongoDB and Redis operations, enhancing code organization and maintainability.
SIENTIAPDE-1314: Update workflow name from 'sub_core_scouter' to 'subworkflow.core_scouter' in scouter.py and core_scouter.py, and adjust test references accordingly in test_scouter.py for consistency.
SIENTIAPDE-1314: Update type annotations in encrypt.py for improved clarity and type safety. Added type hint for file_tree variable and included type ignore for yaml import.
Implement tag value registration in Gates activity and update related tests. Enhanced the write_metrics method to process and register tag values, improving metrics tracking. Adjusted tests to validate new functionality.
Update requirements and simplify CI workflow by integrating quality gate from template. Bump sientia-dataops-library version to 1.4.7 and streamline quality-gate.yml to use a shared workflow configuration.
Update Redis and CoreScouter to incorporate 'fill_missing_tags' parameter in data processing. Adjusted related tests to ensure proper handling of missing tags, enhancing overall functionality and consistency across workflows.
SIENTIAPDE-1318: Add 'fill_missing_tags' parameter to CoreScouter for enhanced tag handling in data processing. This update allows for the specification of missing tags during workflow execution.
SIENTIAPDE-1318: Add support for filling missing tags in Redis data processing. Enhanced the Redis class to include a new parameter for handling missing tags, ensuring that absent tags are filled with None in the data package.
Enhance validation script and CI workflow for code quality checks
- Added support for a --fix option in validate.sh to apply Ruff auto-fixes for code formatting and linting.
- Updated quality-gate.yml to include separate steps for installing development and runtime dependencies.
- Introduced dedicated steps for code formatting checks, linting, type checking, and security analysis in the CI workflow.
Update .gitignore and refactor metrics.py, activities.py, and gates.py for improved clarity and consistency. Added coverage.xml and cache directories to .gitignore. Standardized string formatting and parameter handling in metrics and activities classes, enhancing code readability. Removed the deprecated faker.py file and adjusted related tests accordingly.
SIENTIAPDE-1084: Refactor apply_aggregation method in Gates class for improved performance and clarity. Changed parameter name from 'group' to 'values', optimized aggregation logic, and enhanced error handling for invalid functions. Streamlined data processing by reducing unnecessary operations and added batch debug logging.
Update test_activities.py to change assertions for mock_postgres_close and mock_mongodb_close from assert_called_once to assert_called, allowing for more flexible verification of calls during shutdown tests.
Update README.md to modify the Scouter Workflow flowchart, replacing the icon for the core_scouter step with a refresh symbol for better visual representation of data processing continuity.
Update README.md to correct the flowchart syntax for the Scouter Workflow Architecture, changing from 'flowchart-elk' to 'flowchart' to ensure proper rendering and maintain clarity in visualizing data processing steps.
Update README.md to modify the Scouter Workflow Architecture flowchart, changing the layout to 'flowchart-elk' for improved clarity in visualizing data processing steps and interactions.
Refactor README.md to update the Scouter Workflow Architecture flowchart, transitioning to a left-to-right layout for improved readability and clarity in depicting data processing steps and service interactions.
Update README.md to revise the Scouter Workflow Architecture flowchart, replacing outdated steps with a new data processing sequence that includes data quality checks, aggregation, and metrics writing, enhancing clarity on system operations and service interactions.
Update README.md to enhance the architecture diagrams for both Scouter and FakeData workflows, improving clarity and structure by adopting a consistent flowchart format and detailing service interactions.
Update README.md to modify the Scouter Workflow Architecture flowchart, enhancing clarity by restructuring the diagram layout and correcting the connections to Redis and MongoDB components.
Refactor README.md to streamline the Scouter Workflow Architecture section by consolidating the flowchart into a single line format for improved readability and clarity.
Update README.md to include a new section on Scouter Workflow Architecture, detailing the incremental data processing steps and key components, while removing outdated system architecture diagrams for improved clarity and relevance.
Enhance README.md by adding detailed system architecture and data processing flow diagrams using Mermaid syntax, improving clarity on the Scouter system's components and interactions.
