Reads optional simple_metrics_thresholds from model_config and passes
to calculate_simple_metrics. Threshold schema: {rmse_max, r2_min, ...}.
None when not configured (no alerting, no crash).
Updates workflow tests to expect the new key in the activity-call dict.
SIENTIAPDE-1986
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrites 8 existing tests to mock RegressionMetrics instead of verifying
manual numpy math. Adds 2 new tests:
- r2 excluded + warning when model_type is non-linear
- r2 included when model_type is absent (backward compat)
Replaces silent-discard test with ValueError propagation test.
SIENTIAPDE-1986
Extracts model_type from model_config (optional, defaults to None when
absent) and passes it to calculate_simple_metrics. Required for the r2
lock - is_r2_supported() needs model_type to decide whether to compute r2.
Updates workflow tests to expect the new key in the activity-call dict.
SIENTIAPDE-1646 Refactor MLFlow tests and update artifact handling
- Enhanced test cases for MLFlow to improve clarity and accuracy in data handling.
- Updated references in tests to use 'evaluation_data.csv' and 'test_data.csv' instead of 'retrain_input.csv' and 'train_data.csv'.
- Introduced a new helper function for creating prediction frames to streamline test setup.
Enhance MLFlow reference data handling and testing
- Updated the artifact handling in MLFlow to prioritize 'retrain_input.csv' over 'train_data.csv' when resolving reference data.
- Introduced new methods for resolving artifact names and locating downloaded CSV files.
- Modified the `get_reference_data` method to improve artifact resolution and error handling.
- Expanded unit tests to cover scenarios for missing artifacts and preference logic between retrain and train data CSVs.
Refactor ModelMetrics to utilize DriftAnalysis for drift detection
- Replaced ModelAnalysis with DriftAnalysis in the ModelMetrics class to enhance drift detection capabilities.
- Updated method signatures and documentation to reflect the changes in target_name and return values.
- Adjusted data handling to ensure compatibility with the new analysis methods and improved clarity in the drift metrics dataframe preparation.
Update README, requirements, and E2E tests for improved configuration and functionality
- Enhanced the README with updated model configuration examples, including the addition of an alias for production.
- Removed the `requirements-light.txt` file and updated `requirements-local.txt` and `requirements.txt` to replace `asyncua` with `opcua`.
- Refactored E2E test scenarios to utilize scenario input files for better maintainability and clarity.
- Improved test coverage for MinIO offload functionality and added new helper functions for loading scenario inputs.
- Updated `values.yaml` to reflect new global configurations and environment variables for the laborious worker.
Enhance OPC UA communication and metrics tracking
- Updated README.md to include new OPC UA Communication section and detailed metrics for session and write diagnostics.
- Added new metrics in laborious/metrics.py for tracking OPC UA session states and write attempts.
- Refactored OPC activity in laborious/activities/opc.py to handle session errors and improve error reporting.
- Updated e2e tests to cover new scenarios for OPC session/channel errors and reconnect handling.
- Modified .gitignore to include relatorio files and mlruns directory.
- Added ipykernel to requirements-dev.txt for Jupyter notebook support.
SIENTIAPDE-1646 Enhance OPC integration and E2E testing framework
- Updated .gitignore to exclude '.cursor/*' for better file management.
- Added new marker in pyproject.toml for E2E tests using OPC.
- Introduced async OPC server fixtures and improved connection handling in conftest.py.
- Enhanced error handling in OPC activities and added metrics for session management in metrics.py.
- Updated E2E tests to cover new OPC scenarios, including session errors and reconnect handling.
- Refactored helper functions to improve prediction assertion logic in helpers.py.
- Documented new OPC scenarios in scenarios.md for clarity on expected outcomes.
Refactor OPC and OpcRepository classes to remove logger dependency and streamline logging calls
- Removed logger attribute from OPC and OpcRepository classes, replacing direct logger calls with class methods for logging.
- Updated write_data method signatures to eliminate logger parameter, simplifying the interface.
- Adjusted related tests to reflect changes in method signatures and logging behavior.
Implement new scheduling configurations for model retraining and drift analysis in input_sample.json
- Added three new schedule configurations: `minimal-retrain-test-runtime`, `drift-test-runtime`, and `simple-metrics-test-runtime`.
- Each configuration includes parameters such as model ID, workflow type, frequency, and specific queries for data retrieval.
- Enhanced the structure to support active status and updated timestamps for better tracking of schedule states.
SIENTIAPDE-1646 Update dependencies and refactor OPC integration
- Replaced `opcua` with `asyncua` in `requirements-local.txt` and `requirements.txt` to utilize the async capabilities.
- Updated `values.yaml` to change the GitHub branch from `feature/SIENTIAPDE-1646` to `release/SIENTIAPDE-1646`.
- Refactored `opc.py` and `opc_repository.py` to accommodate the new `asyncua` library, ensuring compatibility with the synchronous API.
- Adjusted tests in `test_opc_repository.py` to reflect changes in the client implementation and maintain functionality.
Update E2E test report and enhance drift analysis handling
- Updated the E2E test report metrics to reflect the latest test results, showing 47 collected tests with all passing.
- Removed outdated sections related to failed tests and their causes, streamlining the report.
- Implemented a regression fix in the drift analysis to handle empty merged frames, ensuring workflows skip export when no drift metrics are available.
- Enhanced the `insert_sample_data` and `insert_sample_prediction` functions to allow customizable timestamps for better test accuracy.
- Refactored E2E tests to improve clarity and maintainability, particularly in handling repeat scenarios with distinct timestamps.
Refactor ModelMetrics to utilize DriftAnalysis for drift detection
- Replaced ModelAnalysis with DriftAnalysis in the ModelMetrics class to enhance drift detection capabilities.
- Updated method signatures and documentation to reflect the changes in target_name and return values.
- Adjusted data handling to ensure compatibility with the new analysis methods and improved clarity in the drift metrics dataframe preparation.
Update README, requirements, and E2E tests for improved configuration and functionality
- Enhanced the README with updated model configuration examples, including the addition of an alias for production.
- Removed the `requirements-light.txt` file and updated `requirements-local.txt` and `requirements.txt` to replace `asyncua` with `opcua`.
- Refactored E2E test scenarios to utilize scenario input files for better maintainability and clarity.
- Improved test coverage for MinIO offload functionality and added new helper functions for loading scenario inputs.
- Updated `values.yaml` to reflect new global configurations and environment variables for the laborious worker.
Refactor MLFlow retraining logic to always use retrain method
- Removed the conditional logic for full retraining, ensuring the `retrain` method is always called.
- Updated the documentation in the `retrain_model` method to reflect the changes in retraining flow.
- Adjusted tests to verify that the `retrain` method is invoked correctly, while ensuring `train` is not called when the full retrain flag is set.
Update dependencies and refactor MLFlow activities
- Replaced direct GitHub dependencies in `requirements.txt` with specific versioned packages for `sientia_do` and `sientia_model`.
- Refactored imports in `activities.py` to streamline the code structure.
- Enhanced the `MLFlow` class in `mlflow.py` by introducing a method to resolve model aliases, improving flexibility in model lookups.
- Simplified shutdown logic in `worker.py` for better readability.
- Added new tests for MLFlow activities and improved existing test coverage for data handling and model retraining processes.
Enhance environment configuration and update dependencies
- Added new environment variables for PluginStore and MLflow configuration in `.env.example`, including `RUNTIME`, `STORE_BASE_URL`, `STORE_OWNER`, `STORE_REPO`, `STORE_BRANCH`, `STORE_USERNAME`, `STORE_PASSWORD`, `STORE_CACHE_TTL_SECONDS`, `PYPI_SERVER`, `PYPI_USERNAME`, and `PYPI_PASSWORD`.
- Updated `git-requirements-mapping.txt` to reflect changes in repository names.
- Modified `requirements-light.txt` and `requirements.txt` to upgrade `sientia-dataops-library` to version 1.12.0 and `sientia-mlops-library` to version 0.8.1.
- Updated `values.yaml` to include new environment variables for worker runtime and PluginStore configuration.
- Refactored E2E tests to utilize new MLflow repository stubs and PluginStore mocks for improved testing accuracy.
SIENTIAPDE-1712 Enhance logging across various classes by adding logger parameters and improving debug statements. This update includes adjustments in Gates, MLFlow, ModelMetrics, and MLFlowRepository classes for better traceability and observability during operations.
Update dependencies and refactor input filter handling for consistency
- Updated sientia-dataops-library dependency version from 1.10.3 to 1.10.4 in requirements.txt.
- Refactored input filter handling in the Gates class to read policy and config keys in a case-insensitive manner.
- Updated test cases to ensure consistency in filter key naming conventions across various scenarios.
Refactor metrics and API handling for improved consistency and clarity
- Removed the SIENTIA_CORE_LABELS constant and replaced it with CORE_LABELS for uniformity across metrics.
- Updated the API class to ensure operation_type is always included in core labels for PI Web API metrics.
- Simplified metric tag handling in the Gates class by consolidating common tags into a single core_tags dictionary.
- Enhanced the PredictionProcess class to improve error handling and variable naming for clarity.
Update environment variables in values.yaml and enhance metric labels in metrics.py
- Changed POSTGRES_USER and POSTGRES_PASSWORD values in values.yaml for improved security.
- Added 'runtime' label to metrics in metrics.py for better environment identification.
- Updated CORE_LABELS to include 'runtime' for consistency across metrics.
- Modified type hint for data parameter in PredictionProcess to use a dictionary for better clarity.
- Adjusted tests to reflect changes in core labels and MinIO configuration.
Refactor MinioDataFramePayload usage across activities
- Updated instances of MinioDataFramePayload initialization in Gates, MLFlow, and Storage classes to use the new from_dict method for better data reconstruction from dictionaries.
- Enhanced the PredictionProcess workflow to utilize the updated payload handling.
- Added passthrough fixtures in tests to accommodate the new from_dict method for consistent testing behavior.
Enhance README and Implement Drift Detection and Metrics Workflows
- Added new sections in README for Drift Workflow and Simple Metrics Workflow, detailing their execution flows and functionalities.
- Introduced `drift.py` for data drift detection, comparing current data against reference datasets.
- Added `simple_metrics.py` for calculating regression metrics (RMSE, MSE, MAE, R²).
- Updated `values.yaml` to include configuration for MinIO retention hours and offload threshold.
- Refactored `minio_dataframe_payload.py` to use the new offload threshold environment variable.
- Adjusted tests to reflect changes in environment variable handling for MinIO offload threshold.
Remove code validation script and refactor imports in activities and workflows
- Deleted the `validate.sh` script, which was responsible for running code quality checks.
- Cleaned up import statements in `activities.py`, `gates.py`, `mlflow.py`, and `storage.py` by removing unused imports and organizing them.
- Refactored initialization methods in `MinioManager` and `MLFlow` classes for improved readability.
- Updated various workflows to ensure compatibility with the new structure and removed unnecessary comments.
- Enhanced test cases to accommodate changes in the activities and workflows, ensuring proper mocking of dependencies.
Implement MinIO Offload and Retention Features
- Added configuration options for MinIO retention hours and offload threshold in README.
- Introduced MinIO payload offloading for large DataFrame-derived payloads, storing them as parquet files.
- Updated activities to utilize MinIO for data loading and cleanup, including new methods for offloading and retention management.
- Refactored existing activities to integrate MinIO functionality, ensuring compatibility with previous workflows.
- Removed the legacy MinioRepository class, consolidating MinIO operations under a new manager structure.
- Updated requirements to use the latest version of the sientia-dataops-library.
Refactor PI Web API response handling in tests
- Updated test cases in test_api.py to handle response data as lists instead of dictionaries for consistency with the API's expected output format.
- Adjusted mock responses to reflect the new structure, ensuring tests accurately simulate API behavior.
- Enhanced clarity in test descriptions and improved overall test coverage for response processing scenarios.
Update sonar-project.properties to exclude all worker files from coverage and modify execution counts and timestamps in tests.ipynb. Add a new test for empty DataFrame handling in test_model_repository.py.
Refactor validation script and improve logging in API and model repository
- Updated validation script to include 'e2e/' directory in code formatting and linting checks.
- Enhanced error logging in API class to improve readability of error messages.
- Refactored debug logging in model repository for better structured output.
- Cleaned up import statements in various files for improved organization.
Enhance end-to-end tests for PredictionsBatch workflow scenarios
- Introduced mock repositories for PI Web API and OPC operations to improve test coverage.
- Updated test scenarios to handle partial write errors for PI Web API and OPC.
- Refactored existing tests to assert correct behavior under various error conditions.
- Enhanced logging and error handling in API and OPC activities to provide clearer feedback on failures.
- Removed outdated integration test file to streamline test suite.
SIENTIAPDE-1478
Enhance end-to-end tests for PredictionsBatch workflow
- Added new test scenarios for input and transform gates handling CONTINUE, STOP, and REPEAT policies.
- Implemented sample data insertion functions for testing various prediction outcomes.
- Updated existing tests to verify behavior under different input conditions and response validations.
- Refactored test structure for clarity and maintainability.
Update coverage source in pyproject.toml, add testcontainers for PostgreSQL in requirements-dev.txt, increment image tag and adjust probe delays in values.yaml, and refine condition checks in format_and_export_prediction.py and mlflow.py. Additionally, enhance test coverage in test_gates.py.
SIENTIAPDE-1478
Implement PI Web API response processing and metrics tracking
- Added a new method in the API class to process responses from the PI Web API, validating tag writes and emitting metrics for success and errors.
- Enhanced error handling for missing WebIds and tag names in responses, with appropriate logging and notifications.
- Updated tests to cover various scenarios for processing PI Web API responses, ensuring robust functionality and metrics emission.
- Refactored existing methods to integrate the new response processing logic, improving overall code clarity and maintainability.
Refactor Activities and API Integration for PI Web API
- Reintroduced the API import in the Activities class for proper integration.
- Cleaned up whitespace and formatting in the API class and related tests for improved readability.
- Updated test cases to ensure consistent formatting in error messages and configuration structures for PI Web API.
- Enhanced connectors_config.py with additional whitespace for better organization.
Enhance Activities and Prediction Workflows with PI Web API Integration
- Updated the Activities class to include API integration, allowing for configuration of PI Web API parameters.
- Modified prediction workflows to support output configuration for PI Web API, enabling data writing to the API.
- Refactored connectors_config.py by removing unused PostgreSQL and MongoDB configuration functions.
- Added tests to validate the new PI Web API functionality in activities and workflows, ensuring robust integration and functionality.
Update sientia-mlops-library dependency to version 0.40.6 and refactor get_prediction_data method calls for improved readability in model_repository.py and test_model_repository.py.
Enhance prediction data retrieval in MLFlowRepository by adding predict_flavor parameter
- Updated get_prediction_data method to accept a predict_flavor argument, allowing for different prediction model handling.
- Adjusted calls to get_prediction_data throughout the codebase to include the new parameter.
- Added new test cases to validate behavior for different predict_flavor values, ensuring robust functionality.
Update sientia-mlops-library dependency to version 0.40.5 and fix SQL interval formatting in Drift and SimpleMetrics workflows for improved query accuracy.
Refactor data handling in Laborious workflows to enhance clarity and compatibility
- Improved data validation and formatting in `gates.py`.
- Streamlined model transformation management in `mlflow.py`.
- Enhanced export functionality in `format_and_export_prediction.py` for better data handling flexibility.
- Expanded test coverage for new features and improved documentation in `README.md`.
Enhance data handling and export processes in Laborious workflows
- Updated `gates.py` to improve data quality validation, filtering, and formatting operations, including enhanced metrics recording.
- Refined `mlflow.py` to better manage model transformations and reference data retrieval from MLflow Model Registry.
- Enhanced `format_and_export_prediction.py` to support separate export of transformed data, improving flexibility in data handling.
- Added comprehensive test coverage for new functionalities, including transformed data formatting and retrain report generation.
- Improved documentation in `README.md` to reflect changes in activities and workflows, ensuring clarity on data processing and export paths.
SIENTIAPDE-1273
Enhance security analysis and SQL injection handling
- Added skip for potential SQL injection false positives in Bandit configuration.
- Updated validate.sh to use the pyproject.toml configuration for Bandit security analysis.
- Refactored code to replace ensure_dataframe utility with direct DataFrame usage in multiple activities, improving clarity and reducing dependencies.
- Removed the deprecated dataframe_utils module to streamline the codebase.
Update version and enhance metrics calculation in Laborious system
- Updated image tag in values.yaml from 1.1.0 to 1.1.1.
- Modified GITHUB_BRANCH environment variable for consistency.
- Added a new method `calculate_simple_metrics` in model_metrics.py to compute various model performance metrics including RMSE, MSE, MAE, and R2.
- Integrated the new metrics calculation into the worker setup, allowing for concurrent processing of simple metrics.
- Updated tests to cover the new metrics calculation functionality, ensuring comprehensive validation of the implementation.
Refactor Drift class to improve target name handling
- Extracted target name from model configuration in the Drift class for better clarity and maintainability.
- Updated test cases to reflect changes in input data structure, ensuring consistency in target name retrieval.
Enhance MLFlowRepository and Activities classes with new methods and metrics
- Added `check_artifact_exists` method to MLFlowRepository for verifying artifact presence in the MLflow Model Registry.
- Implemented `get_prediction_data` method in MLFlowRepository to retrieve prediction data from models.
- Updated Activities class to integrate ModelMetrics for improved metrics handling.
- Enhanced tests for artifact existence checks and prediction data retrieval, ensuring robust coverage for new functionalities.
- Updated various workflows to include `transform_table_name` in input data for better data handling.
Update logging in MLFlowRepository and OpcRepository to use unified logging methods
- Refactored logging calls in MLFlowRepository to replace `self.logger.info` and `self.logger.debug` with `self.info` and `self.debug` for consistency.
- Updated connection logging in OpcRepository to format the connection message properly.
- Adjusted test cases to reflect changes in logging behavior and ensure proper assertions.
Enhance metrics handling in Gates and MLFlow classes
- Added checks for `None` response times before emitting OPC writing metrics in the Gates class to prevent unnecessary metric emissions.
- Updated the MLFlow class to conditionally sort and drop duplicates based on the presence of the 'created_at' column, ensuring robustness in data processing.
- Adjusted corresponding tests to validate the new behavior in both classes.
