- Upgraded sientia-model-library from version 0.7.4 to 0.7.5 in both requirements-local.txt and requirements.txt.
- Modified quality-gate.yml to include sientia-model-library in the repositories list for improved CI/CD integration.
- Changed repository links in git-requirements-mapping.txt to a more concise format.
- Updated requirements.txt to reference the sientia-dataops-library and sientia-model-library using simplified names instead of full GitHub URLs.
- Upgraded temporalio to version 1.23.0 for improved functionality.
- Updated sqlalchemy to version 2.0.49, and boto3 and botocore to version 1.42.70 for better compatibility and features.
- Replaced local paths in requirements.txt with specific GitHub repository links for sientia-dataops-library and sientia-model-library.
- Refactored image configuration in values.yaml to move it under the global section, ensuring consistent deployment settings across runtimes.
- Updated `.env.example` to include new environment variables for MinIO and PyPI configuration.
- Refactored `create_cleanup_schedule` to utilize runtime-specific task queues and improve schedule reconciliation logic.
- Enhanced `Activities` class to require a default bucket in MinIO configuration.
- Adjusted `requirements.txt` to specify version for `evidently`.
- Updated tests to reflect changes in schedule creation and configuration handling.
- Added new ignore rule for Ruff to allow temporary paths in tests.
- Introduced MyPy overrides for specific modules to ignore errors.
- Refactored `Cleanup` and `ExperimentTracking` classes to remove async keywords from methods, improving consistency in method signatures.
- Updated `Training` class methods to handle synchronous operations, enhancing performance and clarity.
- Adjusted `requirements.txt` to remove unnecessary Git dependency, streamlining project setup.
- Added `evidently` to requirements for improved model evaluation.
- Introduced `TrainModelResult` class with a `to_dict` method for better result handling.
- Updated `train_model` method to return a comprehensive training result, including run details.
- Enhanced `cleanup_run_directory` method in `DataManagerRepository` for improved resource management.
- Adjusted type hints in `TrainModel` for clarity and consistency.
- Introduced a new activity to load model metadata from the model store.
- Refactored training logic to utilize new model metadata and improved parameter handling.
- Updated the `TrainModelParams` class to include additional fields for model configuration.
- Replaced deprecated utility functions with a custom train-test split implementation.
- Removed unused utility functions and cleaned up the data manager repository.
- Adjusted experiment tracking to include model-specific metadata in notifications.
- Added PluginStore integration for model management.
- Replaced StorageRepository with MinIORepository in Activities, Cleanup, and Training classes.
- Updated training logic to handle validation files and improved data management.
- Enhanced configuration for MinIO and PluginStore in connectors.
- Removed deprecated model repository and storage repository files.
- Updated environment variable handling for new configurations.
This commit introduces MinIO integration for object storage within the Model Manager system. It includes:
- Added MinIO activity class for file operations (fetch, delete).
- Updated Activities orchestrator to include MinIO activities.
- Added MinIO configuration builder to utils/connectors_config.py.
- Added environment variables for MinIO configuration in .env.example.
- Added boto3 and botocore dependencies to requirements.txt.
- Added unit tests for MinIO activities.
This commit removes the OPC server integration from the Model Manager, including related activities, repositories, metrics, and configuration. It also adds code quality tools such as Ruff (linting/formatting), mypy (type checking), and Bandit (security analysis) along with a validation script and CI/CD integration for automated code validation. The README has been updated to reflect these changes.
This commit introduces the initial project structure, including:
- .env.example: Example environment configuration.
- .github/workflows/quality-gate.yml: CI workflow for quality checks.
- .gitignore: Specifies intentionally untracked files that Git should ignore.
- Makefile: Automation of tasks like docker builds.
- README.md: Project documentation.
- Source code for model management, activities, utils, worker and workflows.
- Test suite.
- Dockerfile for the simulator.
- sonar-project.properties: SonarQube configuration file.
- values.yaml: Helm chart values for deployment.
