This commit introduces a new model_repository.py to handle MLFlow artifact generation and model persistence. It also updates the README to reflect this change and modifies training_repository.py to separate training and MLFlow operations.
This commit refactors the MLFlow activities to focus on model training rather than prediction operations. It removes prediction-related activities and metrics, and updates the MLFlow activity descriptions to reflect the change in focus. The README is also updated to reflect these changes.
This commit refactors the training workflow and associated activities to raise exceptions on failure instead of returning success/failure dictionaries. This allows the Temporal workflow to handle errors more effectively and ensures that the workflow stops when a critical error occurs.
Key changes:
- The train_model workflow is introduced to orchestrate the entire training process, including parameter validation, data download, model training, and model saving.
- The validate_train_params activity is added to validate and convert training parameters.
- The train_model and save_model activities are updated to raise exceptions on failure.
- The ExperimentStatus enum is updated to include a new status for orchestrator validation errors.
- The tests are updated to reflect the new exception-based error handling.
- The activities now return the TrainModelResult directly instead of a dictionary.
This commit introduces artifact generation and MLflow logging capabilities to the model training process. It includes the following changes:
- Added methods to generate reports, save data files, and log model parameters, metrics, models, and artifacts to MLflow.
- Implemented error handling for various scenarios, such as missing files, invalid data, and MLflow connection errors.
- Created a new 'header.html' file for report styling and navigation.
- Modified the 'model_repository.py' file to include the new artifact generation and MLflow logging methods.
- Added comprehensive unit tests to ensure the functionality and robustness of the new features.
This commit introduces the 'Training' activity and 'TrainingRepository' for handling ML model training operations within the Model Manager system.
- Added model_manager/activities/training.py for the Training activity, which extends BaseActivity and integrates with Temporal workflows.
- Added model_manager/utils/repository/training_repository.py for the TrainingRepository, which encapsulates the core training logic.
- Updated model_manager/activities/activities.py to include the Training activity in the main activities orchestrator.
- Updated README.md to document the new 'Training' component.
- Added unit tests for the new activity and repository.
This commit introduces MinIO integration for object storage within the Model Manager system. It includes:
- Added MinIO activity class for file operations (fetch, delete).
- Updated Activities orchestrator to include MinIO activities.
- Added MinIO configuration builder to utils/connectors_config.py.
- Added environment variables for MinIO configuration in .env.example.
- Added boto3 and botocore dependencies to requirements.txt.
- Added unit tests for MinIO activities.
This commit refactors the version calculation logic in the quality gate workflow to handle initial releases and improve accuracy. It also adds disk space cleanup to the workflow and uses --no-cache-dir when installing dependencies to prevent caching issues.
