feat: integrate PluginStore and MinIO repository into model manager activities

- Added PluginStore integration for model management.
- Replaced StorageRepository with MinIORepository in Activities, Cleanup, and Training classes.
- Updated training logic to handle validation files and improved data management.
- Enhanced configuration for MinIO and PluginStore in connectors.
- Removed deprecated model repository and storage repository files.
- Updated environment variable handling for new configurations.
This commit is contained in:
vitor-aignosi
2026-03-11 17:35:05 -03:00
parent 9d71c0cf80
commit cf5111e520
23 changed files with 1480 additions and 4588 deletions

View File

@@ -2,7 +2,6 @@ from dataclasses import dataclass
import pandas as pd
from model_manager.sientia.models import DataPreprocessor, LinearRegressionModel
from model_manager.utils.models.train_model_params import TrainModelParams
@@ -12,18 +11,15 @@ class TrainModelResult:
A data container for storing the results of a machine learning training process.
This dataclass encapsulates all outputs from the training pipeline, including
the trained model, datasets, evaluation metrics, and paths to generated artifacts.
the prepared datasets, evaluation metrics, and paths to generated artifacts.
It is used to pass results between activities in the training workflow.
Attributes:
params (TrainModelParams): The parameters used to train the model.
process_data (DataPreprocessor): The data preprocessor object used to process the input data.
x_train (pd.DataFrame): The training dataset features.
x_test (pd.DataFrame): The testing dataset features.
y_train (pd.DataFrame): The training dataset target values.
y_test (pd.DataFrame): The testing dataset target values.
regr (LinearRegressionModel): The trained linear regression model.
scaler_dict (dict): A dictionary containing the scalers used to scale the features and target values.
y_pred (pd.Series | None): The predicted target values for the testing dataset. Default is None.
y_train_pred (pd.Series | None): The predicted target values for the training dataset. Default is None.
mse_val (float | None): The Mean Squared Error (MSE) of the predictions. Default is None.
@@ -35,17 +31,13 @@ class TrainModelResult:
report_path (str | None): The path to the generated HTML report file. Default is None.
train_data_path (str | None): The path to the training dataset CSV file. Default is None.
test_data_path (str | None): The path to the testing dataset CSV file. Default is None.
run_dir (str | None): The path to the run directory containing all artifacts. Default is None.
"""
params: TrainModelParams
process_data: DataPreprocessor
x_train: pd.DataFrame
x_test: pd.DataFrame
y_train: pd.Series
y_test: pd.Series
regr: LinearRegressionModel
scaler_dict: dict
y_pred: pd.Series | None = None
y_train_pred: pd.Series | None = None
mse_val: float | None = None
@@ -57,4 +49,3 @@ class TrainModelResult:
report_path: str | None = None
train_data_path: str | None = None
test_data_path: str | None = None
run_dir: str | None = None