SIENTIAPDE-1273

Update requirements and enhance metrics and data handling

- Updated the sientia-dataops-library dependency version in requirements.txt to 1.5.3.
- Added new metrics for model analysis, including lag, count, and error count in metrics.py.
- Implemented a new method for formatting transformed data in gates.py.
- Enhanced MLFlowRepository with methods to load artifact dataframes and calculate model metrics, including drift and performance metrics.
- Updated the prediction process to handle transformed data and ensure proper execution of related activities in format_and_export_prediction.py and prediction_process.py.
This commit is contained in:
vitor-aignosi
2025-11-11 16:50:22 -03:00
parent 0c59a7def8
commit 6ac0f38d59
6 changed files with 177 additions and 2 deletions

View File

@@ -402,6 +402,28 @@ class Gates(SientiaMonitoring):
return policy_type, int(policy_value)
@activity.defn(name='format_transformed_data')
async def format_transformed_data(self, input_data: dict[str, Any]) -> dict[Any, Any]:
"""
Format transformed data according to configured storage policies.
"""
metadata = input_data['metadata']
model_id = input_data['model_id']
self.info('Formatting transformed data...', metadata)
data = DataFrame(input_data['data'])
data['timestamp'] = data.index
data = data.reset_index(drop=True)
data = data.melt(id_vars='timestamp', var_name='variable', value_name='value')
data['model_id'] = model_id
return data.to_dict()
@activity.defn(name='format_prediction')
async def format_prediction(self, input_data: dict[str, Any]) -> dict[Any, Any]:
"""