SIENTIAPDE-1350: Integrate cleanup workflow and update default task queue names.

This commit is contained in:
Bruno Domingues
2025-11-26 22:11:20 -03:00
parent 11aaa0780f
commit 5dcb62ef4b
4 changed files with 21 additions and 12 deletions

View File

@@ -20,13 +20,12 @@ class Activities(ExperimentTracking, Training, Cleanup):
This class combines functionality from multiple activity classes to provide
a unified interface for all workflow operations. It manages database connections,
MLFlow model interactions, MinIO storage operations, and data quality validation.
MLFlow model interactions, MinIO storage operations, and cleanup operations.
The class implements multiple inheritance to combine specialized functionality:
- ExperimentTracking: ML experiment lifecycle tracking and database operations (extends Postgres)
- MLFlow: Model saving and artifact management operations
- MinIO: Object storage operations (file upload/download/delete)
- Training: ML model training operations (extends BaseActivity)
- ExperimentTracking: ML experiment lifecycle tracking and database operations
- Training: ML model training operations with MLFlow and MinIO integration
- Cleanup: File and directory cleanup operations for MinIO and local filesystem
Attributes:
postgres_config (dict): PostgreSQL connection configuration