SIENTIAPDE-1717: Remove MinIO cleanup functionality and associated components. This change streamlines the cleanup workflow to focus solely on local temporary directories, removes the ModelTrainingError exception, and updates related configurations, documentation, and tests.

README.md

@@ -188,7 +188,6 @@ The Model Manager system uses a Temporal-based workflow architecture with clear
   - **Polynomial Regression**: Support for configurable degree and interaction terms
   - **Training Predictions**: Calculates y_train_pred before denormalization for accurate metrics
 - **Cleanup**: File and directory cleanup operations
-  - `cleanup_minio_files()`: Removes stale files from MinIO based on timestamp prefixes
   - `cleanup_temp_directories()`: Cleans local temporary directories
   - Configurable retention period (default: 24 hours)
   - Dry-run mode for testing
@@ -404,16 +403,13 @@ The cleanup schedule is automatically created when the worker starts:
 | **Execution Timeout** | `CLEANUP_EXECUTION_TIMEOUT_HOURS` | `1` | Maximum execution time (hours) |
 | **Retention Period** | `CLEANUP_RETENTION_HOURS` | `24` | Files older than this are deleted |
 | **Dry Run** | `CLEANUP_DRY_RUN` | `false` | Test mode without actual deletion |
-| **Max Keys** | `MAX_KEYS_CLEANUP` | `1000` | MinIO list operation page size |
 
 #### Architecture Diagram
 ```mermaid
 flowchart TD
-    A[Scheduled Trigger] --> B[cleanup_minio_files]
-    B --> C[cleanup_temp_directories]
-    
-    B -.-> MinIO[MinIO Storage]
-    C -.-> FS[Local Filesystem]
+    A[Scheduled Trigger] --> B[cleanup_temp_directories]
+
+    B -.-> FS[Local Filesystem]
 ```
 
 #### Retry Strategies
@@ -1101,8 +1097,6 @@ The Model Manager system exposes comprehensive Prometheus metrics for operationa
 | `CLEANUP_EXECUTION_TIMEOUT_HOURS` | Cleanup execution timeout | `1` | No |
 | `CLEANUP_RETENTION_HOURS` | File retention period (hours) | `24` | No |
 | `CLEANUP_DRY_RUN` | Dry-run mode (no actual deletion) | `false` | No |
-| `MAX_KEYS_CLEANUP` | MinIO list operation page size | `1000` | No |
-| `DEFAULT_CLEANUP_BUCKET` | Default bucket for cleanup | `model-training` | No |
 | `LOG_LEVEL` | Application log level | `INFO` | No |
 | `PROJECT_NAME` | Project name for metrics | `model-manager` | No |
 | `HTTP_METRICS_PORT` | Prometheus metrics port | `9090` | No |
@@ -1127,7 +1121,6 @@ These timeouts control how long each activity in workflows can run before timing
 
 | Variable | Description | Default | Calculation Basis |
 |----------|-------------|---------|-------------------|
-| `TIMEOUT_CLEANUP_MINIO` | MinIO cleanup timeout | `300` | Scan and delete multiple files (5 min) |
 | `TIMEOUT_CLEANUP_LOCAL` | Local cleanup timeout | `120` | Scan and delete directories (2 min) |
 
 **Note**: These timeouts can be adjusted based on your infrastructure performance and file sizes. If you're processing files larger than 200MB or have slower network/compute resources, increase these values accordingly.
@@ -1341,7 +1334,6 @@ export LOG_LEVEL=DEBUG
 - **Connection Pools**: Optimize database connection pool sizes
 - **Model Retention**: Configure MLFlow model retention based on requirements
 - **Batch Sizes**: Adjust data processing batch sizes for optimal throughput
-- **Cleanup Performance**: Tune `MAX_KEYS_CLEANUP` for MinIO list operation page size
 
 ### Scaling Considerations