SIENTIAPDE-1350: Integrate cleanup workflow and update default task queue names.
This commit is contained in:
@@ -2,9 +2,11 @@
|
||||
|
||||
This module provides the main worker implementation for the Sientia DataOps Model Manager system.
|
||||
It orchestrates Temporal workers, manages task queues, and handles the lifecycle of
|
||||
model training workflows.
|
||||
model training and cleanup workflows.
|
||||
|
||||
The worker supports the train_model-queue task queue for ML model training workflows.
|
||||
The worker supports two task queues:
|
||||
- train_model-queue: For ML model training workflows
|
||||
- cleanup-queue: For file cleanup workflows
|
||||
|
||||
Key Features:
|
||||
- Automatic scaling with PollerBehaviorAutoscaling
|
||||
@@ -12,14 +14,18 @@ Key Features:
|
||||
- Comprehensive error handling and logging
|
||||
- Graceful shutdown with cleanup
|
||||
- ML model training pipeline orchestration
|
||||
- Automated cleanup schedule management
|
||||
|
||||
Environment Variables:
|
||||
- TEMPORAL_HOST: Temporal server address (default: localhost:7233)
|
||||
- TEMPORAL_NAMESPACE: Temporal namespace (default: model_manager)
|
||||
- TEMPORAL_NAMESPACE: Temporal namespace (default: model-manager)
|
||||
- TEMPORAL_USE_TLS: Enable TLS for Temporal connection (default: false)
|
||||
- TRAIN_TASK_QUEUE: Task queue for training workflows (default: train_model-queue)
|
||||
- CLEANUP_TASK_QUEUE: Task queue for cleanup workflows (default: cleanup-queue)
|
||||
- POD_ID: Kubernetes pod identifier for metrics
|
||||
- HTTP_METRICS_PORT: Prometheus metrics server port (default: 9090)
|
||||
- HTTP_SDK_METRICS_PORT: Temporal SDK metrics port (default: 9091)
|
||||
- PROJECT_NAME: Project name for notifications (default: model_manager)
|
||||
- PROJECT_NAME: Project name for notifications (default: model-manager)
|
||||
"""
|
||||
|
||||
from temporalio import client, workflow
|
||||
|
||||
Reference in New Issue
Block a user