diff --git a/README.md b/README.md index ce79521..5a007cc 100644 --- a/README.md +++ b/README.md @@ -345,6 +345,109 @@ flowchart LR - Cloud-managed services - Local installations +### Temporal Namespace Setup + +The Model Manager requires a dedicated Temporal namespace to isolate workflows and maintain proper execution history. The namespace must be created **before** starting the application. + +#### Why Create a Namespace? + +- **Isolation**: Separates Model Manager workflows from other applications +- **Retention Control**: Configures workflow history retention (default: 7 days) +- **Multi-tenancy**: Enables multiple environments (dev, staging, prod) on same cluster +- **Security**: Allows namespace-level access control and permissions + +#### When to Create? + +- ✅ **Before first deployment** in any environment +- ✅ **Once per environment** (dev, staging, production) +- ✅ **After Temporal cluster setup** or upgrade + +#### How to Create the Namespace + +**Option 1: Using Temporal Admin Tools Pod (Recommended for Kubernetes)** + +```bash +# 1. List Temporal pods +kubectl get pods -n temporal + +# 2. Connect to admin tools pod +kubectl exec -it -n temporal -- bash + +# 3. Create namespace +tctl --namespace model-manager namespace register \ + --retention 7 \ + --description "Model Manager - ML Model Orchestration Namespace" + +# 4. Verify creation +tctl --namespace model-manager namespace describe + +# 5. Exit pod +exit +``` + +**Option 2: Using Port Forward (Local Development)** + +```bash +# 1. Port forward Temporal frontend +kubectl port-forward -n temporal svc/temporal-frontend 7233:7233 + +# 2. In another terminal, create namespace +tctl --address localhost:7233 \ + --namespace model-manager \ + namespace register \ + --retention 7 \ + --description "Model Manager - ML Model Orchestration Namespace" + +# 3. Verify +tctl --address localhost:7233 --namespace model-manager namespace describe +``` + +**Option 3: Direct kubectl exec (One-liner)** + +```bash +kubectl exec -n temporal -- \ + tctl --namespace model-manager namespace register \ + --retention 7 \ + --description "Model Manager - ML Model Orchestration Namespace" +``` + +#### Namespace Configuration + +| Parameter | Value | Description | +|-----------|-------|-------------| +| **Name** | `model-manager` | Namespace identifier (configurable via `TEMPORAL_NAMESPACE` env var) | +| **Retention** | `7 days` | Workflow history retention period | +| **Description** | `Model Manager - ML Model Orchestration Namespace` | Human-readable description | + +#### Verification + +To verify the namespace was created successfully: + +```bash +# List all namespaces +kubectl exec -n temporal -- tctl namespace list + +# Describe specific namespace +kubectl exec -n temporal -- \ + tctl --namespace model-manager namespace describe +``` + +#### Troubleshooting + +**Error: "namespace already exists"** +- ✅ This is fine! The namespace is already configured +- No action needed, proceed with application deployment + +**Error: "connection refused"** +- ❌ Temporal server is not accessible +- Verify Temporal cluster is running: `kubectl get pods -n temporal` +- Check network connectivity and port forwarding + +**Error: "permission denied"** +- ❌ Insufficient permissions to create namespace +- Contact cluster administrator for namespace creation +- Or request elevated permissions for your service account + ## 🚀 Installation ### Local Development Setup diff --git a/run_local.sh b/run_local.sh index 680ff15..d4c8c10 100755 --- a/run_local.sh +++ b/run_local.sh @@ -3,10 +3,12 @@ # Exit on any error set -e -echo "Activating virtual environment..." -source ./venv/bin/activate +#echo "Activating virtual environment..." + +#conda activate ./venv echo "Loading environment variables from .env..." + if [ -f .env ]; then export $(cat .env | grep -v '^#' | xargs) echo "Environment variables loaded from .env" @@ -15,4 +17,5 @@ else fi echo "Starting ingestor application..." + python -m model_manager.worker.worker diff --git a/validate.sh b/validate.sh index 6d5ab14..4bab810 100755 --- a/validate.sh +++ b/validate.sh @@ -67,7 +67,7 @@ if ! run_step "4. Security Analysis (Bandit)" "bandit -r model_manager/ -ll -q"; fi # Step 5: Unit Tests (pytest) -if ! run_step "5. Unit Tests (pytest)" "pytest tests/ --cov=model_manager --cov-report=term-missing --cov-fail-under=70 -q"; then +if ! run_step "5. Unit Tests (pytest)" "pytest tests/ --cov=model_manager --cov-report=term-missing --cov-fail-under=80 -q"; then FAILED_STEPS+=("Unit Tests") fi