SIENTIAPDE-1243: Rename project from 'laborious' to 'model-manager' across codebase and configuration. This includes updating project names in environment variables, Makefiles, README, metrics, and Helm chart values.

This commit is contained in:
Bruno Domingues
2025-10-01 14:41:24 -03:00
parent aba4a3f1a5
commit bc4d98f78d
11 changed files with 45 additions and 45 deletions

View File

@@ -17,10 +17,10 @@ OPC_URL="opc.tcp://sientia-opc-simulator-opc.sientia.svc.cluster.local:4840"
LOG_LEVEL="DEBUG"
HTTP_METRICS_PORT="9090"
HTTP_SDK_METRICS_PORT="9091"
PROJECT_NAME="sientia-laborious"
PROJECT_NAME="sientia-model-manager"
TEMPORAL_HOST="temporal-frontend.temporal.svc.cluster.local:7233"
TEMPORAL_NAMESPACE="laborious"
TEMPORAL_NAMESPACE="model-manager"
MONGODB_USERNAME="mongo_user"
MONGODB_PASSWORD="mongo_db_password"

View File

@@ -1,5 +1,5 @@
VERSION = 1.0.8
name = sientia-laborious
name = sientia-model-manager
# ENVIRONMENT = production
docker-hub:

View File

@@ -21,7 +21,7 @@ A comprehensive AI model management platform for the complete machine learning l
## Architecture
The Laborious system uses a Temporal-based workflow architecture with clear separation of concerns and robust error handling. The architecture is designed for high availability, scalability, and operational excellence in production ML environments.
The Model Manager system uses a Temporal-based workflow architecture with clear separation of concerns and robust error handling. The architecture is designed for high availability, scalability, and operational excellence in production ML environments.
### Architecture Principles
@@ -428,7 +428,7 @@ flowchart LR
## 📦 How to Run
### Running the Laborious Application
### Running the Model Manager Application
Use the provided script to run the application locally:
@@ -443,7 +443,7 @@ chmod +x run_local.sh
The script will:
- Activate the virtual environment
- Load environment variables from `.env`
- Start the laborious worker application
- Start the model-manager worker application
### Running Tests and Coverage
@@ -495,7 +495,7 @@ if [ -f .env ]; then
export $(cat .env | grep -v '^#' | xargs)
fi
# Start the laborious worker
# Start the model-manager worker
python -m model_manager.worker.worker
```
@@ -525,25 +525,25 @@ pytest tests/workflow/test_predictions_batch.py
## 📊 Monitoring and Metrics
The Laborious system exposes comprehensive Prometheus metrics for operational visibility and performance monitoring:
The Model Manager system exposes comprehensive Prometheus metrics for operational visibility and performance monitoring:
### Application Health Metrics
- `app_up`: Application health status (1=healthy, 0=unhealthy)
- Labels: `pod_id`
### Prediction Operation Metrics
- `laborious_predictions_written_count`: Counter for successful prediction exports
- `model_manager_predictions_written_count`: Counter for successful prediction exports
- Labels: `pod_id`, `model_name`, `pipeline_name`
- `laborious_prediction_confidence_monitor`: Gauge for current prediction confidence levels
- `model_manager_prediction_confidence_monitor`: Gauge for current prediction confidence levels
- Labels: `pod_id`, `model_name`, `pipeline_name`
- `laborious_prediction_response_time_monitor`: Histogram for prediction response times
- `model_manager_prediction_response_time_monitor`: Histogram for prediction response times
- Labels: `pod_id`, `model_name`, `pipeline_name`
- Buckets: [0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]
### OPC Export Metrics
- `laborious_prediction_opc_writing_count`: Counter for OPC server write operations
- `model_manager_prediction_opc_writing_count`: Counter for OPC server write operations
- Labels: `pod_id`, `model_name`, `pipeline_name`, `opc_server_id`
- `laborious_prediction_opc_writing_response_time_monitor`: Histogram for OPC write response times
- `model_manager_prediction_opc_writing_response_time_monitor`: Histogram for OPC write response times
- Labels: `pod_id`, `model_name`, `pipeline_name`, `opc_server_id`
- Buckets: [0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]
@@ -559,7 +559,7 @@ The Laborious system exposes comprehensive Prometheus metrics for operational vi
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `TEMPORAL_HOST` | Temporal server address | `localhost:7233` | Yes |
| `TEMPORAL_NAMESPACE` | Temporal namespace | `laborious` | No |
| `TEMPORAL_NAMESPACE` | Temporal namespace | `model-manager` | No |
| `POSTGRES_HOST` | PostgreSQL hostname | `localhost` | Yes |
| `POSTGRES_PORT` | PostgreSQL port | `5432` | Yes |
| `POSTGRES_USER` | PostgreSQL username | `sientia` | Yes |
@@ -585,7 +585,7 @@ The Laborious system exposes comprehensive Prometheus metrics for operational vi
| `MONGODB_DATABASE_NAME` | MongoDB database name | `sientia` | Yes |
| `MONGODB_TTL_INDEX_HOURS` | MongoDB TTL index hours | `1` | No |
| `LOG_LEVEL` | Application log level | `INFO` | No |
| `PROJECT_NAME` | Project name for metrics | `laborious` | No |
| `PROJECT_NAME` | Project name for metrics | `model-manager` | No |
| `HTTP_METRICS_PORT` | Prometheus metrics port | `9090` | No |
| `HTTP_SDK_METRICS_PORT` | Temporal SDK metrics port | `9091` | No |
| `POD_ID` | Kubernetes pod identifier | `None` | No |
@@ -838,4 +838,4 @@ For support and questions:
---
**Note**: The Laborious system is designed for production use in industrial ML environments. Ensure proper security configuration and network isolation for production deployments.
**Note**: The Model Manager system is designed for production use in industrial ML environments. Ensure proper security configuration and network isolation for production deployments.

View File

@@ -12,7 +12,7 @@ with workflow.unsafe.imports_passed_through():
class Activities(Postgres, MLFlow, Gates, OPC):
"""
Main activities orchestrator for the Laborious system.
Main activities orchestrator for the Model Manager system.
This class combines functionality from multiple activity classes to provide
a unified interface for all workflow operations. It manages database connections,

View File

@@ -53,7 +53,7 @@ mlflow_content_filter_functions = {
class Gates(BaseActivity):
"""
Data quality gates and filtering activities for the Laborious system.
Data quality gates and filtering activities for the Model Manager system.
This class implements comprehensive data quality validation and filtering
mechanisms that can be applied at different stages of the prediction pipeline.

View File

@@ -1,7 +1,7 @@
"""
Laborious Metrics Module
Model Manager Metrics Module
This module defines all Prometheus metrics used by the Sientia DataOps Laborious system
This module defines all Prometheus metrics used by the Sientia DataOps Model Manager system
for monitoring and observability. The metrics provide insights into system performance,
prediction quality, and operational health.
@@ -37,21 +37,21 @@ CORE_LABELS = ["pod_id", "model_name", "pipeline_name"]
# Prediction operation metrics
PREDICTIONS_WRITTEN_COUNT = Counter(
"laborious_predictions_written_count",
"model_manager_predictions_written_count",
"Number of predictions written to the database table predictions",
CORE_LABELS,
)
# Prediction quality metrics
PREDICTION_CONFIDENCE_MONITOR = Gauge(
"laborious_prediction_confidence_monitor",
"model_manager_prediction_confidence_monitor",
"Current confidence of each prediction",
CORE_LABELS,
)
# Performance monitoring metrics
PREDICTION_RESPONSE_TIME_MONITOR = Histogram(
"laborious_prediction_response_time_monitor",
"model_manager_prediction_response_time_monitor",
"Current response time of each prediction",
CORE_LABELS,
buckets=[0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]
@@ -59,13 +59,13 @@ PREDICTION_RESPONSE_TIME_MONITOR = Histogram(
# OPC export metrics
PREDICTION_OPC_WRITING_COUNT = Counter(
"laborious_prediction_opc_writing_count",
"model_manager_prediction_opc_writing_count",
"Number of predictions written to the OPC server",
[*CORE_LABELS, "opc_server_id"],
)
PREDICTION_OPC_WRITING_RESPONSE_TIME_MONITOR = Histogram(
"laborious_prediction_opc_writing_response_time_monitor",
"model_manager_prediction_opc_writing_response_time_monitor",
"Current response time of each prediction written to the OPC server",
[*CORE_LABELS, "opc_server_id"],
buckets=[0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]

View File

@@ -1,7 +1,7 @@
"""
Laborious Worker Module
Model Manager Worker Module
This module provides the main worker implementation for the Sientia DataOps Laborious system.
This module provides the main worker implementation for the Sientia DataOps Model Manager system.
It orchestrates Temporal workers, manages task queues, and handles the lifecycle of
prediction and retraining workflows.
@@ -18,11 +18,11 @@ Key Features:
Environment Variables:
- TEMPORAL_HOST: Temporal server address (default: localhost:7233)
- TEMPORAL_NAMESPACE: Temporal namespace (default: laborious)
- TEMPORAL_NAMESPACE: Temporal namespace (default: model_manager)
- POD_ID: Kubernetes pod identifier for metrics
- HTTP_METRICS_PORT: Prometheus metrics server port (default: 9090)
- HTTP_SDK_METRICS_PORT: Temporal SDK metrics port (default: 9091)
- PROJECT_NAME: Project name for notifications (default: laborious)
- PROJECT_NAME: Project name for notifications (default: model_manager)
"""
from temporalio import workflow, client
@@ -56,7 +56,7 @@ SDK_METRICS_PORT = int(os.getenv('HTTP_SDK_METRICS_PORT', "9091"))
async def main():
"""
Main entry point for the Laborious worker application.
Main entry point for the Model Manager worker application.
This function initializes and starts all components of the worker:
1. Sets up logging and metadata
@@ -97,7 +97,7 @@ async def main():
connection_string=mongo_config['connection_string'],
database=mongo_config['database_name'],
logger=logger,
project_name=os.getenv('PROJECT_NAME', 'laborious')
project_name=os.getenv('PROJECT_NAME', 'model-manager')
)
logger.custom_info('Starting Activities...', metadata)
@@ -127,7 +127,7 @@ async def main():
temporal_client = await client.Client.connect(
target_host=host,
namespace=os.getenv('TEMPORAL_NAMESPACE', 'laborious'),
namespace=os.getenv('TEMPORAL_NAMESPACE', 'model-manager'),
runtime=new_runtime
)
@@ -214,7 +214,7 @@ def start_prometheus_server():
and sets the application health metric to indicate the service is running.
The server exposes metrics that can be scraped by Prometheus for monitoring
the health and performance of the Laborious worker.
the health and performance of the Model Manager worker.
Environment Variables:
HTTP_METRICS_PORT: Port for the metrics server (default: 9090)

View File

@@ -10,7 +10,7 @@ with workflow.unsafe.imports_passed_through():
@workflow.defn(name="minimal_retrain")
class MinimalRetrain():
"""
Automated model retraining workflow for the Laborious system.
Automated model retraining workflow for the Model Manager system.
This workflow implements a complete model retraining pipeline that loads
training data, executes model retraining, updates production models,

View File

@@ -10,7 +10,7 @@ with workflow.unsafe.imports_passed_through():
@workflow.defn(name="predictions_batch")
class PredictionsBatch():
"""
Main batch prediction workflow for the Laborious system.
Main batch prediction workflow for the Model Manager system.
This workflow orchestrates the complete batch prediction process, handling
data loading, configuration management, and workflow delegation. It serves

View File

@@ -10,7 +10,7 @@ with workflow.unsafe.imports_passed_through():
@workflow.defn(name="prediction_process")
class PredictionProcess():
"""
Core prediction processing workflow for the Laborious system.
Core prediction processing workflow for the Model Manager system.
This workflow implements the complete ML model inference pipeline, handling
data quality validation, MLFlow model interactions, and prediction processing.

View File

@@ -17,8 +17,8 @@ image:
imagePullSecrets:
- name: docker-hub-secret
# This is to override the chart name.
nameOverride: "sientia-laborious-worker"
fullnameOverride: "sientia-laborious-worker"
nameOverride: "sientia-model-manager-worker"
fullnameOverride: "sientia-model-manager-worker"
namespace: sientia
# This section builds out the service account more information can be found here: https://kubernetes.io/docs/concepts/security/service-accounts/
@@ -31,7 +31,7 @@ serviceAccount:
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: "sientia-laborious-worker"
name: "sientia-model-manager-worker"
# This is for setting Kubernetes Annotations to a Pod.
# For more information checkout: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
@@ -149,7 +149,7 @@ serviceMonitor:
env:
# Entrypoint variables
- name: GITHUB_REPO_URL
value: "git@github.com:Aignosi/sientia-dataops-laborious_temporal.git"
value: "git@github.com:Aignosi/sientia-dataops-model-manager.git"
- name: GITHUB_BRANCH
value: SIENTIAPDE-1222-ajustar-a-library-para-fazer-o-download-do-courier
- name: PYTHON_APP
@@ -195,12 +195,12 @@ env:
- name: HTTP_SDK_METRICS_PORT
value: "9091"
- name: PROJECT_NAME
value: "sientia-laborious"
value: "sientia-model-manager"
- name: TEMPORAL_HOST
value: "temporal-frontend.temporal.svc.cluster.local:7233"
- name: TEMPORAL_NAMESPACE
value: "laborious"
value: "model-manager"
- name: MONGODB_USERNAME
value: "root"
@@ -215,15 +215,15 @@ env:
ssh:
enabled: true
secretName: git-ssh-key-sientia-laborious-worker
secretName: git-ssh-key-sientia-model-manager-worker
sshPath: /mnt/.ssh
knownHostsPath: /mnt/known_hosts
# kubectl create secret docker-registry docker-hub-secret --namespace sientia --docker-server=http://aignosi.azurecr.io --docker-username=aignosi --docker-password=5I5zpQ6sRaHqX1hD3dr+2mo647yO3FRc359/wu6gsP+ACRDRz5mp
# helm upgrade --install sientia-laborious-worker sientia/sientia-module -n sientia --create-namespace -f ./values.yaml --version 0.5.0
# helm upgrade --install sientia-model-manager-worker sientia/sientia-module -n sientia --create-namespace -f ./values.yaml --version 0.5.0
# kubectl create secret generic git-ssh-key-sientia-laborious-worker \
# kubectl create secret generic git-ssh-key-sientia-model-manager-worker \
# --namespace sientia \
# --from-file=ssh-privatekey=git_key \
# --type=kubernetes.io/ssh-auth