SIENTIAPDE-1243: Rename project from 'laborious' to 'model-manager' across codebase and configuration. This includes updating project names in environment variables, Makefiles, README, metrics, and Helm chart values.

This commit is contained in:
Bruno Domingues
2025-10-01 14:41:24 -03:00
parent aba4a3f1a5
commit bc4d98f78d
11 changed files with 45 additions and 45 deletions

View File

@@ -17,10 +17,10 @@ OPC_URL="opc.tcp://sientia-opc-simulator-opc.sientia.svc.cluster.local:4840"
LOG_LEVEL="DEBUG" LOG_LEVEL="DEBUG"
HTTP_METRICS_PORT="9090" HTTP_METRICS_PORT="9090"
HTTP_SDK_METRICS_PORT="9091" HTTP_SDK_METRICS_PORT="9091"
PROJECT_NAME="sientia-laborious" PROJECT_NAME="sientia-model-manager"
TEMPORAL_HOST="temporal-frontend.temporal.svc.cluster.local:7233" TEMPORAL_HOST="temporal-frontend.temporal.svc.cluster.local:7233"
TEMPORAL_NAMESPACE="laborious" TEMPORAL_NAMESPACE="model-manager"
MONGODB_USERNAME="mongo_user" MONGODB_USERNAME="mongo_user"
MONGODB_PASSWORD="mongo_db_password" MONGODB_PASSWORD="mongo_db_password"

View File

@@ -1,5 +1,5 @@
VERSION = 1.0.8 VERSION = 1.0.8
name = sientia-laborious name = sientia-model-manager
# ENVIRONMENT = production # ENVIRONMENT = production
docker-hub: docker-hub:

View File

@@ -21,7 +21,7 @@ A comprehensive AI model management platform for the complete machine learning l
## Architecture ## Architecture
The Laborious system uses a Temporal-based workflow architecture with clear separation of concerns and robust error handling. The architecture is designed for high availability, scalability, and operational excellence in production ML environments. The Model Manager system uses a Temporal-based workflow architecture with clear separation of concerns and robust error handling. The architecture is designed for high availability, scalability, and operational excellence in production ML environments.
### Architecture Principles ### Architecture Principles
@@ -428,7 +428,7 @@ flowchart LR
## 📦 How to Run ## 📦 How to Run
### Running the Laborious Application ### Running the Model Manager Application
Use the provided script to run the application locally: Use the provided script to run the application locally:
@@ -443,7 +443,7 @@ chmod +x run_local.sh
The script will: The script will:
- Activate the virtual environment - Activate the virtual environment
- Load environment variables from `.env` - Load environment variables from `.env`
- Start the laborious worker application - Start the model-manager worker application
### Running Tests and Coverage ### Running Tests and Coverage
@@ -495,7 +495,7 @@ if [ -f .env ]; then
export $(cat .env | grep -v '^#' | xargs) export $(cat .env | grep -v '^#' | xargs)
fi fi
# Start the laborious worker # Start the model-manager worker
python -m model_manager.worker.worker python -m model_manager.worker.worker
``` ```
@@ -525,25 +525,25 @@ pytest tests/workflow/test_predictions_batch.py
## 📊 Monitoring and Metrics ## 📊 Monitoring and Metrics
The Laborious system exposes comprehensive Prometheus metrics for operational visibility and performance monitoring: The Model Manager system exposes comprehensive Prometheus metrics for operational visibility and performance monitoring:
### Application Health Metrics ### Application Health Metrics
- `app_up`: Application health status (1=healthy, 0=unhealthy) - `app_up`: Application health status (1=healthy, 0=unhealthy)
- Labels: `pod_id` - Labels: `pod_id`
### Prediction Operation Metrics ### Prediction Operation Metrics
- `laborious_predictions_written_count`: Counter for successful prediction exports - `model_manager_predictions_written_count`: Counter for successful prediction exports
- Labels: `pod_id`, `model_name`, `pipeline_name` - Labels: `pod_id`, `model_name`, `pipeline_name`
- `laborious_prediction_confidence_monitor`: Gauge for current prediction confidence levels - `model_manager_prediction_confidence_monitor`: Gauge for current prediction confidence levels
- Labels: `pod_id`, `model_name`, `pipeline_name` - Labels: `pod_id`, `model_name`, `pipeline_name`
- `laborious_prediction_response_time_monitor`: Histogram for prediction response times - `model_manager_prediction_response_time_monitor`: Histogram for prediction response times
- Labels: `pod_id`, `model_name`, `pipeline_name` - Labels: `pod_id`, `model_name`, `pipeline_name`
- Buckets: [0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0] - Buckets: [0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]
### OPC Export Metrics ### OPC Export Metrics
- `laborious_prediction_opc_writing_count`: Counter for OPC server write operations - `model_manager_prediction_opc_writing_count`: Counter for OPC server write operations
- Labels: `pod_id`, `model_name`, `pipeline_name`, `opc_server_id` - Labels: `pod_id`, `model_name`, `pipeline_name`, `opc_server_id`
- `laborious_prediction_opc_writing_response_time_monitor`: Histogram for OPC write response times - `model_manager_prediction_opc_writing_response_time_monitor`: Histogram for OPC write response times
- Labels: `pod_id`, `model_name`, `pipeline_name`, `opc_server_id` - Labels: `pod_id`, `model_name`, `pipeline_name`, `opc_server_id`
- Buckets: [0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0] - Buckets: [0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]
@@ -559,7 +559,7 @@ The Laborious system exposes comprehensive Prometheus metrics for operational vi
| Variable | Description | Default | Required | | Variable | Description | Default | Required |
|----------|-------------|---------|----------| |----------|-------------|---------|----------|
| `TEMPORAL_HOST` | Temporal server address | `localhost:7233` | Yes | | `TEMPORAL_HOST` | Temporal server address | `localhost:7233` | Yes |
| `TEMPORAL_NAMESPACE` | Temporal namespace | `laborious` | No | | `TEMPORAL_NAMESPACE` | Temporal namespace | `model-manager` | No |
| `POSTGRES_HOST` | PostgreSQL hostname | `localhost` | Yes | | `POSTGRES_HOST` | PostgreSQL hostname | `localhost` | Yes |
| `POSTGRES_PORT` | PostgreSQL port | `5432` | Yes | | `POSTGRES_PORT` | PostgreSQL port | `5432` | Yes |
| `POSTGRES_USER` | PostgreSQL username | `sientia` | Yes | | `POSTGRES_USER` | PostgreSQL username | `sientia` | Yes |
@@ -585,7 +585,7 @@ The Laborious system exposes comprehensive Prometheus metrics for operational vi
| `MONGODB_DATABASE_NAME` | MongoDB database name | `sientia` | Yes | | `MONGODB_DATABASE_NAME` | MongoDB database name | `sientia` | Yes |
| `MONGODB_TTL_INDEX_HOURS` | MongoDB TTL index hours | `1` | No | | `MONGODB_TTL_INDEX_HOURS` | MongoDB TTL index hours | `1` | No |
| `LOG_LEVEL` | Application log level | `INFO` | No | | `LOG_LEVEL` | Application log level | `INFO` | No |
| `PROJECT_NAME` | Project name for metrics | `laborious` | No | | `PROJECT_NAME` | Project name for metrics | `model-manager` | No |
| `HTTP_METRICS_PORT` | Prometheus metrics port | `9090` | No | | `HTTP_METRICS_PORT` | Prometheus metrics port | `9090` | No |
| `HTTP_SDK_METRICS_PORT` | Temporal SDK metrics port | `9091` | No | | `HTTP_SDK_METRICS_PORT` | Temporal SDK metrics port | `9091` | No |
| `POD_ID` | Kubernetes pod identifier | `None` | No | | `POD_ID` | Kubernetes pod identifier | `None` | No |
@@ -838,4 +838,4 @@ For support and questions:
--- ---
**Note**: The Laborious system is designed for production use in industrial ML environments. Ensure proper security configuration and network isolation for production deployments. **Note**: The Model Manager system is designed for production use in industrial ML environments. Ensure proper security configuration and network isolation for production deployments.

View File

@@ -12,7 +12,7 @@ with workflow.unsafe.imports_passed_through():
class Activities(Postgres, MLFlow, Gates, OPC): class Activities(Postgres, MLFlow, Gates, OPC):
""" """
Main activities orchestrator for the Laborious system. Main activities orchestrator for the Model Manager system.
This class combines functionality from multiple activity classes to provide This class combines functionality from multiple activity classes to provide
a unified interface for all workflow operations. It manages database connections, a unified interface for all workflow operations. It manages database connections,

View File

@@ -53,7 +53,7 @@ mlflow_content_filter_functions = {
class Gates(BaseActivity): class Gates(BaseActivity):
""" """
Data quality gates and filtering activities for the Laborious system. Data quality gates and filtering activities for the Model Manager system.
This class implements comprehensive data quality validation and filtering This class implements comprehensive data quality validation and filtering
mechanisms that can be applied at different stages of the prediction pipeline. mechanisms that can be applied at different stages of the prediction pipeline.

View File

@@ -1,7 +1,7 @@
""" """
Laborious Metrics Module Model Manager Metrics Module
This module defines all Prometheus metrics used by the Sientia DataOps Laborious system This module defines all Prometheus metrics used by the Sientia DataOps Model Manager system
for monitoring and observability. The metrics provide insights into system performance, for monitoring and observability. The metrics provide insights into system performance,
prediction quality, and operational health. prediction quality, and operational health.
@@ -37,21 +37,21 @@ CORE_LABELS = ["pod_id", "model_name", "pipeline_name"]
# Prediction operation metrics # Prediction operation metrics
PREDICTIONS_WRITTEN_COUNT = Counter( PREDICTIONS_WRITTEN_COUNT = Counter(
"laborious_predictions_written_count", "model_manager_predictions_written_count",
"Number of predictions written to the database table predictions", "Number of predictions written to the database table predictions",
CORE_LABELS, CORE_LABELS,
) )
# Prediction quality metrics # Prediction quality metrics
PREDICTION_CONFIDENCE_MONITOR = Gauge( PREDICTION_CONFIDENCE_MONITOR = Gauge(
"laborious_prediction_confidence_monitor", "model_manager_prediction_confidence_monitor",
"Current confidence of each prediction", "Current confidence of each prediction",
CORE_LABELS, CORE_LABELS,
) )
# Performance monitoring metrics # Performance monitoring metrics
PREDICTION_RESPONSE_TIME_MONITOR = Histogram( PREDICTION_RESPONSE_TIME_MONITOR = Histogram(
"laborious_prediction_response_time_monitor", "model_manager_prediction_response_time_monitor",
"Current response time of each prediction", "Current response time of each prediction",
CORE_LABELS, CORE_LABELS,
buckets=[0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0] buckets=[0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]
@@ -59,13 +59,13 @@ PREDICTION_RESPONSE_TIME_MONITOR = Histogram(
# OPC export metrics # OPC export metrics
PREDICTION_OPC_WRITING_COUNT = Counter( PREDICTION_OPC_WRITING_COUNT = Counter(
"laborious_prediction_opc_writing_count", "model_manager_prediction_opc_writing_count",
"Number of predictions written to the OPC server", "Number of predictions written to the OPC server",
[*CORE_LABELS, "opc_server_id"], [*CORE_LABELS, "opc_server_id"],
) )
PREDICTION_OPC_WRITING_RESPONSE_TIME_MONITOR = Histogram( PREDICTION_OPC_WRITING_RESPONSE_TIME_MONITOR = Histogram(
"laborious_prediction_opc_writing_response_time_monitor", "model_manager_prediction_opc_writing_response_time_monitor",
"Current response time of each prediction written to the OPC server", "Current response time of each prediction written to the OPC server",
[*CORE_LABELS, "opc_server_id"], [*CORE_LABELS, "opc_server_id"],
buckets=[0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0] buckets=[0.01, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0]

View File

@@ -1,7 +1,7 @@
""" """
Laborious Worker Module Model Manager Worker Module
This module provides the main worker implementation for the Sientia DataOps Laborious system. This module provides the main worker implementation for the Sientia DataOps Model Manager system.
It orchestrates Temporal workers, manages task queues, and handles the lifecycle of It orchestrates Temporal workers, manages task queues, and handles the lifecycle of
prediction and retraining workflows. prediction and retraining workflows.
@@ -18,11 +18,11 @@ Key Features:
Environment Variables: Environment Variables:
- TEMPORAL_HOST: Temporal server address (default: localhost:7233) - TEMPORAL_HOST: Temporal server address (default: localhost:7233)
- TEMPORAL_NAMESPACE: Temporal namespace (default: laborious) - TEMPORAL_NAMESPACE: Temporal namespace (default: model_manager)
- POD_ID: Kubernetes pod identifier for metrics - POD_ID: Kubernetes pod identifier for metrics
- HTTP_METRICS_PORT: Prometheus metrics server port (default: 9090) - HTTP_METRICS_PORT: Prometheus metrics server port (default: 9090)
- HTTP_SDK_METRICS_PORT: Temporal SDK metrics port (default: 9091) - HTTP_SDK_METRICS_PORT: Temporal SDK metrics port (default: 9091)
- PROJECT_NAME: Project name for notifications (default: laborious) - PROJECT_NAME: Project name for notifications (default: model_manager)
""" """
from temporalio import workflow, client from temporalio import workflow, client
@@ -56,7 +56,7 @@ SDK_METRICS_PORT = int(os.getenv('HTTP_SDK_METRICS_PORT', "9091"))
async def main(): async def main():
""" """
Main entry point for the Laborious worker application. Main entry point for the Model Manager worker application.
This function initializes and starts all components of the worker: This function initializes and starts all components of the worker:
1. Sets up logging and metadata 1. Sets up logging and metadata
@@ -97,7 +97,7 @@ async def main():
connection_string=mongo_config['connection_string'], connection_string=mongo_config['connection_string'],
database=mongo_config['database_name'], database=mongo_config['database_name'],
logger=logger, logger=logger,
project_name=os.getenv('PROJECT_NAME', 'laborious') project_name=os.getenv('PROJECT_NAME', 'model-manager')
) )
logger.custom_info('Starting Activities...', metadata) logger.custom_info('Starting Activities...', metadata)
@@ -127,7 +127,7 @@ async def main():
temporal_client = await client.Client.connect( temporal_client = await client.Client.connect(
target_host=host, target_host=host,
namespace=os.getenv('TEMPORAL_NAMESPACE', 'laborious'), namespace=os.getenv('TEMPORAL_NAMESPACE', 'model-manager'),
runtime=new_runtime runtime=new_runtime
) )
@@ -214,7 +214,7 @@ def start_prometheus_server():
and sets the application health metric to indicate the service is running. and sets the application health metric to indicate the service is running.
The server exposes metrics that can be scraped by Prometheus for monitoring The server exposes metrics that can be scraped by Prometheus for monitoring
the health and performance of the Laborious worker. the health and performance of the Model Manager worker.
Environment Variables: Environment Variables:
HTTP_METRICS_PORT: Port for the metrics server (default: 9090) HTTP_METRICS_PORT: Port for the metrics server (default: 9090)

View File

@@ -10,7 +10,7 @@ with workflow.unsafe.imports_passed_through():
@workflow.defn(name="minimal_retrain") @workflow.defn(name="minimal_retrain")
class MinimalRetrain(): class MinimalRetrain():
""" """
Automated model retraining workflow for the Laborious system. Automated model retraining workflow for the Model Manager system.
This workflow implements a complete model retraining pipeline that loads This workflow implements a complete model retraining pipeline that loads
training data, executes model retraining, updates production models, training data, executes model retraining, updates production models,

View File

@@ -10,7 +10,7 @@ with workflow.unsafe.imports_passed_through():
@workflow.defn(name="predictions_batch") @workflow.defn(name="predictions_batch")
class PredictionsBatch(): class PredictionsBatch():
""" """
Main batch prediction workflow for the Laborious system. Main batch prediction workflow for the Model Manager system.
This workflow orchestrates the complete batch prediction process, handling This workflow orchestrates the complete batch prediction process, handling
data loading, configuration management, and workflow delegation. It serves data loading, configuration management, and workflow delegation. It serves

View File

@@ -10,7 +10,7 @@ with workflow.unsafe.imports_passed_through():
@workflow.defn(name="prediction_process") @workflow.defn(name="prediction_process")
class PredictionProcess(): class PredictionProcess():
""" """
Core prediction processing workflow for the Laborious system. Core prediction processing workflow for the Model Manager system.
This workflow implements the complete ML model inference pipeline, handling This workflow implements the complete ML model inference pipeline, handling
data quality validation, MLFlow model interactions, and prediction processing. data quality validation, MLFlow model interactions, and prediction processing.

View File

@@ -17,8 +17,8 @@ image:
imagePullSecrets: imagePullSecrets:
- name: docker-hub-secret - name: docker-hub-secret
# This is to override the chart name. # This is to override the chart name.
nameOverride: "sientia-laborious-worker" nameOverride: "sientia-model-manager-worker"
fullnameOverride: "sientia-laborious-worker" fullnameOverride: "sientia-model-manager-worker"
namespace: sientia namespace: sientia
# This section builds out the service account more information can be found here: https://kubernetes.io/docs/concepts/security/service-accounts/ # This section builds out the service account more information can be found here: https://kubernetes.io/docs/concepts/security/service-accounts/
@@ -31,7 +31,7 @@ serviceAccount:
annotations: {} annotations: {}
# The name of the service account to use. # The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template # If not set and create is true, a name is generated using the fullname template
name: "sientia-laborious-worker" name: "sientia-model-manager-worker"
# This is for setting Kubernetes Annotations to a Pod. # This is for setting Kubernetes Annotations to a Pod.
# For more information checkout: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/ # For more information checkout: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
@@ -149,7 +149,7 @@ serviceMonitor:
env: env:
# Entrypoint variables # Entrypoint variables
- name: GITHUB_REPO_URL - name: GITHUB_REPO_URL
value: "git@github.com:Aignosi/sientia-dataops-laborious_temporal.git" value: "git@github.com:Aignosi/sientia-dataops-model-manager.git"
- name: GITHUB_BRANCH - name: GITHUB_BRANCH
value: SIENTIAPDE-1222-ajustar-a-library-para-fazer-o-download-do-courier value: SIENTIAPDE-1222-ajustar-a-library-para-fazer-o-download-do-courier
- name: PYTHON_APP - name: PYTHON_APP
@@ -195,12 +195,12 @@ env:
- name: HTTP_SDK_METRICS_PORT - name: HTTP_SDK_METRICS_PORT
value: "9091" value: "9091"
- name: PROJECT_NAME - name: PROJECT_NAME
value: "sientia-laborious" value: "sientia-model-manager"
- name: TEMPORAL_HOST - name: TEMPORAL_HOST
value: "temporal-frontend.temporal.svc.cluster.local:7233" value: "temporal-frontend.temporal.svc.cluster.local:7233"
- name: TEMPORAL_NAMESPACE - name: TEMPORAL_NAMESPACE
value: "laborious" value: "model-manager"
- name: MONGODB_USERNAME - name: MONGODB_USERNAME
value: "root" value: "root"
@@ -215,15 +215,15 @@ env:
ssh: ssh:
enabled: true enabled: true
secretName: git-ssh-key-sientia-laborious-worker secretName: git-ssh-key-sientia-model-manager-worker
sshPath: /mnt/.ssh sshPath: /mnt/.ssh
knownHostsPath: /mnt/known_hosts knownHostsPath: /mnt/known_hosts
# kubectl create secret docker-registry docker-hub-secret --namespace sientia --docker-server=http://aignosi.azurecr.io --docker-username=aignosi --docker-password=5I5zpQ6sRaHqX1hD3dr+2mo647yO3FRc359/wu6gsP+ACRDRz5mp # kubectl create secret docker-registry docker-hub-secret --namespace sientia --docker-server=http://aignosi.azurecr.io --docker-username=aignosi --docker-password=5I5zpQ6sRaHqX1hD3dr+2mo647yO3FRc359/wu6gsP+ACRDRz5mp
# helm upgrade --install sientia-laborious-worker sientia/sientia-module -n sientia --create-namespace -f ./values.yaml --version 0.5.0 # helm upgrade --install sientia-model-manager-worker sientia/sientia-module -n sientia --create-namespace -f ./values.yaml --version 0.5.0
# kubectl create secret generic git-ssh-key-sientia-laborious-worker \ # kubectl create secret generic git-ssh-key-sientia-model-manager-worker \
# --namespace sientia \ # --namespace sientia \
# --from-file=ssh-privatekey=git_key \ # --from-file=ssh-privatekey=git_key \
# --type=kubernetes.io/ssh-auth # --type=kubernetes.io/ssh-auth