Commit Graph

12 Commits

Author SHA1 Message Date
Bruno Domingues
7a0961f29d SIENTIAPDE-1717: Remove MinIO cleanup functionality and associated components. This change streamlines the cleanup workflow to focus solely on local temporary directories, removes the ModelTrainingError exception, and updates related configurations, documentation, and tests. 2026-03-30 14:14:10 -03:00
Bruno Domingues
b028dc1b7d SIENTIAPDE-1350: Implement scheduled cleanup workflow using Temporal schedules. Adds schedule creation to worker startup and configures environment variables. 2025-11-25 18:12:53 -03:00
Bruno Domingues
463906073e SIENTIAPDE-1350: Refactor: Improve logging, configuration, and resource management. Includes .env updates, client initialization logging, and resource closing. 2025-11-24 23:08:29 -03:00
Bruno Domingues
2ee66552d2 SIENTIAPDE-1350: Configure separate task queues for train and cleanup workers, update sientia-dataops-library to 1.6.1 and refactor prometheus server startup to use logger. 2025-11-24 11:21:43 -03:00
Bruno Domingues
53f49d7a97 SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics. 2025-11-19 18:38:00 -03:00
Bruno Domingues
5789a13023 SIENTIAPDE-1241: refactor train_model workflow due to I/O errors. 2025-10-22 15:37:56 -03:00
Bruno Domingues
ab7c93480b SIENTIAPDE-1255: Refactor: Move extra pip requirements to environment variable and use constants for preprocessor steps. 2025-10-21 09:20:23 -03:00
Bruno Domingues
780184769d SIENTIAPDE-1253: Expose workflow activity timeouts as environment variables and document them. 2025-10-15 16:03:01 -03:00
Bruno Domingues
9afe711075 SIENTIAPDE-1248: Integrate MinIO for object storage and add related configurations
This commit introduces MinIO integration for object storage within the Model Manager system. It includes:

- Added MinIO activity class for file operations (fetch, delete).
- Updated Activities orchestrator to include MinIO activities.
- Added MinIO configuration builder to utils/connectors_config.py.
- Added environment variables for MinIO configuration in .env.example.
- Added boto3 and botocore dependencies to requirements.txt.
- Added unit tests for MinIO activities.
2025-10-03 21:04:27 -03:00
Bruno Domingues
b102f79087 SIENTIAPDE-1243: Remove OPC server integration and add code quality tools.
This commit removes the OPC server integration from the Model Manager, including related activities, repositories, metrics, and configuration. It also adds code quality tools such as Ruff (linting/formatting), mypy (type checking), and Bandit (security analysis) along with a validation script and CI/CD integration for automated code validation. The README has been updated to reflect these changes.
2025-10-01 16:25:08 -03:00
Bruno Domingues
bc4d98f78d SIENTIAPDE-1243: Rename project from 'laborious' to 'model-manager' across codebase and configuration. This includes updating project names in environment variables, Makefiles, README, metrics, and Helm chart values. 2025-10-01 14:41:24 -03:00
Bruno Domingues
93d0849c80 SIENTIAPDE-1243: Initial commit of the model manager project, adding core files and configurations.
This commit introduces the initial project structure, including:
- .env.example: Example environment configuration.
- .github/workflows/quality-gate.yml: CI workflow for quality checks.
- .gitignore: Specifies intentionally untracked files that Git should ignore.
- Makefile: Automation of tasks like docker builds.
- README.md: Project documentation.
- Source code for model management, activities, utils, worker and workflows.
- Test suite.
- Dockerfile for the simulator.
- sonar-project.properties: SonarQube configuration file.
- values.yaml: Helm chart values for deployment.
2025-09-30 14:55:38 -03:00