Commit Graph

33 Commits

Author SHA1 Message Date
Bruno Domingues
06571011f2 SIENTIAPDE-1430: Introduce static_threshold parameter for static window removal.
This parameter allows customizing the threshold (1-1000) used when rem_static_win is enabled, defaulting to 1 if null.
Updates include parameter definition, business rule validation, repository logic for passing the threshold, documentation in README.md and PIPELINE_PARAMS_CHANGELOG.md, and new unit and integration tests.
2025-12-19 15:44:18 -03:00
Bruno Domingues
9a77257920 SIENTIAPDE-1430: Update README to document new features, expanded validation, and enhanced testing.
*   Documents support for Polynomial Regression, per-variable lag configuration, date range filtering, and NaN treatment options.
*   Includes details on RCE drift metrics and comprehensive business validation rules for training parameters.
*   Describes new JSON-based integration test scenarios with batch execution and updated 100% code coverage.
*   Refines architecture overview and example workflow parameters.
2025-12-19 11:54:08 -03:00
Bruno Domingues
06fd08dc70 SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations. 2025-12-18 17:05:10 -03:00
Bruno Domingues
32aa66e5d8 SIENTIAPDE-1352: Update README with improved instructions, project structure details, and environment variable descriptions. 2025-12-04 19:31:35 -03:00
Bruno Domingues
5fb8aead22 SIENTIAPDE-1352: Update deploy workflow to use reusable workflow and add permissions, update README with CI/CD details. 2025-12-04 12:54:03 -03:00
Bruno Domingues
cdaafd12cc SIENTIAPDE-1352: Add deploy workflow and update README with local repository execution instructions. 2025-12-04 12:09:50 -03:00
Bruno Domingues
46df241f7d SIENTIAPDE-1352: Add documentation for local GitHub Actions testing with act.
This commit adds a new section to the README.md file detailing how to use the 'act' tool for local testing of GitHub Actions workflows. It includes installation instructions, configuration details, usage examples, and troubleshooting tips.
2025-12-02 17:28:04 -03:00
Bruno Domingues
486e1756c9 SIENTIAPDE-1350: Implement automated file cleanup workflow and update documentation. Added a new workflow for automated cleanup of stale files from MinIO and local filesystem, including scheduled execution and configurable retention. Updated the README with detailed information about the new workflow, its configuration, and related metrics. (118 additions, 5 deletions) 2025-11-27 10:22:38 -03:00
Bruno Domingues
7ff54a1112 SIENTIAPDE-1309: Improve Dockerignore, Gitignore, Dockerfile, README, model repository, tests, todo list and values.yaml files. 2025-11-07 15:08:50 -03:00
Bruno Domingues
c7f44a2423 SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations. 2025-11-06 15:34:03 -03:00
Bruno Domingues
a625ef6c19 SIENTIAPDE-1309: Add Helm chart documentation to README and update image tag in values.yaml 2025-11-05 21:26:58 -03:00
Bruno Domingues
d9d3df5723 SIENTIAPDE-1308: Initial Docker setup, improved .dockerignore, updated README, and minor fixes. 2025-11-04 14:07:37 -03:00
Bruno Domingues
9f56a128e6 SIENTIAPDE-1308: Update documentation, remove install_dependencies.sh script from README, correct Training class initialization, and update sientia-dataops-library version in requirements.txt. 2025-11-03 17:35:33 -03:00
Kou-Kinoshita
63fa264ec4 SIENTIAPDE-1241: Updated readme (removed port foward script)) 2025-10-29 16:27:56 -03:00
Bruno Domingues
9e31ab679b SIENTIAPDE-1255: Implement MLFlow artifact management and model persistence
This commit introduces a new model_repository.py to handle MLFlow artifact generation and model persistence. It also updates the README to reflect this change and modifies training_repository.py to separate training and MLFlow operations.
2025-10-17 09:11:54 -03:00
Bruno Domingues
a66b996cc1 SIENTIAPDE-1255: Refactor MLFlow activities for training operations and update metrics
This commit refactors the MLFlow activities to focus on model training rather than prediction operations. It removes prediction-related activities and metrics, and updates the MLFlow activity descriptions to reflect the change in focus. The README is also updated to reflect these changes.
2025-10-17 00:51:29 -03:00
Bruno Domingues
85a8ba1e68 SIENTIAPDE-1255: Refactor data quality gates to training focused metrics and repositories. This commit removes the data quality gates and filters, focusing on training-specific metrics and data repositories. It also updates the README to reflect these changes, including new training metrics and a streamlined data services section. 2025-10-17 00:39:06 -03:00
Bruno Domingues
e191e7849f SIENTIAPDE-1255: Refactor Model Manager to focus on ML model training pipeline and simplify architecture. This includes removing prediction workflows, updating core functionality, and improving parameter validation and resource management.
README.md | 411 deletions(-) 117 insertions(+)
1 file changed, 117 insertions(+), 411 deletions(-)
2025-10-16 18:26:56 -03:00
Bruno Domingues
8f1cc21bb1 SIENTIAPDE-1255: Add port forwarding setup script and documentation to README.md 2025-10-16 18:00:28 -03:00
Bruno Domingues
4ff796a56f SIENTIAPDE-1253: Update README.md to include documentation for the Train Model workflow and renumber the Minimal Retrain workflow. (+101, -1) 2025-10-15 16:35:13 -03:00
Bruno Domingues
780184769d SIENTIAPDE-1253: Expose workflow activity timeouts as environment variables and document them. 2025-10-15 16:03:01 -03:00
Bruno Domingues
bee6036205 SIENTIAPDE-1251: Implement ML model training activity and repository
This commit introduces the 'Training' activity and 'TrainingRepository' for handling ML model training operations within the Model Manager system.

- Added model_manager/activities/training.py for the Training activity, which extends BaseActivity and integrates with Temporal workflows.
- Added model_manager/utils/repository/training_repository.py for the TrainingRepository, which encapsulates the core training logic.
- Updated model_manager/activities/activities.py to include the Training activity in the main activities orchestrator.
- Updated README.md to document the new 'Training' component.
- Added unit tests for the new activity and repository.
2025-10-09 15:07:27 -03:00
Bruno Domingues
7e1eace77f SIENTIAPDE-1250: Implement experiment tracking activity with status updates, error handling, and model registration. This includes a unified update method, error message truncation, and integration into the main activities orchestrator. (236+, 2-) 2025-10-07 11:27:23 -03:00
Bruno Domingues
2628dc92d2 SIENTIAPDE-1250: Refactor: Remove 'laborious' directory from test structure and update README.md accordingly. 2025-10-07 10:02:38 -03:00
Bruno Domingues
ec5b9df125 SIENTIAPDE-1248: Refactor README.md to include detailed documentation on features, architecture, workflows, installation, code quality, testing, monitoring, configuration, development, troubleshooting, performance tuning, contributing, license, and support. 2025-10-04 14:45:42 -03:00
Bruno Domingues
9a1355fe9b SIENTIAPDE-1243: Add Temporal namespace setup guide to README and improve validation script. 2025-10-02 10:08:02 -03:00
Bruno Domingues
8f4ac3284e SIENTIAPDE-1243: Remove Redis dependency and related configurations. 2025-10-01 18:16:21 -03:00
Bruno Domingues
b102f79087 SIENTIAPDE-1243: Remove OPC server integration and add code quality tools.
This commit removes the OPC server integration from the Model Manager, including related activities, repositories, metrics, and configuration. It also adds code quality tools such as Ruff (linting/formatting), mypy (type checking), and Bandit (security analysis) along with a validation script and CI/CD integration for automated code validation. The README has been updated to reflect these changes.
2025-10-01 16:25:08 -03:00
Bruno Domingues
bc4d98f78d SIENTIAPDE-1243: Rename project from 'laborious' to 'model-manager' across codebase and configuration. This includes updating project names in environment variables, Makefiles, README, metrics, and Helm chart values. 2025-10-01 14:41:24 -03:00
Bruno Domingues
aba4a3f1a5 SIENTIAPDE-1243: Refactor: Rename 'laborious' package to 'model_manager'
This commit renames the 'laborious' package to 'model_manager' across the entire project. This includes renaming directories, modules, references in code, configuration files, and documentation to reflect the new package name. This change improves clarity and consistency within the project.
2025-10-01 14:29:24 -03:00
Bruno Domingues
e318b91c63 SIENTIAPDE-1243: Update README with correct repository name and installation instructions. 2025-10-01 14:11:36 -03:00
Bruno Domingues
93d0849c80 SIENTIAPDE-1243: Initial commit of the model manager project, adding core files and configurations.
This commit introduces the initial project structure, including:
- .env.example: Example environment configuration.
- .github/workflows/quality-gate.yml: CI workflow for quality checks.
- .gitignore: Specifies intentionally untracked files that Git should ignore.
- Makefile: Automation of tasks like docker builds.
- README.md: Project documentation.
- Source code for model management, activities, utils, worker and workflows.
- Test suite.
- Dockerfile for the simulator.
- sonar-project.properties: SonarQube configuration file.
- values.yaml: Helm chart values for deployment.
2025-09-30 14:55:38 -03:00
Bruno Domingues
e79b35a1fb Initial commit 2025-09-30 13:38:26 -03:00