c7f44a2423
SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations.
Bruno Domingues2025-11-06 15:34:03 -03:00
a625ef6c19
SIENTIAPDE-1309: Add Helm chart documentation to README and update image tag in values.yaml
Bruno Domingues2025-11-05 21:26:58 -03:00
005c33151c
Merge pull request #12 from Aignosi/feature/SIENTIAPDE-1308
Bruno Domingues
2025-11-04 15:42:06 -03:00
d9d3df5723
SIENTIAPDE-1308: Initial Docker setup, improved .dockerignore, updated README, and minor fixes.
Bruno Domingues2025-11-04 14:07:37 -03:00
9f56a128e6
SIENTIAPDE-1308: Update documentation, remove install_dependencies.sh script from README, correct Training class initialization, and update sientia-dataops-library version in requirements.txt.
Bruno Domingues2025-11-03 17:35:33 -03:00
12f8b21625
Merge pull request #11 from Aignosi/feature/SIENTIAPDE-1307
Bruno Domingues
2025-11-03 15:26:58 -03:00
f32d647d1c
SIENTIAPDE-1307: Implement metrics for training activities and workflow executions, tracking success/failure status.
Bruno Domingues2025-11-03 14:41:37 -03:00
d8e8d3c481
SIENTIAPDE-1307: Fix PytestUnraisableExceptionWarning in activity and experiment tracking tests by controlling exception raising in __del__ mocks.
Bruno Domingues2025-10-31 17:21:27 -03:00
a258036397
SIENTIAPDE-1307: Refactor: Remove redundant patching of SientiaMonitoring and Postgres init in tests, and add shutdown mock to metrics controller.
Bruno Domingues2025-10-31 17:15:59 -03:00
db581f60f9
SIENTIAPDE-1307: Integrate metrics controller and update sientia-dataops-library to 1.5.1. This change adds metrics collection capabilities to activities and updates the dataops library dependency. (18 files changed, 143 insertions(+), 16 deletions(-))
Bruno Domingues2025-10-31 17:06:26 -03:00
326f9f9dc6
Merge pull request #10 from Aignosi/feature/SIENTIAPDE-1241
Bruno Domingues
2025-10-30 17:04:51 -03:00
5b14f1016b
SIENTIAPDE-1241: Suppress PytestUnraisableExceptionWarning and update test data in training repository tests.
Bruno Domingues2025-10-30 15:22:59 -03:00
bb25a19d8c
SIENTIAPDE-1241: Simplify worker shutdown logic by removing conditional checks.
Bruno Domingues2025-10-30 15:14:43 -03:00
f5c1ac3615
SIENTIAPDE-1241: Remove model equation implementation documentation and add test coverage for _generate_artifacts in ModelRepository.
Bruno Domingues2025-10-30 14:56:10 -03:00
a80d65d7ba
SIENTIAPDE-1321: Added equation related methods
Kou-Kinoshita
2025-10-24 08:03:27 -03:00
7826e68954
SIENTIAPDE-1241: Add unit tests for ModelRepository with 100% coverage.
Bruno Domingues2025-10-23 15:09:40 -03:00
0a33bb86ed
SIENTIAPDE-1241: Add unit tests for TrainModelParams model with 100% coverage.
Bruno Domingues2025-10-23 11:18:19 -03:00
2262dcf4a0
SIENTIAPDE-1241: Add unit tests for logger_helper module.
Bruno Domingues2025-10-23 09:15:09 -03:00
a66c5abdfd
SIENTIAPDE-1241: Add unit tests for custom exceptions in tests/utils/test_exceptions.py
Bruno Domingues2025-10-23 00:30:48 -03:00
3fe139763a
SIENTIAPDE-1241: Add unit tests for the Training activity with 100% coverage.
Bruno Domingues2025-10-23 00:25:25 -03:00
a6f1563036
SIENTIAPDE-1241: Add unit tests for ExperimentTracking class with 100% coverage.
Bruno Domingues2025-10-23 00:18:14 -03:00
fc2fe33b59
SIENTIAPDE-1241: Add unit tests for Activities class and enhance validation script with fix and test options
Bruno Domingues2025-10-22 23:38:01 -03:00
f07bc8ff30
SIENTIAPDE-1241: Refactor: Remove redundant logging and fix duplicate logs
Bruno Domingues2025-10-22 22:39:26 -03:00
ac97092ebf
SIENTIAPDE-1241: Remove unused metadata fields and update todo list.
Bruno Domingues2025-10-22 21:51:29 -03:00
94697215aa
SIENTIAPDE-1241: Refactor: Improve documentation, exception handling, and configuration in model manager. This commit enhances clarity and robustness by adding detailed docstrings to methods, standardizing exception handling with custom types, and simplifying MLflow configuration.
Bruno Domingues2025-10-22 21:48:12 -03:00
758ffb10b6
SIENTIAPDE-1241: Fix: Correctly handle feature normalization and denormalization, and add scaler parameters to metadata
Bruno Domingues2025-10-22 21:26:11 -03:00
5789a13023
SIENTIAPDE-1241: refactor train_model workflow due to I/O errors.
Bruno Domingues2025-10-22 15:37:56 -03:00
f2a1c88ff3
Merge pull request #8 from Aignosi/feature/SIENTIAPDE-1255
Bruno Domingues
2025-10-21 09:42:17 -03:00
ab7c93480b
SIENTIAPDE-1255: Refactor: Move extra pip requirements to environment variable and use constants for preprocessor steps.
Bruno Domingues2025-10-21 09:20:23 -03:00
4686b438db
SIENTIAPDE-1255: Update quality gate workflow: Remove redundant repository and disk space cleanup, update cache key.
Bruno Domingues2025-10-20 22:35:20 -03:00
3e52253a98
SIENTIAPDE-1255: Add tests to ensure get_required_columns removes columns from self_operations and cross_operations.
Bruno Domingues2025-10-20 22:27:58 -03:00
614eb0cb7f
SIENTIAPDE-1255: Add unit tests for utils.split_train_test function
Bruno Domingues2025-10-20 22:22:38 -03:00
089f312575
SIENTIAPDE-1255: Add unit tests for the reports module
Bruno Domingues2025-10-20 22:20:26 -03:00
0de2ece063
SIENTIAPDE-1255: Add unit tests for sientia models (DataPreprocessor and LinearRegressionModel)
Bruno Domingues2025-10-20 20:59:55 -03:00
3c09e59f6a
SIENTIAPDE-1255: Add unit tests for the ModelServing class.
Bruno Domingues2025-10-20 20:24:50 -03:00
b1fe62f68b
SIENTIAPDE-1255: Add unit tests for sientia metrics module (MAE, MSE, R2)
Bruno Domingues2025-10-20 20:13:08 -03:00
b3e4bb8660
SIENTIAPDE-1255: Add comprehensive tests for __del__ methods in Activities and ExperimentTracking to ensure proper resource cleanup and exception handling.
Bruno Domingues2025-10-20 20:09:49 -03:00
4a32d712ba
SIENTIAPDE-1255: Implement object destructors for Activities and ExperimentTracking to prevent AttributeError during garbage collection, and mock workflow logger in tests to avoid RuntimeWarning.
Bruno Domingues2025-10-20 17:53:49 -03:00
52be5c4e8e
SIENTIAPDE-1255: Improve documentation for split_train_test function in utils.py
Bruno Domingues2025-10-20 17:41:05 -03:00
0dbe179367
SIENTIAPDE-1255: Add thread-safety documentation and I/O notes to Reports class, improve save_all_sections documentation and error handling.
Bruno Domingues2025-10-20 17:38:44 -03:00
2292ae57cc
SIENTIAPDE-1255: Add thread-safety documentation and in-place modification warnings to LinearRegressionModel and DataPreprocessor classes.
Bruno Domingues2025-10-20 17:27:58 -03:00
56f3db350c
SIENTIAPDE-1255: Refactor ModelServing class to improve thread safety, resource management, and documentation. Adds context manager for experiment saving and clarifies thread-safety concerns.
Bruno Domingues2025-10-20 17:16:45 -03:00
56a21a16da
SIENTIAPDE-1255: Integrate sientia-mlops-library into model-manager, adding model serving, reporting, and updated model definitions.
Bruno Domingues2025-10-20 16:55:52 -03:00
9e31ab679b
SIENTIAPDE-1255: Implement MLFlow artifact management and model persistence
Bruno Domingues2025-10-17 09:11:54 -03:00
a66b996cc1
SIENTIAPDE-1255: Refactor MLFlow activities for training operations and update metrics
Bruno Domingues2025-10-17 00:51:29 -03:00
85a8ba1e68
SIENTIAPDE-1255: Refactor data quality gates to training focused metrics and repositories. This commit removes the data quality gates and filters, focusing on training-specific metrics and data repositories. It also updates the README to reflect these changes, including new training metrics and a streamlined data services section.
Bruno Domingues2025-10-17 00:39:06 -03:00
e191e7849f
SIENTIAPDE-1255: Refactor Model Manager to focus on ML model training pipeline and simplify architecture. This includes removing prediction workflows, updating core functionality, and improving parameter validation and resource management. README.md | 411 deletions(-) 117 insertions(+) 1 file changed, 117 insertions(+), 411 deletions(-)
Bruno Domingues2025-10-16 18:26:56 -03:00
7abd951806
SIENTIAPDE-1255: Refactor worker to support only the train_model-queue and remove prediction workflows.
Bruno Domingues2025-10-16 18:15:52 -03:00
8f1cc21bb1
SIENTIAPDE-1255: Add port forwarding setup script and documentation to README.md
Bruno Domingues2025-10-16 18:00:28 -03:00
9ee12373b4
Merge pull request #7 from Aignosi/feature/SIENTIAPDE-1253
Bruno Domingues
2025-10-15 17:04:49 -03:00
4ff796a56f
SIENTIAPDE-1253: Update README.md to include documentation for the Train Model workflow and renumber the Minimal Retrain workflow. (+101, -1)
Bruno Domingues2025-10-15 16:35:13 -03:00
98c216733b
SIENTIAPDE-1253: Enhance error logging in train_model workflow with rich context for debugging.
Bruno Domingues2025-10-15 16:19:15 -03:00
305a39b44c
SIENTIAPDE-1253: Implement business rule validation for training parameters. Adds a validate_business_rules method to the TrainModelParams class to enforce constraints on training parameters, improving data integrity and preventing errors. Also updates tests to include target variable in variable columns and adds tests for business rule validations.
Bruno Domingues2025-10-15 16:16:09 -03:00
4b28d5f1c8
SIENTIAPDE-1253: Implement granular retry policies for activities in train_model workflow.
Bruno Domingues2025-10-15 16:06:13 -03:00
780184769d
SIENTIAPDE-1253: Expose workflow activity timeouts as environment variables and document them.
Bruno Domingues2025-10-15 16:03:01 -03:00
1ec40bfb2a
SIENTIAPDE-1253: Implement activity for cleaning up temporary run directory and integrate into train model workflow. This change introduces a new activity for idempotent cleanup of the run directory after model training, replacing the direct directory removal in the workflow. This improves determinism and error handling. Also includes unit tests for the new activity.
Bruno Domingues2025-10-15 15:49:05 -03:00
8ea98360c3
SIENTIAPDE-1253: Refactor training workflow and activities to raise exceptions on failure
Bruno Domingues2025-10-15 15:19:29 -03:00
61267ec49d
SIENTIAPDE-1253: Enforce TrainModelParams object in Training activity and tests, removing dict conversion.
Bruno Domingues2025-10-14 10:03:34 -03:00
014e1cd910
Merge pull request #6 from Aignosi/feature/SIENTIAPDE-1252
Bruno Domingues
2025-10-13 14:44:39 -03:00
9d4ec19586
SIENTIAPDE-1252: Improve caching in quality gate, add alt text to logo, and change exception type in MLflow repository.
Bruno Domingues2025-10-13 10:54:38 -03:00
b3c749872c
SIENTIAPDE-1252: Implement save_model activity to save trained models to MLflow with comprehensive error handling and add corresponding unit tests.
Bruno Domingues2025-10-13 10:18:45 -03:00
3fd5ab79fe
SIENTIAPDE-1252: Implement artifact generation and MLflow logging for model training results
Bruno Domingues2025-10-10 17:56:09 -03:00
52b7b23261
SIENTIAPDE-1252: Update Python setup action and simplify dependency caching in quality gate workflow
Bruno Domingues2025-10-09 17:52:35 -03:00
94e11df803
SIENTIAPDE-1252: Remove experiment_description from TrainModelParams and related tests.
Bruno Domingues2025-10-09 17:48:43 -03:00
5b0cd92dfe
Merge pull request #5 from Aignosi/feature/SIENTIAPDE-1251
Bruno Domingues
2025-10-09 17:09:56 -03:00
bee6036205
SIENTIAPDE-1251: Implement ML model training activity and repository
Bruno Domingues2025-10-09 15:07:27 -03:00
c970754e1b
Merge pull request #4 from Aignosi/feature/SIENTIAPDE-1250
Bruno Domingues
2025-10-07 14:51:14 -03:00
97f5ee08f1
SIENTIAPDE-1250: Fix: Update pip cache key and replace pytz with datetime.UTC in experiment tracking.
Bruno Domingues2025-10-07 14:12:35 -03:00
7e1eace77f
SIENTIAPDE-1250: Implement experiment tracking activity with status updates, error handling, and model registration. This includes a unified update method, error message truncation, and integration into the main activities orchestrator. (236+, 2-)
Bruno Domingues2025-10-07 11:27:23 -03:00
2628dc92d2
SIENTIAPDE-1250: Refactor: Remove 'laborious' directory from test structure and update README.md accordingly.
Bruno Domingues2025-10-07 10:02:38 -03:00
851f34c80e
Merge pull request #3 from Aignosi/feature/SIENTIAPDE-1249
Bruno Domingues
2025-10-06 19:53:28 -03:00
04c94efd98
SIENTIAPDE-1249: Refactor TrainModelParams to use dataclass and add from_dict method for validation, remove no-cache-dir from pip install in quality gate workflow, and rename X_train/X_test to x_train/x_test in TrainModelResult.
Bruno Domingues2025-10-06 19:11:02 -03:00
d8583cdae7
SIENTIAPDE-1249: Implement data models for Model Manager and add unit tests. This commit introduces data transfer objects (DTOs) and model classes for experiment status, training parameters, and training results, along with corresponding unit tests to ensure their correct behavior.
Bruno Domingues2025-10-06 16:49:18 -03:00
18ccfed78a
Merge pull request #2 from Aignosi/feature/SIENTIAPDE-1248
Bruno Domingues
2025-10-06 11:27:19 -03:00
520975e7aa
SIENTIAPDE-1248: Refactor MinIO activity to raise specific exception types for better error handling
Bruno Domingues2025-10-06 10:58:48 -03:00
ec5b9df125
SIENTIAPDE-1248: Refactor README.md to include detailed documentation on features, architecture, workflows, installation, code quality, testing, monitoring, configuration, development, troubleshooting, performance tuning, contributing, license, and support.
Bruno Domingues2025-10-04 14:45:42 -03:00
0db7f7462a
SIENTIAPDE-1248: Add worker tests, including Prometheus server startup, main function execution, exception handling, worker creation, activity initialization, and environment variable usage.
Bruno Domingues2025-10-04 13:02:24 -03:00
Bruno Domingues
Bruno Domingues