Bruno Domingues
d2c4803a75
SIENTIAPDE-1350: Add unit tests for the Cleanup activity, ensuring 100% code coverage, and update todo-list.
2025-11-25 16:01:10 -03:00
Bruno Domingues
f3c88885ea
SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config.
2025-11-25 00:44:55 -03:00
Bruno Domingues
2ee66552d2
SIENTIAPDE-1350: Configure separate task queues for train and cleanup workers, update sientia-dataops-library to 1.6.1 and refactor prometheus server startup to use logger.
2025-11-24 11:21:43 -03:00
Bruno Domingues
53f49d7a97
SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics.
2025-11-19 18:38:00 -03:00
Bruno Domingues
7ff54a1112
SIENTIAPDE-1309: Improve Dockerignore, Gitignore, Dockerfile, README, model repository, tests, todo list and values.yaml files.
2025-11-07 15:08:50 -03:00
Bruno Domingues
c7f44a2423
SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations.
2025-11-06 15:34:03 -03:00
Bruno Domingues
f32d647d1c
SIENTIAPDE-1307: Implement metrics for training activities and workflow executions, tracking success/failure status.
2025-11-03 14:41:37 -03:00
Bruno Domingues
d8e8d3c481
SIENTIAPDE-1307: Fix PytestUnraisableExceptionWarning in activity and experiment tracking tests by controlling exception raising in __del__ mocks.
2025-10-31 17:21:27 -03:00
Bruno Domingues
a258036397
SIENTIAPDE-1307: Refactor: Remove redundant patching of SientiaMonitoring and Postgres init in tests, and add shutdown mock to metrics controller.
2025-10-31 17:15:59 -03:00
Bruno Domingues
db581f60f9
SIENTIAPDE-1307: Integrate metrics controller and update sientia-dataops-library to 1.5.1. This change adds metrics collection capabilities to activities and updates the dataops library dependency. (18 files changed, 143 insertions(+), 16 deletions(-))
2025-10-31 17:06:26 -03:00
Bruno Domingues
5b14f1016b
SIENTIAPDE-1241: Suppress PytestUnraisableExceptionWarning and update test data in training repository tests.
2025-10-30 15:22:59 -03:00
Bruno Domingues
f5c1ac3615
SIENTIAPDE-1241: Remove model equation implementation documentation and add test coverage for _generate_artifacts in ModelRepository.
2025-10-30 14:56:10 -03:00
Kou-Kinoshita
a5c0cb6e47
SIENTIAPDE-1241: Fixed formatting and linted code
2025-10-30 10:38:29 -03:00
Kou-Kinoshita
17a551d6f9
SIENTIAPDE-1241: Added remaining training repository tests
2025-10-30 10:29:51 -03:00
Kou-Kinoshita
7e9d6683e4
SIENTIAPDE-1241: Added worker test
2025-10-30 10:21:39 -03:00
Kou-Kinoshita
c63389620a
SIENTIAPDE-1241: Train_model tests
2025-10-30 10:11:07 -03:00
Kou-Kinoshita
928c679223
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-30 09:58:46 -03:00
Kou-Kinoshita
c699a47261
SIENTIAPDE-1241: worker tests
2025-10-30 09:52:34 -03:00
Kou-Kinoshita
94a407f1b8
SIENTIAPDE-1241: Storage repository tests
2025-10-30 09:49:10 -03:00
Kou-Kinoshita
87c64eac2e
SIENTIAPDE-1241: Metrics test
2025-10-30 09:47:23 -03:00
Kou-Kinoshita
1f6765dc86
SIENTIAPDE-1241: Linted files
2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64
SIENTIAPDE-1241: Formatted files
2025-10-29 11:55:03 -03:00
Kou-Kinoshita
e8250d8dd7
SIENTIAPDE-1241: fixed test
2025-10-29 11:49:08 -03:00
Kou-Kinoshita
64590032f3
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-29 11:46:29 -03:00
Kou-Kinoshita
61364671db
SIENTIAPDE-1241: Added remaining trainig_repository tests and some model_repository tests
2025-10-29 11:41:36 -03:00
Kou-Kinoshita
4fdefd8263
SIENTIAPDE-1241: Added training repository tests
2025-10-29 11:32:24 -03:00
Kou-Kinoshita
63a5de5938
SIENTIAPDE-1321: Fixed tests
2025-10-29 10:29:43 -03:00
Kou-Kinoshita
734221e32d
SIENTIAPDE-1321: Updated tests
2025-10-27 19:45:47 -03:00
Bruno Domingues
7826e68954
SIENTIAPDE-1241: Add unit tests for ModelRepository with 100% coverage.
2025-10-23 15:09:40 -03:00
Bruno Domingues
0a33bb86ed
SIENTIAPDE-1241: Add unit tests for TrainModelParams model with 100% coverage.
2025-10-23 11:18:19 -03:00
Bruno Domingues
2262dcf4a0
SIENTIAPDE-1241: Add unit tests for logger_helper module.
2025-10-23 09:15:09 -03:00
Bruno Domingues
a66c5abdfd
SIENTIAPDE-1241: Add unit tests for custom exceptions in tests/utils/test_exceptions.py
2025-10-23 00:30:48 -03:00
Bruno Domingues
3fe139763a
SIENTIAPDE-1241: Add unit tests for the Training activity with 100% coverage.
2025-10-23 00:25:25 -03:00
Bruno Domingues
a6f1563036
SIENTIAPDE-1241: Add unit tests for ExperimentTracking class with 100% coverage.
...
This commit introduces a new test file tests/activities/test_experiment_tracking.py containing comprehensive unit tests for the ExperimentTracking class, achieving 100% test coverage. The tests cover initialization, deletion, execution of updates, and various scenarios for updating experiment run status, including error handling and edge cases.
2025-10-23 00:18:14 -03:00
Bruno Domingues
fc2fe33b59
SIENTIAPDE-1241: Add unit tests for Activities class and enhance validation script with fix and test options
...
This commit introduces a new test suite for the Activities class, achieving 100% coverage. Additionally, the validation script (validate.sh) is enhanced with new options:
- --fix: Automatically fixes code formatting and linting issues using Ruff.
- --only-tests: Runs only the unit tests, skipping other validation steps.
The validation script now also supports skipping tests and provides more informative output.
2025-10-22 23:38:01 -03:00
Bruno Domingues
5789a13023
SIENTIAPDE-1241: refactor train_model workflow due to I/O errors.
2025-10-22 15:37:56 -03:00
Bruno Domingues
3e52253a98
SIENTIAPDE-1255: Add tests to ensure get_required_columns removes columns from self_operations and cross_operations.
2025-10-20 22:27:58 -03:00
Bruno Domingues
614eb0cb7f
SIENTIAPDE-1255: Add unit tests for utils.split_train_test function
2025-10-20 22:22:38 -03:00
Bruno Domingues
089f312575
SIENTIAPDE-1255: Add unit tests for the reports module
2025-10-20 22:20:26 -03:00
Bruno Domingues
0de2ece063
SIENTIAPDE-1255: Add unit tests for sientia models (DataPreprocessor and LinearRegressionModel)
2025-10-20 20:59:55 -03:00
Bruno Domingues
3c09e59f6a
SIENTIAPDE-1255: Add unit tests for the ModelServing class.
2025-10-20 20:24:50 -03:00
Bruno Domingues
b1fe62f68b
SIENTIAPDE-1255: Add unit tests for sientia metrics module (MAE, MSE, R2)
2025-10-20 20:13:08 -03:00
Bruno Domingues
b3e4bb8660
SIENTIAPDE-1255: Add comprehensive tests for __del__ methods in Activities and ExperimentTracking to ensure proper resource cleanup and exception handling.
2025-10-20 20:09:49 -03:00
Bruno Domingues
4a32d712ba
SIENTIAPDE-1255: Implement object destructors for Activities and ExperimentTracking to prevent AttributeError during garbage collection, and mock workflow logger in tests to avoid RuntimeWarning.
2025-10-20 17:53:49 -03:00
Bruno Domingues
9e31ab679b
SIENTIAPDE-1255: Implement MLFlow artifact management and model persistence
...
This commit introduces a new model_repository.py to handle MLFlow artifact generation and model persistence. It also updates the README to reflect this change and modifies training_repository.py to separate training and MLFlow operations.
2025-10-17 09:11:54 -03:00
Bruno Domingues
a66b996cc1
SIENTIAPDE-1255: Refactor MLFlow activities for training operations and update metrics
...
This commit refactors the MLFlow activities to focus on model training rather than prediction operations. It removes prediction-related activities and metrics, and updates the MLFlow activity descriptions to reflect the change in focus. The README is also updated to reflect these changes.
2025-10-17 00:51:29 -03:00
Bruno Domingues
85a8ba1e68
SIENTIAPDE-1255: Refactor data quality gates to training focused metrics and repositories. This commit removes the data quality gates and filters, focusing on training-specific metrics and data repositories. It also updates the README to reflect these changes, including new training metrics and a streamlined data services section.
2025-10-17 00:39:06 -03:00
Bruno Domingues
e191e7849f
SIENTIAPDE-1255: Refactor Model Manager to focus on ML model training pipeline and simplify architecture. This includes removing prediction workflows, updating core functionality, and improving parameter validation and resource management.
...
README.md | 411 deletions(-) 117 insertions(+)
1 file changed, 117 insertions(+), 411 deletions(-)
2025-10-16 18:26:56 -03:00
Bruno Domingues
7abd951806
SIENTIAPDE-1255: Refactor worker to support only the train_model-queue and remove prediction workflows.
2025-10-16 18:15:52 -03:00
Bruno Domingues
305a39b44c
SIENTIAPDE-1253: Implement business rule validation for training parameters. Adds a validate_business_rules method to the TrainModelParams class to enforce constraints on training parameters, improving data integrity and preventing errors. Also updates tests to include target variable in variable columns and adds tests for business rule validations.
2025-10-15 16:16:09 -03:00
