Bruno Domingues
cce350bfb1
SIENTIAPDE-1717: Remove FILE_DELETE_ERROR from ExperimentStatus enum and related tests.
2026-03-30 14:33:19 -03:00
Bruno Domingues
f5eb8d1e5b
SIENTIAPDE-1717: Remove list_bucket_objects method from StorageRepository and its associated test.
2026-03-30 14:18:14 -03:00
Bruno Domingues
7a0961f29d
SIENTIAPDE-1717: Remove MinIO cleanup functionality and associated components. This change streamlines the cleanup workflow to focus solely on local temporary directories, removes the ModelTrainingError exception, and updates related configurations, documentation, and tests.
2026-03-30 14:14:10 -03:00
Bruno Domingues
41c4490cad
SIENTIAPDE-1579: Fix SientiaMlException propagation in ModelServing.search_runs_by_name to correctly raise the exception with its message, resolving a TypeError. Enhance training test script to support scenario-specific CSV files for different test cases. Update search_runs_by_name return type hint and apply minor code formatting.
2026-02-19 21:13:41 -03:00
Kou Kinoshita
14f3fbadbe
SIENTIAPDE-1579: Switched from replace logic to validation + mapping
2026-02-18 18:57:41 -03:00
Kou Kinoshita
5dab1ee576
SIENTIAPDE-1579?: removed duplicated method
2026-02-18 18:18:39 -03:00
Kou Kinoshita
a7922ac3ce
SIENTIAPDE-1579: Lint fix
2026-02-18 17:50:58 -03:00
Kou Kinoshita
665870320f
SIENTIAPDE-1579: Updated tests and refactored apply filters method
2026-02-18 17:44:46 -03:00
Kou Kinoshita
43cbc31e10
SIENTIAPDE-1579: Lint fixes and formatting
2026-02-18 17:31:58 -03:00
Kou Kinoshita
c6d6f94e05
SIENTIAPDE-1579: Updated tests
2026-02-18 15:05:37 -03:00
Bruno Domingues
df6bf1daba
SIENTIAPDE-1430: Refactor static threshold calculation logic and enhance test coverage
2025-12-19 16:19:10 -03:00
Bruno Domingues
06571011f2
SIENTIAPDE-1430: Introduce static_threshold parameter for static window removal.
...
This parameter allows customizing the threshold (1-1000) used when rem_static_win is enabled, defaulting to 1 if null.
Updates include parameter definition, business rule validation, repository logic for passing the threshold, documentation in README.md and PIPELINE_PARAMS_CHANGELOG.md, and new unit and integration tests.
2025-12-19 15:44:18 -03:00
Bruno Domingues
13941fc3f7
SIENTIAPDE-1430: Add unit tests for _configure_datetime_index (handling various datetime column formats), _init_data_preprocessor (with removed intervals), and _extract_model_equation (supporting polynomial features) in TrainingRepository.
2025-12-18 18:58:12 -03:00
Bruno Domingues
71b654f24c
SIENTIAPDE-1430: Add test case for ValueError when y_train_pred is None during artifact data initialization.
2025-12-18 18:48:42 -03:00
Bruno Domingues
573834fb99
SIENTIAPDE-1430: Add comprehensive tests for TrainModelParams.validate_business_rules to improve coverage.
2025-12-18 18:46:01 -03:00
Bruno Domingues
06fd08dc70
SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations.
2025-12-18 17:05:10 -03:00
Bruno Domingues
6e8f87b2a3
SIENTIAPDE-1430: Implement advanced model training capabilities and enhanced data preprocessing. This includes support for Polynomial Regression with configurable degree and interaction terms, flexible per-variable lag configurations, and new data filtering options by date range and removed intervals. Comprehensive business validations are now enforced for all parameters, and MLflow logging has been extended to capture these detailed configurations. Additionally, Reduced Coulomb Energy (RCE) metrics are added for drift detection, with a new changelog documenting all pipeline parameter updates.
2025-12-17 21:39:43 -03:00
Bruno Domingues
d0b5b74f88
SIENTIAPDE-1350: Add tests for close method and paginated list_bucket_objects
2025-11-25 16:04:59 -03:00
Bruno Domingues
f3c88885ea
SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config.
2025-11-25 00:44:55 -03:00
Bruno Domingues
7ff54a1112
SIENTIAPDE-1309: Improve Dockerignore, Gitignore, Dockerfile, README, model repository, tests, todo list and values.yaml files.
2025-11-07 15:08:50 -03:00
Bruno Domingues
c7f44a2423
SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations.
2025-11-06 15:34:03 -03:00
Bruno Domingues
5b14f1016b
SIENTIAPDE-1241: Suppress PytestUnraisableExceptionWarning and update test data in training repository tests.
2025-10-30 15:22:59 -03:00
Bruno Domingues
f5c1ac3615
SIENTIAPDE-1241: Remove model equation implementation documentation and add test coverage for _generate_artifacts in ModelRepository.
2025-10-30 14:56:10 -03:00
Kou-Kinoshita
a5c0cb6e47
SIENTIAPDE-1241: Fixed formatting and linted code
2025-10-30 10:38:29 -03:00
Kou-Kinoshita
17a551d6f9
SIENTIAPDE-1241: Added remaining training repository tests
2025-10-30 10:29:51 -03:00
Kou-Kinoshita
c63389620a
SIENTIAPDE-1241: Train_model tests
2025-10-30 10:11:07 -03:00
Kou-Kinoshita
928c679223
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-30 09:58:46 -03:00
Kou-Kinoshita
94a407f1b8
SIENTIAPDE-1241: Storage repository tests
2025-10-30 09:49:10 -03:00
Kou-Kinoshita
1f6765dc86
SIENTIAPDE-1241: Linted files
2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64
SIENTIAPDE-1241: Formatted files
2025-10-29 11:55:03 -03:00
Kou-Kinoshita
e8250d8dd7
SIENTIAPDE-1241: fixed test
2025-10-29 11:49:08 -03:00
Kou-Kinoshita
64590032f3
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-29 11:46:29 -03:00
Kou-Kinoshita
61364671db
SIENTIAPDE-1241: Added remaining trainig_repository tests and some model_repository tests
2025-10-29 11:41:36 -03:00
Kou-Kinoshita
4fdefd8263
SIENTIAPDE-1241: Added training repository tests
2025-10-29 11:32:24 -03:00
Kou-Kinoshita
63a5de5938
SIENTIAPDE-1321: Fixed tests
2025-10-29 10:29:43 -03:00
Kou-Kinoshita
734221e32d
SIENTIAPDE-1321: Updated tests
2025-10-27 19:45:47 -03:00
Bruno Domingues
7826e68954
SIENTIAPDE-1241: Add unit tests for ModelRepository with 100% coverage.
2025-10-23 15:09:40 -03:00
Bruno Domingues
0a33bb86ed
SIENTIAPDE-1241: Add unit tests for TrainModelParams model with 100% coverage.
2025-10-23 11:18:19 -03:00
Bruno Domingues
2262dcf4a0
SIENTIAPDE-1241: Add unit tests for logger_helper module.
2025-10-23 09:15:09 -03:00
Bruno Domingues
a66c5abdfd
SIENTIAPDE-1241: Add unit tests for custom exceptions in tests/utils/test_exceptions.py
2025-10-23 00:30:48 -03:00
Bruno Domingues
5789a13023
SIENTIAPDE-1241: refactor train_model workflow due to I/O errors.
2025-10-22 15:37:56 -03:00
Bruno Domingues
9e31ab679b
SIENTIAPDE-1255: Implement MLFlow artifact management and model persistence
...
This commit introduces a new model_repository.py to handle MLFlow artifact generation and model persistence. It also updates the README to reflect this change and modifies training_repository.py to separate training and MLFlow operations.
2025-10-17 09:11:54 -03:00
Bruno Domingues
85a8ba1e68
SIENTIAPDE-1255: Refactor data quality gates to training focused metrics and repositories. This commit removes the data quality gates and filters, focusing on training-specific metrics and data repositories. It also updates the README to reflect these changes, including new training metrics and a streamlined data services section.
2025-10-17 00:39:06 -03:00
Bruno Domingues
305a39b44c
SIENTIAPDE-1253: Implement business rule validation for training parameters. Adds a validate_business_rules method to the TrainModelParams class to enforce constraints on training parameters, improving data integrity and preventing errors. Also updates tests to include target variable in variable columns and adds tests for business rule validations.
2025-10-15 16:16:09 -03:00
Bruno Domingues
8ea98360c3
SIENTIAPDE-1253: Refactor training workflow and activities to raise exceptions on failure
...
This commit refactors the training workflow and associated activities to raise exceptions on failure instead of returning success/failure dictionaries. This allows the Temporal workflow to handle errors more effectively and ensures that the workflow stops when a critical error occurs.
Key changes:
- The train_model workflow is introduced to orchestrate the entire training process, including parameter validation, data download, model training, and model saving.
- The validate_train_params activity is added to validate and convert training parameters.
- The train_model and save_model activities are updated to raise exceptions on failure.
- The ExperimentStatus enum is updated to include a new status for orchestrator validation errors.
- The tests are updated to reflect the new exception-based error handling.
- The activities now return the TrainModelResult directly instead of a dictionary.
2025-10-15 15:19:29 -03:00
Bruno Domingues
9d4ec19586
SIENTIAPDE-1252: Improve caching in quality gate, add alt text to logo, and change exception type in MLflow repository.
2025-10-13 10:54:38 -03:00
Bruno Domingues
3fd5ab79fe
SIENTIAPDE-1252: Implement artifact generation and MLflow logging for model training results
...
This commit introduces artifact generation and MLflow logging capabilities to the model training process. It includes the following changes:
- Added methods to generate reports, save data files, and log model parameters, metrics, models, and artifacts to MLflow.
- Implemented error handling for various scenarios, such as missing files, invalid data, and MLflow connection errors.
- Created a new 'header.html' file for report styling and navigation.
- Modified the 'model_repository.py' file to include the new artifact generation and MLflow logging methods.
- Added comprehensive unit tests to ensure the functionality and robustness of the new features.
2025-10-10 17:56:09 -03:00
Bruno Domingues
94e11df803
SIENTIAPDE-1252: Remove experiment_description from TrainModelParams and related tests.
2025-10-09 17:48:43 -03:00
Bruno Domingues
bee6036205
SIENTIAPDE-1251: Implement ML model training activity and repository
...
This commit introduces the 'Training' activity and 'TrainingRepository' for handling ML model training operations within the Model Manager system.
- Added model_manager/activities/training.py for the Training activity, which extends BaseActivity and integrates with Temporal workflows.
- Added model_manager/utils/repository/training_repository.py for the TrainingRepository, which encapsulates the core training logic.
- Updated model_manager/activities/activities.py to include the Training activity in the main activities orchestrator.
- Updated README.md to document the new 'Training' component.
- Added unit tests for the new activity and repository.
2025-10-09 15:07:27 -03:00
Bruno Domingues
2628dc92d2
SIENTIAPDE-1250: Refactor: Remove 'laborious' directory from test structure and update README.md accordingly.
2025-10-07 10:02:38 -03:00
