vitor-aignosi
342a02d6f7
feat: enhance training workflow with model metadata loading and refactor data handling
...
- Introduced a new activity to load model metadata from the model store.
- Refactored training logic to utilize new model metadata and improved parameter handling.
- Updated the `TrainModelParams` class to include additional fields for model configuration.
- Replaced deprecated utility functions with a custom train-test split implementation.
- Removed unused utility functions and cleaned up the data manager repository.
- Adjusted experiment tracking to include model-specific metadata in notifications.
2026-03-24 14:39:28 -03:00
vitor-aignosi
cf5111e520
feat: integrate PluginStore and MinIO repository into model manager activities
...
- Added PluginStore integration for model management.
- Replaced StorageRepository with MinIORepository in Activities, Cleanup, and Training classes.
- Updated training logic to handle validation files and improved data management.
- Enhanced configuration for MinIO and PluginStore in connectors.
- Removed deprecated model repository and storage repository files.
- Updated environment variable handling for new configurations.
2026-03-11 17:35:05 -03:00
github-actions[bot]
9d71c0cf80
chore: bump version to 1.1.2 [skip ci]
2026-02-20 00:20:15 +00:00
Bruno Domingues
0a0adac86c
Merge pull request #24 from Aignosi/fix/SIENTIAPDE-1579
...
Upgrade MLflow to 2.18.0 and constrain setuptools
2026-02-19 21:19:53 -03:00
Bruno Domingues
41c4490cad
SIENTIAPDE-1579: Fix SientiaMlException propagation in ModelServing.search_runs_by_name to correctly raise the exception with its message, resolving a TypeError. Enhance training test script to support scenario-specific CSV files for different test cases. Update search_runs_by_name return type hint and apply minor code formatting.
2026-02-19 21:13:41 -03:00
Bruno Domingues
f952647cf6
SIENTIAPDE-1579: Update MLflow to 2.18.0 and add setuptools version constraint.
2026-02-19 17:56:08 -03:00
github-actions[bot]
b0629cb34a
chore: bump version to 1.1.1 [skip ci]
2026-02-19 12:34:59 +00:00
Lucas Kou Kinoshita
8609245e77
Merge pull request #23 from Aignosi/fix/SIENTIAPDE-1579
...
Fix/SIENTIAPDE-1579: Corrigir bug durante o treinamento de certos arquivos no Soft Sensor
2026-02-19 09:34:34 -03:00
Kou Kinoshita
14f3fbadbe
SIENTIAPDE-1579: Switched from replace logic to validation + mapping
2026-02-18 18:57:41 -03:00
Kou Kinoshita
e7e851dbb5
SIENTIAPDE-1579: Lint fix
2026-02-18 18:20:43 -03:00
Kou Kinoshita
5dab1ee576
SIENTIAPDE-1579?: removed duplicated method
2026-02-18 18:18:39 -03:00
Kou Kinoshita
a7922ac3ce
SIENTIAPDE-1579: Lint fix
2026-02-18 17:50:58 -03:00
Kou Kinoshita
665870320f
SIENTIAPDE-1579: Updated tests and refactored apply filters method
2026-02-18 17:44:46 -03:00
Kou Kinoshita
43cbc31e10
SIENTIAPDE-1579: Lint fixes and formatting
2026-02-18 17:31:58 -03:00
Kou Kinoshita
c6d6f94e05
SIENTIAPDE-1579: Updated tests
2026-02-18 15:05:37 -03:00
Kou Kinoshita
b6eb346a0b
SIENTIAPDE-1579: Added test case scenario to test filter, also updated old scenarios
2026-02-18 14:24:17 -03:00
Kou Kinoshita
54a47a1d96
SIENTIAPDE-1579: Added extra test files and support filters implementation
2026-02-18 14:16:21 -03:00
Kou Kinoshita
8b7bd81328
SIENTIAPDE-1579: Fixed data preprocessor
2026-02-18 11:14:27 -03:00
Kou Kinoshita
a4d94dd2ff
SIENTIAPDE-1579: Added date format and convert methods
2026-02-13 18:41:46 -03:00
Kou Kinoshita
9e3585afce
SIENTIAPDE-1579: Added local and validation flags to test script
2026-02-13 17:51:17 -03:00
github-actions[bot]
f0cdace820
chore: bump version to 1.1.0 [skip ci]
2025-12-19 19:43:10 +00:00
Bruno Domingues
68ed97583a
Merge pull request #22 from Aignosi/feature/SIENTIAPDE-1430
...
Implement Advanced Training Features, Enhance Data Preprocessing, and Bolster Testing & Documentation
2025-12-19 16:42:51 -03:00
Bruno Domingues
df6bf1daba
SIENTIAPDE-1430: Refactor static threshold calculation logic and enhance test coverage
2025-12-19 16:19:10 -03:00
Bruno Domingues
06571011f2
SIENTIAPDE-1430: Introduce static_threshold parameter for static window removal.
...
This parameter allows customizing the threshold (1-1000) used when rem_static_win is enabled, defaulting to 1 if null.
Updates include parameter definition, business rule validation, repository logic for passing the threshold, documentation in README.md and PIPELINE_PARAMS_CHANGELOG.md, and new unit and integration tests.
2025-12-19 15:44:18 -03:00
Bruno Domingues
7cbb7022e5
SIENTIAPDE-1430: Refactor DataPreprocessor date filtering, make rce_train radius optional, and introduce constants for model names.
2025-12-19 12:28:07 -03:00
Bruno Domingues
9a77257920
SIENTIAPDE-1430: Update README to document new features, expanded validation, and enhanced testing.
...
* Documents support for Polynomial Regression, per-variable lag configuration, date range filtering, and NaN treatment options.
* Includes details on RCE drift metrics and comprehensive business validation rules for training parameters.
* Describes new JSON-based integration test scenarios with batch execution and updated 100% code coverage.
* Refines architecture overview and example workflow parameters.
2025-12-19 11:54:08 -03:00
Bruno Domingues
44dca9b66b
SIENTIAPDE-1430: Add option to run all training test scenarios with summary reporting.
...
- Introduce --all argument to run_training_test.py to execute all available scenarios sequentially.
- Refactor single scenario execution into a run_single_scenario function for better modularity and error handling.
- Implement a print_summary function to report results for multiple scenario runs.
2025-12-18 21:46:51 -03:00
Bruno Domingues
38fb9a9c53
SIENTIAPDE-1430: Remove verbose options and direct print statements from model, preprocessor, and report utilities.
2025-12-18 21:09:12 -03:00
Bruno Domingues
13941fc3f7
SIENTIAPDE-1430: Add unit tests for _configure_datetime_index (handling various datetime column formats), _init_data_preprocessor (with removed intervals), and _extract_model_equation (supporting polynomial features) in TrainingRepository.
2025-12-18 18:58:12 -03:00
Bruno Domingues
71b654f24c
SIENTIAPDE-1430: Add test case for ValueError when y_train_pred is None during artifact data initialization.
2025-12-18 18:48:42 -03:00
Bruno Domingues
573834fb99
SIENTIAPDE-1430: Add comprehensive tests for TrainModelParams.validate_business_rules to improve coverage.
2025-12-18 18:46:01 -03:00
Bruno Domingues
70067fd992
SIENTIAPDE-1430: Improve test coverage for LinearRegressionModel and DataPreprocessor. Added extensive new tests covering various methods, edge cases, and scenarios to enhance overall code coverage.
2025-12-18 18:41:03 -03:00
Bruno Domingues
0d50a76f4c
SIENTIAPDE-1430: Add unit tests for Silverman radius and RCE drift metrics.
2025-12-18 18:19:35 -03:00
Bruno Domingues
06fd08dc70
SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations.
2025-12-18 17:05:10 -03:00
Bruno Domingues
6e8f87b2a3
SIENTIAPDE-1430: Implement advanced model training capabilities and enhanced data preprocessing. This includes support for Polynomial Regression with configurable degree and interaction terms, flexible per-variable lag configurations, and new data filtering options by date range and removed intervals. Comprehensive business validations are now enforced for all parameters, and MLflow logging has been extended to capture these detailed configurations. Additionally, Reduced Coulomb Energy (RCE) metrics are added for drift detection, with a new changelog documenting all pipeline parameter updates.
2025-12-17 21:39:43 -03:00
Bruno Domingues
4d6674758c
SIENTIAPDE-1430: Remove todo-list.txt
2025-12-17 15:45:00 -03:00
Bruno Domingues
cf73def257
ci(workflows): Trigger PR workflows on release and feature branches
2025-12-09 11:54:06 -03:00
github-actions[bot]
923f2cb675
chore: bump version to 1.0.3 [skip ci]
2025-12-05 14:45:10 +00:00
Bruno Domingues
c39a1428c7
Merge pull request #21 from Aignosi/fix/SIENTIAPDE-1352
...
SIENTIAPDE-1352: Update workflow references to main and fix quality gate
2025-12-05 11:41:02 -03:00
Bruno Domingues
9d118fa6fa
SIENTIAPDE-1352: Update workflow references to use main branch and correct quality gate workflow.
2025-12-05 11:32:59 -03:00
github-actions[bot]
530e8cbb08
chore: bump version to 1.0.2 [skip ci]
2025-12-04 22:39:52 +00:00
Bruno Domingues
6d19acd5a2
Merge pull request #20 from Aignosi/fix/SIENTIAPDE-1352
...
Update README with comprehensive project documentation
2025-12-04 19:36:06 -03:00
Bruno Domingues
32aa66e5d8
SIENTIAPDE-1352: Update README with improved instructions, project structure details, and environment variable descriptions.
2025-12-04 19:31:35 -03:00
Bruno Domingues
01b8c06d75
Merge branch 'main' into fix/SIENTIAPDE-1352
2025-12-04 19:26:08 -03:00
Bruno Domingues
f05b4d3253
Merge pull request #19 from Aignosi/fix/SIENTIAPDE-1352
...
SIENTIAPDE-1352: Configure Helm deployment to update version in pyproject and values files
2025-12-04 18:12:19 -03:00
Bruno Domingues
8cb39605d2
SIENTIAPDE-1352: Configure helm deployment to update version in pyproject and values files.
2025-12-04 18:08:44 -03:00
Bruno Domingues
0a3135a8a5
Merge pull request #18 from Aignosi/release/SIENTIAPDE-1352
...
Simplify deploy workflow by inheriting secrets
2025-12-04 16:44:14 -03:00
Bruno Domingues
a974926f3d
SIENTIAPDE-1352: Simplify deploy workflow by inheriting secrets.
2025-12-04 16:41:03 -03:00
Bruno Domingues
463d086c5e
Merge branch 'main' into release/SIENTIAPDE-1352
2025-12-04 16:36:25 -03:00
Bruno Domingues
e337373f59
Merge pull request #17 from Aignosi/release/SIENTIAPDE-1352
...
SIENTIAPDE-1352: Update deploy workflow to use reusable workflow and add permissions, update README with CI/CD details.
2025-12-04 13:54:10 -03:00
