38fb9a9c53
SIENTIAPDE-1430: Remove verbose options and direct print statements from model, preprocessor, and report utilities.
Bruno Domingues
2025-12-18 21:09:12 -03:00
13941fc3f7
SIENTIAPDE-1430: Add unit tests for _configure_datetime_index (handling various datetime column formats), _init_data_preprocessor (with removed intervals), and _extract_model_equation (supporting polynomial features) in TrainingRepository.
Bruno Domingues
2025-12-18 18:58:12 -03:00
71b654f24c
SIENTIAPDE-1430: Add test case for ValueError when y_train_pred is None during artifact data initialization.
Bruno Domingues
2025-12-18 18:48:42 -03:00
573834fb99
SIENTIAPDE-1430: Add comprehensive tests for TrainModelParams.validate_business_rules to improve coverage.
Bruno Domingues
2025-12-18 18:46:01 -03:00
70067fd992
SIENTIAPDE-1430: Improve test coverage for LinearRegressionModel and DataPreprocessor. Added extensive new tests covering various methods, edge cases, and scenarios to enhance overall code coverage.
Bruno Domingues
2025-12-18 18:41:03 -03:00
0d50a76f4c
SIENTIAPDE-1430: Add unit tests for Silverman radius and RCE drift metrics.
Bruno Domingues
2025-12-18 18:19:35 -03:00
06fd08dc70
SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations.
Bruno Domingues
2025-12-18 17:05:10 -03:00
6e8f87b2a3
SIENTIAPDE-1430: Implement advanced model training capabilities and enhanced data preprocessing. This includes support for Polynomial Regression with configurable degree and interaction terms, flexible per-variable lag configurations, and new data filtering options by date range and removed intervals. Comprehensive business validations are now enforced for all parameters, and MLflow logging has been extended to capture these detailed configurations. Additionally, Reduced Coulomb Energy (RCE) metrics are added for drift detection, with a new changelog documenting all pipeline parameter updates.
Bruno Domingues
2025-12-17 21:39:43 -03:00
4d6674758c
SIENTIAPDE-1430: Remove todo-list.txt
Bruno Domingues
2025-12-17 15:45:00 -03:00
cf73def257
ci(workflows): Trigger PR workflows on release and feature branches
Bruno Domingues
2025-12-09 11:54:06 -03:00
923f2cb675
chore: bump version to 1.0.3 [skip ci]
github-actions[bot]
2025-12-05 14:45:10 +00:00
c39a1428c7
Merge pull request #21 from Aignosi/fix/SIENTIAPDE-1352
Bruno Domingues
2025-12-05 11:41:02 -03:00
9d118fa6fa
SIENTIAPDE-1352: Update workflow references to use main branch and correct quality gate workflow.
Bruno Domingues
2025-12-05 11:32:59 -03:00
530e8cbb08
chore: bump version to 1.0.2 [skip ci]
github-actions[bot]
2025-12-04 22:39:52 +00:00
6d19acd5a2
Merge pull request #20 from Aignosi/fix/SIENTIAPDE-1352
Bruno Domingues
2025-12-04 19:36:06 -03:00
32aa66e5d8
SIENTIAPDE-1352: Update README with improved instructions, project structure details, and environment variable descriptions.
Bruno Domingues
2025-12-04 19:31:35 -03:00
01b8c06d75
Merge branch 'main' into fix/SIENTIAPDE-1352
Bruno Domingues
2025-12-04 19:26:08 -03:00
f05b4d3253
Merge pull request #19 from Aignosi/fix/SIENTIAPDE-1352
Bruno Domingues
2025-12-04 18:12:19 -03:00
8cb39605d2
SIENTIAPDE-1352: Configure helm deployment to update version in pyproject and values files.
Bruno Domingues
2025-12-04 18:08:44 -03:00
0a3135a8a5
Merge pull request #18 from Aignosi/release/SIENTIAPDE-1352
Bruno Domingues
2025-12-04 16:44:14 -03:00
a974926f3d
SIENTIAPDE-1352: Simplify deploy workflow by inheriting secrets.
Bruno Domingues
2025-12-04 16:41:03 -03:00
463d086c5e
Merge branch 'main' into release/SIENTIAPDE-1352
Bruno Domingues
2025-12-04 16:36:25 -03:00
e337373f59
Merge pull request #17 from Aignosi/release/SIENTIAPDE-1352
Bruno Domingues
2025-12-04 13:54:10 -03:00
5fb8aead22
SIENTIAPDE-1352: Update deploy workflow to use reusable workflow and add permissions, update README with CI/CD details.
Bruno Domingues
2025-12-04 12:54:03 -03:00
f0aaa6e2c1
feat(deploy): add write-all permissions to deploy workflow
Bruno Domingues
2025-12-04 12:48:11 -03:00
9871867b76
Merge pull request #16 from Aignosi/release/SIENTIAPDE-1352
Bruno Domingues
2025-12-04 12:41:54 -03:00
cdaafd12cc
SIENTIAPDE-1352: Add deploy workflow and update README with local repository execution instructions.
Bruno Domingues
2025-12-04 12:09:50 -03:00
46df241f7d
SIENTIAPDE-1352: Add documentation for local GitHub Actions testing with act.
Bruno Domingues
2025-12-02 17:28:04 -03:00
2b729c65e6
SIENTIAPDE-1352: Refactor quality gate workflow to use reusable dataops-module-quality-gate workflow and update .gitignore. ( -230, +19 lines)
Bruno Domingues
2025-12-02 17:20:31 -03:00
3a61633ec6
Merge pull request #15 from Aignosi/feature/SIENTIAPDE-1350
Bruno Domingues
2025-11-27 12:25:58 -03:00
9d3ecc1c2f
SIENTIAPDE-1350: Simplify exception handling for file and directory deletion in cleanup activity
Bruno Domingues
2025-11-27 11:00:16 -03:00
486e1756c9
SIENTIAPDE-1350: Implement automated file cleanup workflow and update documentation. Added a new workflow for automated cleanup of stale files from MinIO and local filesystem, including scheduled execution and configurable retention. Updated the README with detailed information about the new workflow, its configuration, and related metrics. (118 additions, 5 deletions)
Bruno Domingues
2025-11-27 10:22:38 -03:00
5dcb62ef4b
SIENTIAPDE-1350: Integrate cleanup workflow and update default task queue names.
Bruno Domingues
2025-11-26 22:11:20 -03:00
11aaa0780f
SIENTIAPDE-1350: Improve Docker image build context and fix queue names
Bruno Domingues
2025-11-26 17:52:44 -03:00
c281969381
SIENTIAPDE-1350: Add test to ensure worker starts despite cleanup schedule creation failure.
Bruno Domingues
2025-11-26 11:30:22 -03:00
bc917e97d8
SIENTIAPDE-1350: Add cleanup schedule tests and configure worker with task queues from env vars
Bruno Domingues
2025-11-25 19:21:55 -03:00
b028dc1b7d
SIENTIAPDE-1350: Implement scheduled cleanup workflow using Temporal schedules. Adds schedule creation to worker startup and configures environment variables.
Bruno Domingues
2025-11-25 18:12:53 -03:00
9ecac9c8fd
SIENTIAPDE-1350: Update values.yaml with new environment variables and remove outdated task from todo-list.txt
Bruno Domingues
2025-11-25 17:03:06 -03:00
823df2d65d
SIENTIAPDE-1350: Refactor: Replace AsyncMock with MagicMock in cleanup tests for synchronous calls
Bruno Domingues
2025-11-25 16:54:45 -03:00
33480fa108
SIENTIAPDE-1350: Add unit tests for the CleanupFiles workflow, covering both input bucket and default bucket scenarios.
Bruno Domingues
2025-11-25 16:44:51 -03:00
d0b5b74f88
SIENTIAPDE-1350: Add tests for close method and paginated list_bucket_objects
Bruno Domingues
2025-11-25 16:04:59 -03:00
d2c4803a75
SIENTIAPDE-1350: Add unit tests for the Cleanup activity, ensuring 100% code coverage, and update todo-list.
Bruno Domingues
2025-11-25 16:01:10 -03:00
f3c88885ea
SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config.
Bruno Domingues
2025-11-25 00:44:55 -03:00
463906073e
SIENTIAPDE-1350: Refactor: Improve logging, configuration, and resource management. Includes .env updates, client initialization logging, and resource closing.
Bruno Domingues
2025-11-24 23:08:29 -03:00
2ee66552d2
SIENTIAPDE-1350: Configure separate task queues for train and cleanup workers, update sientia-dataops-library to 1.6.1 and refactor prometheus server startup to use logger.
Bruno Domingues
2025-11-24 11:21:43 -03:00
53f49d7a97
SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics.
Bruno Domingues
2025-11-19 18:38:00 -03:00
cea3ef60cb
Merge pull request #14 from Aignosi/feature/SIENTIAPDE-1349
Bruno Domingues
2025-11-11 11:46:49 -03:00
731a23a92e
SIENTIAPDE-1349: Add Grafana dashboard configuration to values.yaml
Bruno Domingues
2025-11-11 11:37:28 -03:00
e51daca073
Merge pull request #13 from Aignosi/feature/SIENTIAPDE-1309
Bruno Domingues
2025-11-07 16:32:36 -03:00
f232406bd3
SIENTIAPDE-1309: Improve .gitignore to preserve .gitkeep in model_manager reports temp directory.
Bruno Domingues
2025-11-07 15:25:10 -03:00
18103f2436
SIENTIAPDE-1309: Refine .gitignore to ignore test run reports within the model_manager/reports/temp directory, while preserving the temp directory itself.
Bruno Domingues
2025-11-07 15:19:21 -03:00
7ff54a1112
SIENTIAPDE-1309: Improve Dockerignore, Gitignore, Dockerfile, README, model repository, tests, todo list and values.yaml files.
Bruno Domingues
2025-11-07 15:08:50 -03:00
258bad84c6
SIENTIAPDE-1309: Add ignore rule for test run reports in model_manager/reports
Bruno Domingues
2025-11-06 15:59:44 -03:00
c7f44a2423
SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations.
Bruno Domingues
2025-11-06 15:34:03 -03:00
a625ef6c19
SIENTIAPDE-1309: Add Helm chart documentation to README and update image tag in values.yaml
Bruno Domingues
2025-11-05 21:26:58 -03:00
005c33151c
Merge pull request #12 from Aignosi/feature/SIENTIAPDE-1308
Bruno Domingues
2025-11-04 15:42:06 -03:00
d9d3df5723
SIENTIAPDE-1308: Initial Docker setup, improved .dockerignore, updated README, and minor fixes.
Bruno Domingues
2025-11-04 14:07:37 -03:00
9f56a128e6
SIENTIAPDE-1308: Update documentation, remove install_dependencies.sh script from README, correct Training class initialization, and update sientia-dataops-library version in requirements.txt.
Bruno Domingues
2025-11-03 17:35:33 -03:00
6ff2d9857d
feat: update todo list
Bruno Domingues
2025-11-03 15:36:35 -03:00
12f8b21625
Merge pull request #11 from Aignosi/feature/SIENTIAPDE-1307
Bruno Domingues
2025-11-03 15:26:58 -03:00
f32d647d1c
SIENTIAPDE-1307: Implement metrics for training activities and workflow executions, tracking success/failure status.
Bruno Domingues
2025-11-03 14:41:37 -03:00
d8e8d3c481
SIENTIAPDE-1307: Fix PytestUnraisableExceptionWarning in activity and experiment tracking tests by controlling exception raising in __del__ mocks.
Bruno Domingues
2025-10-31 17:21:27 -03:00
a258036397
SIENTIAPDE-1307: Refactor: Remove redundant patching of SientiaMonitoring and Postgres init in tests, and add shutdown mock to metrics controller.
Bruno Domingues
2025-10-31 17:15:59 -03:00
db581f60f9
SIENTIAPDE-1307: Integrate metrics controller and update sientia-dataops-library to 1.5.1. This change adds metrics collection capabilities to activities and updates the dataops library dependency. (18 files changed, 143 insertions(+), 16 deletions(-))
Bruno Domingues
2025-10-31 17:06:26 -03:00
326f9f9dc6
Merge pull request #10 from Aignosi/feature/SIENTIAPDE-1241
Bruno Domingues
2025-10-30 17:04:51 -03:00
5b14f1016b
SIENTIAPDE-1241: Suppress PytestUnraisableExceptionWarning and update test data in training repository tests.
Bruno Domingues
2025-10-30 15:22:59 -03:00
bb25a19d8c
SIENTIAPDE-1241: Simplify worker shutdown logic by removing conditional checks.
Bruno Domingues
2025-10-30 15:14:43 -03:00
f5c1ac3615
SIENTIAPDE-1241: Remove model equation implementation documentation and add test coverage for _generate_artifacts in ModelRepository.
Bruno Domingues
2025-10-30 14:56:10 -03:00
a80d65d7ba
SIENTIAPDE-1321: Added equation related methods
Kou-Kinoshita
2025-10-24 08:03:27 -03:00
7826e68954
SIENTIAPDE-1241: Add unit tests for ModelRepository with 100% coverage.
Bruno Domingues
2025-10-23 15:09:40 -03:00
0a33bb86ed
SIENTIAPDE-1241: Add unit tests for TrainModelParams model with 100% coverage.
Bruno Domingues
2025-10-23 11:18:19 -03:00
2262dcf4a0
SIENTIAPDE-1241: Add unit tests for logger_helper module.
Bruno Domingues
2025-10-23 09:15:09 -03:00
a66c5abdfd
SIENTIAPDE-1241: Add unit tests for custom exceptions in tests/utils/test_exceptions.py
Bruno Domingues
2025-10-23 00:30:48 -03:00
3fe139763a
SIENTIAPDE-1241: Add unit tests for the Training activity with 100% coverage.
Bruno Domingues
2025-10-23 00:25:25 -03:00
a6f1563036
SIENTIAPDE-1241: Add unit tests for ExperimentTracking class with 100% coverage.
Bruno Domingues
2025-10-23 00:18:14 -03:00
fc2fe33b59
SIENTIAPDE-1241: Add unit tests for Activities class and enhance validation script with fix and test options
Bruno Domingues
2025-10-22 23:38:01 -03:00
f07bc8ff30
SIENTIAPDE-1241: Refactor: Remove redundant logging and fix duplicate logs
Bruno Domingues
2025-10-22 22:39:26 -03:00
ac97092ebf
SIENTIAPDE-1241: Remove unused metadata fields and update todo list.
Bruno Domingues
2025-10-22 21:51:29 -03:00
94697215aa
SIENTIAPDE-1241: Refactor: Improve documentation, exception handling, and configuration in model manager. This commit enhances clarity and robustness by adding detailed docstrings to methods, standardizing exception handling with custom types, and simplifying MLflow configuration.
Bruno Domingues
2025-10-22 21:48:12 -03:00
Bruno Domingues