0d50a76f4c
SIENTIAPDE-1430: Add unit tests for Silverman radius and RCE drift metrics.
2025-12-18 18:19:35 -03:00
06fd08dc70
SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations.
2025-12-18 17:05:10 -03:00
6e8f87b2a3
SIENTIAPDE-1430: Implement advanced model training capabilities and enhanced data preprocessing. This includes support for Polynomial Regression with configurable degree and interaction terms, flexible per-variable lag configurations, and new data filtering options by date range and removed intervals. Comprehensive business validations are now enforced for all parameters, and MLflow logging has been extended to capture these detailed configurations. Additionally, Reduced Coulomb Energy (RCE) metrics are added for drift detection, with a new changelog documenting all pipeline parameter updates.
2025-12-17 21:39:43 -03:00
5dcb62ef4b
SIENTIAPDE-1350: Integrate cleanup workflow and update default task queue names.
2025-11-26 22:11:20 -03:00
11aaa0780f
SIENTIAPDE-1350: Improve Docker image build context and fix queue names
...
This commit refactors the .dockerignore file to use a whitelist approach, significantly reducing the Docker image size by excluding unnecessary files. It also fixes the queue names in values.yaml and adds TEMPORAL_USE_TLS to the test environment.
2025-11-26 17:52:44 -03:00
c281969381
SIENTIAPDE-1350: Add test to ensure worker starts despite cleanup schedule creation failure.
2025-11-26 11:30:22 -03:00
bc917e97d8
SIENTIAPDE-1350: Add cleanup schedule tests and configure worker with task queues from env vars
2025-11-25 19:21:55 -03:00
b028dc1b7d
SIENTIAPDE-1350: Implement scheduled cleanup workflow using Temporal schedules. Adds schedule creation to worker startup and configures environment variables.
2025-11-25 18:12:53 -03:00
823df2d65d
SIENTIAPDE-1350: Refactor: Replace AsyncMock with MagicMock in cleanup tests for synchronous calls
2025-11-25 16:54:45 -03:00
33480fa108
SIENTIAPDE-1350: Add unit tests for the CleanupFiles workflow, covering both input bucket and default bucket scenarios.
2025-11-25 16:44:51 -03:00
d0b5b74f88
SIENTIAPDE-1350: Add tests for close method and paginated list_bucket_objects
2025-11-25 16:04:59 -03:00
d2c4803a75
SIENTIAPDE-1350: Add unit tests for the Cleanup activity, ensuring 100% code coverage, and update todo-list.
2025-11-25 16:01:10 -03:00
f3c88885ea
SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config.
2025-11-25 00:44:55 -03:00
2ee66552d2
SIENTIAPDE-1350: Configure separate task queues for train and cleanup workers, update sientia-dataops-library to 1.6.1 and refactor prometheus server startup to use logger.
2025-11-24 11:21:43 -03:00
53f49d7a97
SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics.
2025-11-19 18:38:00 -03:00
7ff54a1112
SIENTIAPDE-1309: Improve Dockerignore, Gitignore, Dockerfile, README, model repository, tests, todo list and values.yaml files.
2025-11-07 15:08:50 -03:00
c7f44a2423
SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations.
2025-11-06 15:34:03 -03:00
f32d647d1c
SIENTIAPDE-1307: Implement metrics for training activities and workflow executions, tracking success/failure status.
2025-11-03 14:41:37 -03:00
d8e8d3c481
SIENTIAPDE-1307: Fix PytestUnraisableExceptionWarning in activity and experiment tracking tests by controlling exception raising in __del__ mocks.
2025-10-31 17:21:27 -03:00
a258036397
SIENTIAPDE-1307: Refactor: Remove redundant patching of SientiaMonitoring and Postgres init in tests, and add shutdown mock to metrics controller.
2025-10-31 17:15:59 -03:00
db581f60f9
SIENTIAPDE-1307: Integrate metrics controller and update sientia-dataops-library to 1.5.1. This change adds metrics collection capabilities to activities and updates the dataops library dependency. (18 files changed, 143 insertions(+), 16 deletions(-))
2025-10-31 17:06:26 -03:00
5b14f1016b
SIENTIAPDE-1241: Suppress PytestUnraisableExceptionWarning and update test data in training repository tests.
2025-10-30 15:22:59 -03:00
f5c1ac3615
SIENTIAPDE-1241: Remove model equation implementation documentation and add test coverage for _generate_artifacts in ModelRepository.
2025-10-30 14:56:10 -03:00
Kou-Kinoshita
a5c0cb6e47
SIENTIAPDE-1241: Fixed formatting and linted code
2025-10-30 10:38:29 -03:00
Kou-Kinoshita
17a551d6f9
SIENTIAPDE-1241: Added remaining training repository tests
2025-10-30 10:29:51 -03:00
Kou-Kinoshita
7e9d6683e4
SIENTIAPDE-1241: Added worker test
2025-10-30 10:21:39 -03:00
Kou-Kinoshita
c63389620a
SIENTIAPDE-1241: Train_model tests
2025-10-30 10:11:07 -03:00
Kou-Kinoshita
928c679223
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-30 09:58:46 -03:00
Kou-Kinoshita
c699a47261
SIENTIAPDE-1241: worker tests
2025-10-30 09:52:34 -03:00
Kou-Kinoshita
94a407f1b8
SIENTIAPDE-1241: Storage repository tests
2025-10-30 09:49:10 -03:00
Kou-Kinoshita
87c64eac2e
SIENTIAPDE-1241: Metrics test
2025-10-30 09:47:23 -03:00
Kou-Kinoshita
1f6765dc86
SIENTIAPDE-1241: Linted files
2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64
SIENTIAPDE-1241: Formatted files
2025-10-29 11:55:03 -03:00
Kou-Kinoshita
e8250d8dd7
SIENTIAPDE-1241: fixed test
2025-10-29 11:49:08 -03:00
Kou-Kinoshita
64590032f3
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-29 11:46:29 -03:00
Kou-Kinoshita
61364671db
SIENTIAPDE-1241: Added remaining trainig_repository tests and some model_repository tests
2025-10-29 11:41:36 -03:00
Kou-Kinoshita
4fdefd8263
SIENTIAPDE-1241: Added training repository tests
2025-10-29 11:32:24 -03:00
Kou-Kinoshita
63a5de5938
SIENTIAPDE-1321: Fixed tests
2025-10-29 10:29:43 -03:00
Kou-Kinoshita
734221e32d
SIENTIAPDE-1321: Updated tests
2025-10-27 19:45:47 -03:00
7826e68954
SIENTIAPDE-1241: Add unit tests for ModelRepository with 100% coverage.
2025-10-23 15:09:40 -03:00
0a33bb86ed
SIENTIAPDE-1241: Add unit tests for TrainModelParams model with 100% coverage.
2025-10-23 11:18:19 -03:00
2262dcf4a0
SIENTIAPDE-1241: Add unit tests for logger_helper module.
2025-10-23 09:15:09 -03:00
a66c5abdfd
SIENTIAPDE-1241: Add unit tests for custom exceptions in tests/utils/test_exceptions.py
2025-10-23 00:30:48 -03:00
3fe139763a
SIENTIAPDE-1241: Add unit tests for the Training activity with 100% coverage.
2025-10-23 00:25:25 -03:00
a6f1563036
SIENTIAPDE-1241: Add unit tests for ExperimentTracking class with 100% coverage.
...
This commit introduces a new test file tests/activities/test_experiment_tracking.py containing comprehensive unit tests for the ExperimentTracking class, achieving 100% test coverage. The tests cover initialization, deletion, execution of updates, and various scenarios for updating experiment run status, including error handling and edge cases.
2025-10-23 00:18:14 -03:00
fc2fe33b59
SIENTIAPDE-1241: Add unit tests for Activities class and enhance validation script with fix and test options
...
This commit introduces a new test suite for the Activities class, achieving 100% coverage. Additionally, the validation script (validate.sh) is enhanced with new options:
- --fix: Automatically fixes code formatting and linting issues using Ruff.
- --only-tests: Runs only the unit tests, skipping other validation steps.
The validation script now also supports skipping tests and provides more informative output.
2025-10-22 23:38:01 -03:00
5789a13023
SIENTIAPDE-1241: refactor train_model workflow due to I/O errors.
2025-10-22 15:37:56 -03:00
3e52253a98
SIENTIAPDE-1255: Add tests to ensure get_required_columns removes columns from self_operations and cross_operations.
2025-10-20 22:27:58 -03:00
614eb0cb7f
SIENTIAPDE-1255: Add unit tests for utils.split_train_test function
2025-10-20 22:22:38 -03:00
089f312575
SIENTIAPDE-1255: Add unit tests for the reports module
2025-10-20 22:20:26 -03:00
