Bruno Domingues
06fd08dc70
SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations.
2025-12-18 17:05:10 -03:00
Bruno Domingues
6e8f87b2a3
SIENTIAPDE-1430: Implement advanced model training capabilities and enhanced data preprocessing. This includes support for Polynomial Regression with configurable degree and interaction terms, flexible per-variable lag configurations, and new data filtering options by date range and removed intervals. Comprehensive business validations are now enforced for all parameters, and MLflow logging has been extended to capture these detailed configurations. Additionally, Reduced Coulomb Energy (RCE) metrics are added for drift detection, with a new changelog documenting all pipeline parameter updates.
2025-12-17 21:39:43 -03:00
Bruno Domingues
5dcb62ef4b
SIENTIAPDE-1350: Integrate cleanup workflow and update default task queue names.
2025-11-26 22:11:20 -03:00
Bruno Domingues
11aaa0780f
SIENTIAPDE-1350: Improve Docker image build context and fix queue names
...
This commit refactors the .dockerignore file to use a whitelist approach, significantly reducing the Docker image size by excluding unnecessary files. It also fixes the queue names in values.yaml and adds TEMPORAL_USE_TLS to the test environment.
2025-11-26 17:52:44 -03:00
Bruno Domingues
c281969381
SIENTIAPDE-1350: Add test to ensure worker starts despite cleanup schedule creation failure.
2025-11-26 11:30:22 -03:00
Bruno Domingues
bc917e97d8
SIENTIAPDE-1350: Add cleanup schedule tests and configure worker with task queues from env vars
2025-11-25 19:21:55 -03:00
Bruno Domingues
b028dc1b7d
SIENTIAPDE-1350: Implement scheduled cleanup workflow using Temporal schedules. Adds schedule creation to worker startup and configures environment variables.
2025-11-25 18:12:53 -03:00
Bruno Domingues
823df2d65d
SIENTIAPDE-1350: Refactor: Replace AsyncMock with MagicMock in cleanup tests for synchronous calls
2025-11-25 16:54:45 -03:00
Bruno Domingues
33480fa108
SIENTIAPDE-1350: Add unit tests for the CleanupFiles workflow, covering both input bucket and default bucket scenarios.
2025-11-25 16:44:51 -03:00
Bruno Domingues
d0b5b74f88
SIENTIAPDE-1350: Add tests for close method and paginated list_bucket_objects
2025-11-25 16:04:59 -03:00
Bruno Domingues
d2c4803a75
SIENTIAPDE-1350: Add unit tests for the Cleanup activity, ensuring 100% code coverage, and update todo-list.
2025-11-25 16:01:10 -03:00
Bruno Domingues
f3c88885ea
SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config.
2025-11-25 00:44:55 -03:00
Bruno Domingues
2ee66552d2
SIENTIAPDE-1350: Configure separate task queues for train and cleanup workers, update sientia-dataops-library to 1.6.1 and refactor prometheus server startup to use logger.
2025-11-24 11:21:43 -03:00
Bruno Domingues
53f49d7a97
SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics.
2025-11-19 18:38:00 -03:00
Bruno Domingues
7ff54a1112
SIENTIAPDE-1309: Improve Dockerignore, Gitignore, Dockerfile, README, model repository, tests, todo list and values.yaml files.
2025-11-07 15:08:50 -03:00
Bruno Domingues
c7f44a2423
SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations.
2025-11-06 15:34:03 -03:00
Bruno Domingues
f32d647d1c
SIENTIAPDE-1307: Implement metrics for training activities and workflow executions, tracking success/failure status.
2025-11-03 14:41:37 -03:00
Bruno Domingues
d8e8d3c481
SIENTIAPDE-1307: Fix PytestUnraisableExceptionWarning in activity and experiment tracking tests by controlling exception raising in __del__ mocks.
2025-10-31 17:21:27 -03:00
Bruno Domingues
a258036397
SIENTIAPDE-1307: Refactor: Remove redundant patching of SientiaMonitoring and Postgres init in tests, and add shutdown mock to metrics controller.
2025-10-31 17:15:59 -03:00
Bruno Domingues
db581f60f9
SIENTIAPDE-1307: Integrate metrics controller and update sientia-dataops-library to 1.5.1. This change adds metrics collection capabilities to activities and updates the dataops library dependency. (18 files changed, 143 insertions(+), 16 deletions(-))
2025-10-31 17:06:26 -03:00
Bruno Domingues
5b14f1016b
SIENTIAPDE-1241: Suppress PytestUnraisableExceptionWarning and update test data in training repository tests.
2025-10-30 15:22:59 -03:00
Bruno Domingues
f5c1ac3615
SIENTIAPDE-1241: Remove model equation implementation documentation and add test coverage for _generate_artifacts in ModelRepository.
2025-10-30 14:56:10 -03:00
Kou-Kinoshita
a5c0cb6e47
SIENTIAPDE-1241: Fixed formatting and linted code
2025-10-30 10:38:29 -03:00
Kou-Kinoshita
17a551d6f9
SIENTIAPDE-1241: Added remaining training repository tests
2025-10-30 10:29:51 -03:00
Kou-Kinoshita
7e9d6683e4
SIENTIAPDE-1241: Added worker test
2025-10-30 10:21:39 -03:00
Kou-Kinoshita
c63389620a
SIENTIAPDE-1241: Train_model tests
2025-10-30 10:11:07 -03:00
Kou-Kinoshita
928c679223
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-30 09:58:46 -03:00
Kou-Kinoshita
c699a47261
SIENTIAPDE-1241: worker tests
2025-10-30 09:52:34 -03:00
Kou-Kinoshita
94a407f1b8
SIENTIAPDE-1241: Storage repository tests
2025-10-30 09:49:10 -03:00
Kou-Kinoshita
87c64eac2e
SIENTIAPDE-1241: Metrics test
2025-10-30 09:47:23 -03:00
Kou-Kinoshita
1f6765dc86
SIENTIAPDE-1241: Linted files
2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64
SIENTIAPDE-1241: Formatted files
2025-10-29 11:55:03 -03:00
Kou-Kinoshita
e8250d8dd7
SIENTIAPDE-1241: fixed test
2025-10-29 11:49:08 -03:00
Kou-Kinoshita
64590032f3
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-29 11:46:29 -03:00
Kou-Kinoshita
61364671db
SIENTIAPDE-1241: Added remaining trainig_repository tests and some model_repository tests
2025-10-29 11:41:36 -03:00
Kou-Kinoshita
4fdefd8263
SIENTIAPDE-1241: Added training repository tests
2025-10-29 11:32:24 -03:00
Kou-Kinoshita
63a5de5938
SIENTIAPDE-1321: Fixed tests
2025-10-29 10:29:43 -03:00
Kou-Kinoshita
734221e32d
SIENTIAPDE-1321: Updated tests
2025-10-27 19:45:47 -03:00
Bruno Domingues
7826e68954
SIENTIAPDE-1241: Add unit tests for ModelRepository with 100% coverage.
2025-10-23 15:09:40 -03:00
Bruno Domingues
0a33bb86ed
SIENTIAPDE-1241: Add unit tests for TrainModelParams model with 100% coverage.
2025-10-23 11:18:19 -03:00
Bruno Domingues
2262dcf4a0
SIENTIAPDE-1241: Add unit tests for logger_helper module.
2025-10-23 09:15:09 -03:00
Bruno Domingues
a66c5abdfd
SIENTIAPDE-1241: Add unit tests for custom exceptions in tests/utils/test_exceptions.py
2025-10-23 00:30:48 -03:00
Bruno Domingues
3fe139763a
SIENTIAPDE-1241: Add unit tests for the Training activity with 100% coverage.
2025-10-23 00:25:25 -03:00
Bruno Domingues
a6f1563036
SIENTIAPDE-1241: Add unit tests for ExperimentTracking class with 100% coverage.
...
This commit introduces a new test file tests/activities/test_experiment_tracking.py containing comprehensive unit tests for the ExperimentTracking class, achieving 100% test coverage. The tests cover initialization, deletion, execution of updates, and various scenarios for updating experiment run status, including error handling and edge cases.
2025-10-23 00:18:14 -03:00
Bruno Domingues
fc2fe33b59
SIENTIAPDE-1241: Add unit tests for Activities class and enhance validation script with fix and test options
...
This commit introduces a new test suite for the Activities class, achieving 100% coverage. Additionally, the validation script (validate.sh) is enhanced with new options:
- --fix: Automatically fixes code formatting and linting issues using Ruff.
- --only-tests: Runs only the unit tests, skipping other validation steps.
The validation script now also supports skipping tests and provides more informative output.
2025-10-22 23:38:01 -03:00
Bruno Domingues
5789a13023
SIENTIAPDE-1241: refactor train_model workflow due to I/O errors.
2025-10-22 15:37:56 -03:00
Bruno Domingues
3e52253a98
SIENTIAPDE-1255: Add tests to ensure get_required_columns removes columns from self_operations and cross_operations.
2025-10-20 22:27:58 -03:00
Bruno Domingues
614eb0cb7f
SIENTIAPDE-1255: Add unit tests for utils.split_train_test function
2025-10-20 22:22:38 -03:00
Bruno Domingues
089f312575
SIENTIAPDE-1255: Add unit tests for the reports module
2025-10-20 22:20:26 -03:00
Bruno Domingues
0de2ece063
SIENTIAPDE-1255: Add unit tests for sientia models (DataPreprocessor and LinearRegressionModel)
2025-10-20 20:59:55 -03:00
