Commit Graph

23 Commits

Author SHA1 Message Date
vitor-aignosi
bcd4fab259 refactor: update input dataset and training parameters
- Renamed columns in `input_dataset.csv` from `feature_a`, `feature_b`, and `target` to `Counter`, `Rollout`, and `Square` for better clarity.
- Updated the `run_training_test.py` script to reflect the new column names in the workflow input, ensuring consistency in data processing.
- Added `date_column` parameter to the workflow input for improved data handling.
2026-05-11 12:49:04 -03:00
vitor-aignosi
388d4c95e4 chore: update configuration and imports for improved functionality
- Updated image tag in values.yaml from "1.0.0" to "1.0.2" for the latest version.
- Modified STORE_BASE_URL to include port 3000 for proper service access.
- Changed import from MinioRepository to MinioRepositorySync for synchronization support.
- Updated import from Postgres to PostgresSync to enhance experiment tracking capabilities.
- Renamed TRAIN_TASK_QUEUE from "train_model-single-queue" to "train_model-basic-queue" for clarity.
2026-04-13 14:57:06 -03:00
vitor-aignosi
526edcb50e feat: update training workflow and repository management
- Replaced synchronous MinIO repository calls with asynchronous counterparts in the Training class for improved performance.
- Enhanced logging throughout the training process to provide better insights into model metadata loading, parameter validation, and training execution.
- Updated the train_test_split function to enforce DataFrame input type, ensuring consistency in data handling.
- Removed the deprecated model_repository.py file to streamline the codebase.
- Adjusted cleanup schedule logic to improve error handling and logging during schedule reconciliation.
- Updated tests to reflect changes in the training workflow and repository interactions.
2026-04-09 12:09:52 -03:00
vitor-aignosi
0ae03b246f feat: update environment configuration and remove deprecated model serving
- Modified `.env.example` to set local defaults for PostgreSQL, MLflow, and MinIO configurations.
- Added MongoDB configuration parameters to the environment setup.
- Updated `README.md` to reflect changes in workflow input parameters and task queue naming conventions.
- Removed the `ModelServing` class to streamline the codebase, as it was deemed unnecessary.
- Adjusted `connectors_config.py` to align with new environment variable names and improve clarity.
- Updated tests to reflect changes in configuration handling and removed tests related to the deleted `ModelServing` class.
2026-04-07 16:58:50 -03:00
vitor-aignosi
d2bb588739 Merge branch 'main' into release/SIENTIAPDE-1645 2026-04-06 08:34:15 -03:00
Bruno Domingues
7a0961f29d SIENTIAPDE-1717: Remove MinIO cleanup functionality and associated components. This change streamlines the cleanup workflow to focus solely on local temporary directories, removes the ModelTrainingError exception, and updates related configurations, documentation, and tests. 2026-03-30 14:14:10 -03:00
vitor-aignosi
342a02d6f7 feat: enhance training workflow with model metadata loading and refactor data handling
- Introduced a new activity to load model metadata from the model store.
- Refactored training logic to utilize new model metadata and improved parameter handling.
- Updated the `TrainModelParams` class to include additional fields for model configuration.
- Replaced deprecated utility functions with a custom train-test split implementation.
- Removed unused utility functions and cleaned up the data manager repository.
- Adjusted experiment tracking to include model-specific metadata in notifications.
2026-03-24 14:39:28 -03:00
vitor-aignosi
cf5111e520 feat: integrate PluginStore and MinIO repository into model manager activities
- Added PluginStore integration for model management.
- Replaced StorageRepository with MinIORepository in Activities, Cleanup, and Training classes.
- Updated training logic to handle validation files and improved data management.
- Enhanced configuration for MinIO and PluginStore in connectors.
- Removed deprecated model repository and storage repository files.
- Updated environment variable handling for new configurations.
2026-03-11 17:35:05 -03:00
Bruno Domingues
41c4490cad SIENTIAPDE-1579: Fix SientiaMlException propagation in ModelServing.search_runs_by_name to correctly raise the exception with its message, resolving a TypeError. Enhance training test script to support scenario-specific CSV files for different test cases. Update search_runs_by_name return type hint and apply minor code formatting. 2026-02-19 21:13:41 -03:00
Kou Kinoshita
43cbc31e10 SIENTIAPDE-1579: Lint fixes and formatting 2026-02-18 17:31:58 -03:00
Kou Kinoshita
c6d6f94e05 SIENTIAPDE-1579: Updated tests 2026-02-18 15:05:37 -03:00
Kou Kinoshita
8b7bd81328 SIENTIAPDE-1579: Fixed data preprocessor 2026-02-18 11:14:27 -03:00
Kou Kinoshita
a4d94dd2ff SIENTIAPDE-1579: Added date format and convert methods 2026-02-13 18:41:46 -03:00
Kou Kinoshita
9e3585afce SIENTIAPDE-1579: Added local and validation flags to test script 2026-02-13 17:51:17 -03:00
Bruno Domingues
44dca9b66b SIENTIAPDE-1430: Add option to run all training test scenarios with summary reporting.
- Introduce --all argument to run_training_test.py to execute all available scenarios sequentially.
- Refactor single scenario execution into a run_single_scenario function for better modularity and error handling.
- Implement a print_summary function to report results for multiple scenario runs.
2025-12-18 21:46:51 -03:00
Bruno Domingues
06fd08dc70 SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations. 2025-12-18 17:05:10 -03:00
Bruno Domingues
f3c88885ea SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config. 2025-11-25 00:44:55 -03:00
Bruno Domingues
463906073e SIENTIAPDE-1350: Refactor: Improve logging, configuration, and resource management. Includes .env updates, client initialization logging, and resource closing. 2025-11-24 23:08:29 -03:00
Bruno Domingues
53f49d7a97 SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics. 2025-11-19 18:38:00 -03:00
Bruno Domingues
c7f44a2423 SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations. 2025-11-06 15:34:03 -03:00
Kou-Kinoshita
1f6765dc86 SIENTIAPDE-1241: Linted files 2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64 SIENTIAPDE-1241: Formatted files 2025-10-29 11:55:03 -03:00
Bruno Domingues
5789a13023 SIENTIAPDE-1241: refactor train_model workflow due to I/O errors. 2025-10-22 15:37:56 -03:00