Commit Graph

23 Commits

Author SHA1 Message Date
bcd4fab259 refactor: update input dataset and training parameters
- Renamed columns in `input_dataset.csv` from `feature_a`, `feature_b`, and `target` to `Counter`, `Rollout`, and `Square` for better clarity.
- Updated the `run_training_test.py` script to reflect the new column names in the workflow input, ensuring consistency in data processing.
- Added `date_column` parameter to the workflow input for improved data handling.
2026-05-11 12:49:04 -03:00
388d4c95e4 chore: update configuration and imports for improved functionality
- Updated image tag in values.yaml from "1.0.0" to "1.0.2" for the latest version.
- Modified STORE_BASE_URL to include port 3000 for proper service access.
- Changed import from MinioRepository to MinioRepositorySync for synchronization support.
- Updated import from Postgres to PostgresSync to enhance experiment tracking capabilities.
- Renamed TRAIN_TASK_QUEUE from "train_model-single-queue" to "train_model-basic-queue" for clarity.
2026-04-13 14:57:06 -03:00
526edcb50e feat: update training workflow and repository management
- Replaced synchronous MinIO repository calls with asynchronous counterparts in the Training class for improved performance.
- Enhanced logging throughout the training process to provide better insights into model metadata loading, parameter validation, and training execution.
- Updated the train_test_split function to enforce DataFrame input type, ensuring consistency in data handling.
- Removed the deprecated model_repository.py file to streamline the codebase.
- Adjusted cleanup schedule logic to improve error handling and logging during schedule reconciliation.
- Updated tests to reflect changes in the training workflow and repository interactions.
2026-04-09 12:09:52 -03:00
0ae03b246f feat: update environment configuration and remove deprecated model serving
- Modified `.env.example` to set local defaults for PostgreSQL, MLflow, and MinIO configurations.
- Added MongoDB configuration parameters to the environment setup.
- Updated `README.md` to reflect changes in workflow input parameters and task queue naming conventions.
- Removed the `ModelServing` class to streamline the codebase, as it was deemed unnecessary.
- Adjusted `connectors_config.py` to align with new environment variable names and improve clarity.
- Updated tests to reflect changes in configuration handling and removed tests related to the deleted `ModelServing` class.
2026-04-07 16:58:50 -03:00
d2bb588739 Merge branch 'main' into release/SIENTIAPDE-1645 2026-04-06 08:34:15 -03:00
7a0961f29d SIENTIAPDE-1717: Remove MinIO cleanup functionality and associated components. This change streamlines the cleanup workflow to focus solely on local temporary directories, removes the ModelTrainingError exception, and updates related configurations, documentation, and tests. 2026-03-30 14:14:10 -03:00
342a02d6f7 feat: enhance training workflow with model metadata loading and refactor data handling
- Introduced a new activity to load model metadata from the model store.
- Refactored training logic to utilize new model metadata and improved parameter handling.
- Updated the `TrainModelParams` class to include additional fields for model configuration.
- Replaced deprecated utility functions with a custom train-test split implementation.
- Removed unused utility functions and cleaned up the data manager repository.
- Adjusted experiment tracking to include model-specific metadata in notifications.
2026-03-24 14:39:28 -03:00
cf5111e520 feat: integrate PluginStore and MinIO repository into model manager activities
- Added PluginStore integration for model management.
- Replaced StorageRepository with MinIORepository in Activities, Cleanup, and Training classes.
- Updated training logic to handle validation files and improved data management.
- Enhanced configuration for MinIO and PluginStore in connectors.
- Removed deprecated model repository and storage repository files.
- Updated environment variable handling for new configurations.
2026-03-11 17:35:05 -03:00
41c4490cad SIENTIAPDE-1579: Fix SientiaMlException propagation in ModelServing.search_runs_by_name to correctly raise the exception with its message, resolving a TypeError. Enhance training test script to support scenario-specific CSV files for different test cases. Update search_runs_by_name return type hint and apply minor code formatting. 2026-02-19 21:13:41 -03:00
Kou Kinoshita
43cbc31e10 SIENTIAPDE-1579: Lint fixes and formatting 2026-02-18 17:31:58 -03:00
Kou Kinoshita
c6d6f94e05 SIENTIAPDE-1579: Updated tests 2026-02-18 15:05:37 -03:00
Kou Kinoshita
8b7bd81328 SIENTIAPDE-1579: Fixed data preprocessor 2026-02-18 11:14:27 -03:00
Kou Kinoshita
a4d94dd2ff SIENTIAPDE-1579: Added date format and convert methods 2026-02-13 18:41:46 -03:00
Kou Kinoshita
9e3585afce SIENTIAPDE-1579: Added local and validation flags to test script 2026-02-13 17:51:17 -03:00
44dca9b66b SIENTIAPDE-1430: Add option to run all training test scenarios with summary reporting.
- Introduce --all argument to run_training_test.py to execute all available scenarios sequentially.
- Refactor single scenario execution into a run_single_scenario function for better modularity and error handling.
- Implement a print_summary function to report results for multiple scenario runs.
2025-12-18 21:46:51 -03:00
06fd08dc70 SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations. 2025-12-18 17:05:10 -03:00
f3c88885ea SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config. 2025-11-25 00:44:55 -03:00
463906073e SIENTIAPDE-1350: Refactor: Improve logging, configuration, and resource management. Includes .env updates, client initialization logging, and resource closing. 2025-11-24 23:08:29 -03:00
53f49d7a97 SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics. 2025-11-19 18:38:00 -03:00
c7f44a2423 SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations. 2025-11-06 15:34:03 -03:00
Kou-Kinoshita
1f6765dc86 SIENTIAPDE-1241: Linted files 2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64 SIENTIAPDE-1241: Formatted files 2025-10-29 11:55:03 -03:00
5789a13023 SIENTIAPDE-1241: refactor train_model workflow due to I/O errors. 2025-10-22 15:37:56 -03:00