Commit Graph

21 Commits

Author SHA1 Message Date
vitor-aignosi
76f926a8ab SIENTIAPDE-1645: Add comprehensive model training observability metrics and Grafana dashboard. 2026-06-17 10:34:56 -03:00
vitor-aignosi
445fe643fe chore: update .gitignore and requirements for development
- Added new entries to .gitignore to exclude temporary files and training input datasets, ensuring a cleaner repository.
- Included type stubs for psycopg2 in requirements-dev.txt to enhance type checking support for database interactions.
- Refactored the run_training_test.py script to implement a structured approach for loading and validating training input JSON files, improving the robustness of the training workflow.
2026-05-26 09:38:24 -03:00
vitor-aignosi
bcd4fab259 refactor: update input dataset and training parameters
- Renamed columns in `input_dataset.csv` from `feature_a`, `feature_b`, and `target` to `Counter`, `Rollout`, and `Square` for better clarity.
- Updated the `run_training_test.py` script to reflect the new column names in the workflow input, ensuring consistency in data processing.
- Added `date_column` parameter to the workflow input for improved data handling.
2026-05-11 12:49:04 -03:00
vitor-aignosi
388d4c95e4 chore: update configuration and imports for improved functionality
- Updated image tag in values.yaml from "1.0.0" to "1.0.2" for the latest version.
- Modified STORE_BASE_URL to include port 3000 for proper service access.
- Changed import from MinioRepository to MinioRepositorySync for synchronization support.
- Updated import from Postgres to PostgresSync to enhance experiment tracking capabilities.
- Renamed TRAIN_TASK_QUEUE from "train_model-single-queue" to "train_model-basic-queue" for clarity.
2026-04-13 14:57:06 -03:00
vitor-aignosi
526edcb50e feat: update training workflow and repository management
- Replaced synchronous MinIO repository calls with asynchronous counterparts in the Training class for improved performance.
- Enhanced logging throughout the training process to provide better insights into model metadata loading, parameter validation, and training execution.
- Updated the train_test_split function to enforce DataFrame input type, ensuring consistency in data handling.
- Removed the deprecated model_repository.py file to streamline the codebase.
- Adjusted cleanup schedule logic to improve error handling and logging during schedule reconciliation.
- Updated tests to reflect changes in the training workflow and repository interactions.
2026-04-09 12:09:52 -03:00
vitor-aignosi
0ae03b246f feat: update environment configuration and remove deprecated model serving
- Modified `.env.example` to set local defaults for PostgreSQL, MLflow, and MinIO configurations.
- Added MongoDB configuration parameters to the environment setup.
- Updated `README.md` to reflect changes in workflow input parameters and task queue naming conventions.
- Removed the `ModelServing` class to streamline the codebase, as it was deemed unnecessary.
- Adjusted `connectors_config.py` to align with new environment variable names and improve clarity.
- Updated tests to reflect changes in configuration handling and removed tests related to the deleted `ModelServing` class.
2026-04-07 16:58:50 -03:00
vitor-aignosi
342a02d6f7 feat: enhance training workflow with model metadata loading and refactor data handling
- Introduced a new activity to load model metadata from the model store.
- Refactored training logic to utilize new model metadata and improved parameter handling.
- Updated the `TrainModelParams` class to include additional fields for model configuration.
- Replaced deprecated utility functions with a custom train-test split implementation.
- Removed unused utility functions and cleaned up the data manager repository.
- Adjusted experiment tracking to include model-specific metadata in notifications.
2026-03-24 14:39:28 -03:00
vitor-aignosi
cf5111e520 feat: integrate PluginStore and MinIO repository into model manager activities
- Added PluginStore integration for model management.
- Replaced StorageRepository with MinIORepository in Activities, Cleanup, and Training classes.
- Updated training logic to handle validation files and improved data management.
- Enhanced configuration for MinIO and PluginStore in connectors.
- Removed deprecated model repository and storage repository files.
- Updated environment variable handling for new configurations.
2026-03-11 17:35:05 -03:00
Bruno Domingues
41c4490cad SIENTIAPDE-1579: Fix SientiaMlException propagation in ModelServing.search_runs_by_name to correctly raise the exception with its message, resolving a TypeError. Enhance training test script to support scenario-specific CSV files for different test cases. Update search_runs_by_name return type hint and apply minor code formatting. 2026-02-19 21:13:41 -03:00
Kou Kinoshita
43cbc31e10 SIENTIAPDE-1579: Lint fixes and formatting 2026-02-18 17:31:58 -03:00
Kou Kinoshita
c6d6f94e05 SIENTIAPDE-1579: Updated tests 2026-02-18 15:05:37 -03:00
Kou Kinoshita
8b7bd81328 SIENTIAPDE-1579: Fixed data preprocessor 2026-02-18 11:14:27 -03:00
Kou Kinoshita
a4d94dd2ff SIENTIAPDE-1579: Added date format and convert methods 2026-02-13 18:41:46 -03:00
Kou Kinoshita
9e3585afce SIENTIAPDE-1579: Added local and validation flags to test script 2026-02-13 17:51:17 -03:00
Bruno Domingues
44dca9b66b SIENTIAPDE-1430: Add option to run all training test scenarios with summary reporting.
- Introduce --all argument to run_training_test.py to execute all available scenarios sequentially.
- Refactor single scenario execution into a run_single_scenario function for better modularity and error handling.
- Implement a print_summary function to report results for multiple scenario runs.
2025-12-18 21:46:51 -03:00
Bruno Domingues
06fd08dc70 SIENTIAPDE-1430: Introduce comprehensive integration testing with JSON-based scenarios and detailed README documentation. Enhance training workflow to support advanced model configurations, including polynomial regression with mandatory scaler validation. Ensure robust prediction handling by calculating training predictions (y_train_pred) before denormalization and automatically configuring datetime indices for time-series operations. 2025-12-18 17:05:10 -03:00
Bruno Domingues
463906073e SIENTIAPDE-1350: Refactor: Improve logging, configuration, and resource management. Includes .env updates, client initialization logging, and resource closing. 2025-11-24 23:08:29 -03:00
Bruno Domingues
c7f44a2423 SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations. 2025-11-06 15:34:03 -03:00
Kou-Kinoshita
1f6765dc86 SIENTIAPDE-1241: Linted files 2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64 SIENTIAPDE-1241: Formatted files 2025-10-29 11:55:03 -03:00
Bruno Domingues
5789a13023 SIENTIAPDE-1241: refactor train_model workflow due to I/O errors. 2025-10-22 15:37:56 -03:00