Commit Graph

287 Commits

Author SHA1 Message Date
d0b4644f63 chore(ci): add version-bump.yml, decoupled from deploy
New workflow bumps pyproject.toml's version on PR merge, calling the
shared reusable-version-bump.yml (SIENTIAPDE-2217 version-bump reform) —
zero build/image/chart/deploy side effects. deploy.yml (workflow_dispatch-
only) will read this committed value instead of calculating one.

[skip ci]
2026-10-21 00:18:51 -03:00
ced847bbe7 chore(ci): switch deploy.yml to manual workflow_dispatch with required chart_ref
Removes auto-deploy-on-PR-merge trigger and the old PR-merge gate. Deploy now
triggers only via workflow_dispatch, with chart_ref required (no default) as
the literal sientia-helm-chart branch bumped and deployed — hard error if it
doesn't exist, no silent fallback to development/main.

- .github/workflows/deploy.yml: drop pull_request trigger, add chart_ref
  input, branch_name now github.ref_name, drop pull-requests:read permission
- Delete .github/workflows/require-chart-ref-label.yml
- README.md: update Deploy section to describe manual trigger

[skip ci]
2026-10-20 17:03:33 -03:00
49c8942a87 chore(ci): remove feature/** do push trigger (evita run duplicado) [skip ci] 2026-10-19 15:38:47 -03:00
github-actions[bot]
3b90812c12 chore: bump version to 2.0.3 2026-10-16 15:10:26 +00:00
0b673b35a1 Merge pull request #33 from Aignosi/fix/SIENTIAPDE-2201
SIENTIAPDE-2201: Keep the timestamp column in train/test data CSVs
2026-10-16 12:10:01 -03:00
PedroHMCosme
37f16f34ff SIENTIAPDE-2201: fix: keep the timestamp column in train/test data CSVs
generate_report wrote train_data.csv and test_data.csv with index=False, which
dropped the UTC time index. laborious drift reads test_data.csv from MLflow as
its reference and needs a timestamp column; without it, every model trained here
drifted against the first 30% of its own target window. Both CSVs now write the
index as timestamp. If a column already has that name, the CSV is written as
before and a warning is logged, so training never fails over drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 10:09:06 -03:00
dd981b8027 fix(ci): Update env_list_path to array format in deploy workflow
This commit changes the env_list_path parameter in the deploy GitHub Actions workflow to an array format, aligning with expected input types for the action.
2026-09-11 00:41:42 -03:00
github-actions[bot]
5641734c3c chore: bump version to 2.0.2 2026-09-10 13:48:02 +00:00
Bruno Domingues
c51466f304 Merge pull request #32 from Aignosi/fix/SIENTIAPDE-2068
Configure deploy workflow for automated version updates
2026-09-10 10:47:41 -03:00
44a4ffe6d0 SIENTIAPDE-2068: Configure deploy workflow to update version in pyproject.toml. 2026-09-10 10:39:49 -03:00
Bruno Domingues
3f5f61dd07 Merge pull request #31 from Aignosi/fix/SIENTIAPDE-2068
CI/CD: Migrate Model Manager Deployment to Flow B
2026-09-09 22:11:09 -03:00
274192a5f3 SIENTIAPDE-2068: feat(cicd): migrate model-manager deploy to Flow B
- Replace legacy reusable-deploy.yml call (stale input shape, pointed at
  sientia-dataops-helm-repo) with reusable-deploy-dataops.yml — mirror to
  Gitea, dual-bump GITHUB_BRANCH (development + chart-ref label branch),
  helmfile deploy. First tarball release for this repo (create_tarball
  default true). workflow_dispatch has no inputs.
- Add require-chart-ref-label.yml.
2026-09-09 21:49:46 -03:00
c464681db5 Merge pull request #29 from Aignosi/fix/SIENTIAPDE-2072
SIENTIAPDE-2072: bump sientia_do pin to 1.12.2
2026-09-09 11:53:06 -03:00
20cb9827b0 Merge branch 'main' into fix/SIENTIAPDE-2072 2026-09-09 10:34:51 -03:00
github-actions[bot]
22d0c9c342 chore: bump version to 2.0.1 [skip ci] 2026-08-20 13:47:38 +00:00
a7d3b43db9 Merge pull request #30 from Aignosi/fix/model-library-version
Bump sientia-model-library to 0.14.0
2026-08-20 10:47:16 -03:00
5048ecbdd4 model-library-version: Update sientia-model-library from 0.8.3 to 0.14.0 in local requirements. 2026-08-20 10:40:02 -03:00
Eduardo Rios
ae8066cb26 SIENTIAPDE-2072: bump sientia_do pin to 1.12.2
Picks up the notification timestamp -> native datetime fix; also
aligns requirements.txt's floor with requirements-local.txt's pin
(was >=1.11.0/@1.12.0, out of sync with the other repos).
2026-08-17 16:42:24 -03:00
Bruno Domingues
85642bed25 Merge pull request #27 from Aignosi/release/SIENTIAPDE-1645
Comprehensive Updates: Training Workflow, Configuration, and Dependency Management
2026-08-05 13:17:43 -03:00
e3e1c712a8 SIENTIAPDE-1645: Refactor temporary directory cleanup activity by extracting processing and deletion logic into dedicated helper methods. Update train_test_split to use a local numpy.random generator for improved reproducibility. Remove an unnecessary else: pass block. 2026-08-05 12:10:54 -03:00
4133e200f7 SIENTIAPDE-1645: Remove validate.sh script, add jsonschema dependency, and improve test code readability 2026-08-05 11:23:22 -03:00
ebc7e5ce6f SIENTIAPDE-1645: Overhaul deployment, add Claude AI guidance, and enhance CI/CD.
Migrated to a shared sientia-module image and umbrella Helm chart, removing the standalone Dockerfile, .dockerignore, and values.yaml. Introduced CLAUDE.md for AI-assisted development. Updated quality-gate.yml with push triggers, concurrency, and refined permissions. Bumped project version to 2.0.0 to reflect these architectural changes.
2026-08-05 10:38:38 -03:00
github-actions[bot]
b6e59cf18e chore: bump version to 1.3.0 [skip ci] 2026-07-10 14:48:08 +00:00
Bruno Domingues
3457496c0a Merge pull request #28 from Aignosi/feature/SIENTIAPDE-1945
SIENTIAPDE-1945: Adopt Shared Docker/Kubernetes Deployment Strategy
2026-07-10 11:47:50 -03:00
159607fc72 SIENTIAPDE-1945: Adopt shared Docker image and Kubernetes deployment strategy.
Removed the local Dockerfile, .dockerignore, and values.yaml, as image building and Kubernetes deployment are now handled by external sientia-module components. Updated README accordingly.
2026-07-08 20:22:32 -03:00
1c06a8e584 ci(sonar): Update project key 2026-07-01 21:07:47 -03:00
2c63414770 SIENTIAPDE-1645: Expose detailed model training information via a new Prometheus gauge. This gauge, sientia_training_info, records metadata such as dataset sizes, feature count, and evaluation metrics (MSE, MAE, R2) along with the training run's timestamp. 2026-06-18 09:56:30 -03:00
bc65ebb955 SIENTIAPDE-1645: Use absolute time (time.time()) for training start timestamps instead of monotonic time. 2026-06-17 17:26:34 -03:00
214923b93e SIENTIAPDE-1645: Update pod_id label retrieval in training activities to use HOSTNAME environment variable with localhost fallback. 2026-06-17 16:12:51 -03:00
fe8a9384ba SIENTIAPDE-1645: Rename Prometheus 'sientia_app_up' metric to 'app_up' and update test assertion. 2026-06-17 12:26:25 -03:00
ea98032057 SIENTIAPDE-1645: Upgrade sqlalchemy to version 2.0.50 in requirements.txt 2026-06-17 11:20:23 -03:00
76f926a8ab SIENTIAPDE-1645: Add comprehensive model training observability metrics and Grafana dashboard. 2026-06-17 10:34:56 -03:00
445fe643fe chore: update .gitignore and requirements for development
- Added new entries to .gitignore to exclude temporary files and training input datasets, ensuring a cleaner repository.
- Included type stubs for psycopg2 in requirements-dev.txt to enhance type checking support for database interactions.
- Refactored the run_training_test.py script to implement a structured approach for loading and validating training input JSON files, improving the robustness of the training workflow.
2026-05-26 09:38:24 -03:00
64295d3855 refactor: update runtime configuration in values.yaml
- Changed the structure of the `runtimes` section in `values.yaml` to use a key-value format for better clarity and organization, replacing the previous list format.
- Updated the names of the runtimes to be keys, improving readability and maintainability of the configuration.
2026-05-22 16:25:20 -03:00
bcd4fab259 refactor: update input dataset and training parameters
- Renamed columns in `input_dataset.csv` from `feature_a`, `feature_b`, and `target` to `Counter`, `Rollout`, and `Square` for better clarity.
- Updated the `run_training_test.py` script to reflect the new column names in the workflow input, ensuring consistency in data processing.
- Added `date_column` parameter to the workflow input for improved data handling.
2026-05-11 12:49:04 -03:00
544c768ef4 chore: update sientia library versions in requirements files
- Updated `sientia-dataops-library` to version 1.12.0 and `sientia-model-library` to version 0.8.3 in `requirements-local.txt` for improved features and compatibility.
- Changed `sientia_do` and `sientia_model` in `requirements.txt` to use version specifiers for better flexibility in dependency management.
2026-05-06 17:15:16 -03:00
78167f7a44 fix: downgrade sientia_do version and update CSV saving logic
- Downgraded `sientia_do` version to 1.11.0 in `requirements.txt` to address compatibility issues.
- Updated `data_manager_repository.py` to save training and test data CSVs using float-cast versions of the data, ensuring consistency in the saved outputs.
- Enhanced test coverage in `test_data_manager_repository.py` to verify the presence of the target alias in the generated CSV files.
2026-05-05 16:26:47 -03:00
23912d854e chore: update dependencies and improve import statements
- Updated `sientia_do` version to 1.12.0 in `requirements.txt` for compatibility and feature enhancements.
- Refined import statements in `conftest.py` to improve type checking, ensuring better compatibility with type checkers.
2026-05-05 15:52:39 -03:00
ad125e22e1 chore: update .gitignore to include models directory
- Added 'models/' to .gitignore to prevent tracking of model files, ensuring a cleaner repository and avoiding unnecessary clutter in version control.
2026-05-05 11:00:14 -03:00
d1f9394879 feat: enhance E2E testing setup and model reporting
- Added a new fixture to manage runtime report artifacts in a writable temp directory during E2E tests, addressing permission issues in local CI/dev environments.
- Updated `conftest.py` to include a requirements.txt file in the model packaging path for training activities.
- Refactored existing fixtures to use `pytest.fixture` instead of `pytest_asyncio.fixture` for better compatibility.
- Enhanced the `Reports` class to include a target alias for report metrics, ensuring compatibility with Evidently's reporting requirements.
- Introduced new test scenarios to validate the handling of missing and whitespace-only `date_column` inputs in the training workflow.

These changes improve the robustness of the E2E testing framework and enhance the clarity of model reporting metrics.
2026-05-05 10:59:51 -03:00
ba9eb3d7c7 feat: require date_column in training parameters and update documentation
- Made `date_column` a required field in `TrainModelParams`, ensuring it must be present in the input data.
- Updated related documentation in `input-sample.md`, `README.md`, and various test scenarios to reflect the change in requirement.
- Adjusted the handling of `date_format` to default to `yyyy-MM-dd HH:mm:ss` if omitted, enhancing usability.
- Refined test scenarios to include new examples and ensure compliance with the updated parameter structure.

These changes improve the robustness of the model training workflow and clarify the expectations for input data.
2026-05-05 08:35:12 -03:00
6bd30e3328 feat: enhance test scenarios and configuration for regression models
- Updated `pyproject.toml` to include new linting rules for end-to-end tests.
- Modified `requirements-dev.txt` to add dependencies for E2E testing with `testcontainers` and `requests`.
- Refactored multiple JSON test scenario files to standardize structure, including new fields for `experiment_run_id`, `bucket_name`, and `file_name`.
- Improved model training parameters in `train_model_params.py` to use `experiment_name` directly.
- Adjusted `data_manager_repository.py` to utilize the updated `experiment_name` for logging.

These changes improve the organization and clarity of regression model tests and enhance the overall testing framework.
2026-05-04 11:11:16 -03:00
50d0ea6f32 doc update 2026-04-17 12:12:58 -03:00
31e95cbdf8 feat: log regression metrics as parameters in Training class
- Added a method to persist computed regression metrics (MSE, MAE, R²) as MLflow parameters during model training, enhancing model evaluation and tracking.
- Updated the Training class to log the equation path if available, improving artifact management.
2026-04-17 10:52:32 -03:00
b245801e09 fix: clear input example in Training class before model storage
- Set wrapper._input_example to None to ensure no residual data is stored with the model, enhancing data integrity during model storage.
2026-04-16 15:08:26 -03:00
e2b2f702f8 feat: integrate SientiaModel wrapper and enhance logging in Training class
- Imported SientiaModel to standardize the wrapper type in the Training class.
- Added conditional logging to capture training process details when a logger is provided, improving traceability during model training.
2026-04-16 14:51:41 -03:00
30dc98f140 feat: add template path debug print in Reports class for enhanced traceability
- Added a debug print statement to log the template path during report generation, improving visibility into the report generation process.
2026-04-16 14:43:59 -03:00
de2d7712e8 feat: add project base path and template path handling in report generation
- Introduced PROJECT_BASE_PATH constant for consistent project directory reference.
- Updated Reports class to require template_path for loading HTML templates.
- Modified DataManagerRepository to pass the new template_path when generating reports.
2026-04-16 14:43:25 -03:00
b2458ec354 refactor: simplify load_html_from_file function and add debug prints in Reports class
- Removed exception handling in load_html_from_file for cleaner code.
- Added debug print statements in the Reports class to log output directory, report path, and base path for better traceability during report generation.
2026-04-16 14:17:54 -03:00
17632477a8 chore: downgrade evidently dependency to version 0.6.7 in requirements.txt 2026-04-16 12:46:51 -03:00