From the code-review-python pass on #34:
- README, CLAUDE.md and the module docstring: the worker re-executes
itself after install_runtime, and SIENTIA_RUNTIME_READY is internal;
setting it in .env/the chart would skip the install silently.
- "Installing runtime" is now logged only on the pass that installs; the
re-executed pass logs that the install was skipped.
- The ponytail comment notes that the install success metrics are lost
to the exec.
- New test: a failed install propagates without setting the flag or
calling os.execv (locks the install -> flag -> exec order).
The worker imports sklearn (metrics, evidently) before
PluginStore.install_runtime replaces it on disk with the runtime's pinned
version, the same order that breaks every sklearn prediction in laborious.
After installing the runtime, the worker now replaces its process once
(SIENTIA_RUNTIME_READY=1 skips the second install), so every module is
loaded from the runtime's packages.
generate_report wrote train_data.csv and test_data.csv with index=False, which
dropped the UTC time index. laborious drift reads test_data.csv from MLflow as
its reference and needs a timestamp column; without it, every model trained here
drifted against the first 30% of its own target window. Both CSVs now write the
index as timestamp. If a column already has that name, the CSV is written as
before and a warning is logged, so training never fails over drift.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit changes the env_list_path parameter in the deploy GitHub Actions workflow to an array format, aligning with expected input types for the action.
Picks up the notification timestamp -> native datetime fix; also
aligns requirements.txt's floor with requirements-local.txt's pin
(was >=1.11.0/@1.12.0, out of sync with the other repos).
Migrated to a shared sientia-module image and umbrella Helm chart, removing the standalone Dockerfile, .dockerignore, and values.yaml. Introduced CLAUDE.md for AI-assisted development. Updated quality-gate.yml with push triggers, concurrency, and refined permissions. Bumped project version to 2.0.0 to reflect these architectural changes.
Removed the local Dockerfile, .dockerignore, and values.yaml, as image building and Kubernetes deployment are now handled by external sientia-module components. Updated README accordingly.
- Added new entries to .gitignore to exclude temporary files and training input datasets, ensuring a cleaner repository.
- Included type stubs for psycopg2 in requirements-dev.txt to enhance type checking support for database interactions.
- Refactored the run_training_test.py script to implement a structured approach for loading and validating training input JSON files, improving the robustness of the training workflow.
- Changed the structure of the `runtimes` section in `values.yaml` to use a key-value format for better clarity and organization, replacing the previous list format.
- Updated the names of the runtimes to be keys, improving readability and maintainability of the configuration.
- Renamed columns in `input_dataset.csv` from `feature_a`, `feature_b`, and `target` to `Counter`, `Rollout`, and `Square` for better clarity.
- Updated the `run_training_test.py` script to reflect the new column names in the workflow input, ensuring consistency in data processing.
- Added `date_column` parameter to the workflow input for improved data handling.
- Updated `sientia-dataops-library` to version 1.12.0 and `sientia-model-library` to version 0.8.3 in `requirements-local.txt` for improved features and compatibility.
- Changed `sientia_do` and `sientia_model` in `requirements.txt` to use version specifiers for better flexibility in dependency management.
- Downgraded `sientia_do` version to 1.11.0 in `requirements.txt` to address compatibility issues.
- Updated `data_manager_repository.py` to save training and test data CSVs using float-cast versions of the data, ensuring consistency in the saved outputs.
- Enhanced test coverage in `test_data_manager_repository.py` to verify the presence of the target alias in the generated CSV files.
- Updated `sientia_do` version to 1.12.0 in `requirements.txt` for compatibility and feature enhancements.
- Refined import statements in `conftest.py` to improve type checking, ensuring better compatibility with type checkers.
- Added a new fixture to manage runtime report artifacts in a writable temp directory during E2E tests, addressing permission issues in local CI/dev environments.
- Updated `conftest.py` to include a requirements.txt file in the model packaging path for training activities.
- Refactored existing fixtures to use `pytest.fixture` instead of `pytest_asyncio.fixture` for better compatibility.
- Enhanced the `Reports` class to include a target alias for report metrics, ensuring compatibility with Evidently's reporting requirements.
- Introduced new test scenarios to validate the handling of missing and whitespace-only `date_column` inputs in the training workflow.
These changes improve the robustness of the E2E testing framework and enhance the clarity of model reporting metrics.
- Made `date_column` a required field in `TrainModelParams`, ensuring it must be present in the input data.
- Updated related documentation in `input-sample.md`, `README.md`, and various test scenarios to reflect the change in requirement.
- Adjusted the handling of `date_format` to default to `yyyy-MM-dd HH:mm:ss` if omitted, enhancing usability.
- Refined test scenarios to include new examples and ensure compliance with the updated parameter structure.
These changes improve the robustness of the model training workflow and clarify the expectations for input data.
- Updated `pyproject.toml` to include new linting rules for end-to-end tests.
- Modified `requirements-dev.txt` to add dependencies for E2E testing with `testcontainers` and `requests`.
- Refactored multiple JSON test scenario files to standardize structure, including new fields for `experiment_run_id`, `bucket_name`, and `file_name`.
- Improved model training parameters in `train_model_params.py` to use `experiment_name` directly.
- Adjusted `data_manager_repository.py` to utilize the updated `experiment_name` for logging.
These changes improve the organization and clarity of regression model tests and enhance the overall testing framework.
- Added a method to persist computed regression metrics (MSE, MAE, R²) as MLflow parameters during model training, enhancing model evaluation and tracking.
- Updated the Training class to log the equation path if available, improving artifact management.
- Imported SientiaModel to standardize the wrapper type in the Training class.
- Added conditional logging to capture training process details when a logger is provided, improving traceability during model training.
- Introduced PROJECT_BASE_PATH constant for consistent project directory reference.
- Updated Reports class to require template_path for loading HTML templates.
- Modified DataManagerRepository to pass the new template_path when generating reports.
- Removed exception handling in load_html_from_file for cleaner code.
- Added debug print statements in the Reports class to log output directory, report path, and base path for better traceability during report generation.
