SIENTIAPDE-2226: Stop installing runtime in worker; unpin requirements
- The sientia-module entrypoint now merges requirements.txt with the Plugin Store runtime and installs both before boot (SIENTIAPDE-2223), so the worker no longer calls install_runtime or runs the loaded package version guard. - Leave runtime libs unpinned in requirements files so the runtime version wins; cap sqlalchemy<2.1 and evidently<0.7 for compatibility. - Update README, CLAUDE.md and worker tests to match.
This commit is contained in:
24
README.md
24
README.md
@@ -90,7 +90,7 @@ An enterprise-grade ML model training orchestration platform built on Temporal.
|
||||
- **Scheduled Maintenance**: Automated lifecycle jobs for filesystem hygiene and stale file cleanup
|
||||
|
||||
### Advanced Capabilities
|
||||
- **Dynamic Runtime Provisioning**: Automated installation of required model runtimes from the Plugin Store
|
||||
- **Dynamic Runtime Provisioning**: The `sientia-module` entrypoint merges `requirements.txt` with the Plugin Store runtime and installs both before the worker starts
|
||||
- **Granular Retry Policies**: Tailored strategies for network, MLFlow, database, and filesystem operations
|
||||
- **Scalable Infrastructure**: Kubernetes-ready design with support for horizontal scaling and poller autoscaling
|
||||
- **Secure Configuration**: Environment-driven connection management with fallback to sensible defaults
|
||||
@@ -148,8 +148,7 @@ The Model Manager system uses a Temporal-based workflow architecture with clear
|
||||
- Task queue configuration and load balancing
|
||||
- Prometheus metrics server initialization
|
||||
- Notification handler setup and configuration
|
||||
- **Runtime Installation**: Automatically installs the required model runtime from the Plugin Store
|
||||
- **Runtime version guard**: boot fails if the install changes the version of a package the worker already imported, so the `requirements.txt` pins must match `model-plugin-factory` `runtime/basic.yaml` and `runtime/xgboost.yaml`; `app_up` goes to 1 only after this check
|
||||
- **Runtime libraries**: not installed by the worker. The `sientia-module` entrypoint merges `requirements.txt` with the Plugin Store runtime (`PLUGIN_RUNTIME`), writes the audited `requirements.resolved.txt` and installs it before Python starts; on a conflict the pod does not start (SIENTIAPDE-2223). `app_up` goes to 1 only after initialization
|
||||
- **Key Features**:
|
||||
- Automatic scaling with `PollerBehaviorAutoscaling`
|
||||
- Health check endpoints for Kubernetes liveness/readiness probes
|
||||
@@ -578,6 +577,18 @@ kubectl exec -n temporal <temporal-admin-tools-pod-name> -- \
|
||||
pip install -r requirements-dev.txt
|
||||
```
|
||||
|
||||
4. **Install the runtime libraries**
|
||||
|
||||
In the cluster, the `sientia-module` entrypoint installs the runtime libraries before the
|
||||
worker starts; the worker itself installs nothing. Locally, install them by hand from
|
||||
`runtime/<RUNTIME>.yaml` of the Plugin Store (`Aignosi/model-plugin-warehouse`), skipping
|
||||
`sientia_do`/`sientia_model`, which the requirements file already brings. For `basic`:
|
||||
```bash
|
||||
pip install mlflow==3.8.1 pathspec==1.0.4 pandas==2.3.3 cloudpickle==3.1.2 scikit-learn==1.8.0
|
||||
```
|
||||
Copy the versions from the YAML, not from this example: they change with the runtime. With
|
||||
`RUNTIME=single` there is nothing to install.
|
||||
|
||||
4. **Create environment configuration file**
|
||||
```bash
|
||||
cp .env.example .env
|
||||
@@ -812,6 +823,7 @@ Runs automatically on each Pull Request to `main`:
|
||||
- ✅ Tests must pass with minimum 80% coverage
|
||||
- ✅ SonarQube analysis for code quality metrics
|
||||
- ✅ Automatic version calculation based on branch pattern
|
||||
- ✅ Runtime requirements check (`runtime_check: true`): `requirements.txt` is merged with every `runtime/*.yaml` of `model-plugin-warehouse@main`, using the `sientia-module` entrypoint's own merge script; a conflict fails the gate, "runtime prevaleceu" is a warning. The warehouse changes on its own, so a push with no code change can turn red
|
||||
|
||||
#### Deploy (`.github/workflows/deploy.yml`)
|
||||
Triggered manually via `workflow_dispatch`, with a required `chart_ref` input (no
|
||||
@@ -1250,9 +1262,9 @@ sientia-dataops-model-manager/
|
||||
- Check training parameter validation errors
|
||||
- Verify input data format and required fields
|
||||
|
||||
5. **Worker exits at boot with `Runtime ... changed packages already imported by the worker`**
|
||||
- A runtime pin in `model-plugin-factory` `runtime/*.yaml` no longer matches `requirements.txt`; the message lists each package as `old -> new`
|
||||
- Set the runtime version in `requirements.txt` and `requirements-local.txt` here and in `sientia-dataops-laborious`, then redeploy
|
||||
5. **Pod does not start, with `# CONFLICTS` in the log**
|
||||
- The `sientia-module` entrypoint found a library whose version in `requirements.txt` conflicts with the Plugin Store runtime; the log lists each library with both specs
|
||||
- Leave libraries the runtime already defines without a version in `requirements.txt` (the runtime's version wins), or align the version, then redeploy. The quality gate catches this on the PR against the warehouse `main`
|
||||
|
||||
### Debug Mode
|
||||
|
||||
|
||||
Reference in New Issue
Block a user