SIENTIAPDE-2226: Stop installing runtime in worker; unpin requirements

- The sientia-module entrypoint now merges requirements.txt with the
  Plugin Store runtime and installs both before boot (SIENTIAPDE-2223),
  so the worker no longer calls install_runtime or runs the loaded
  package version guard.
- Leave runtime libs unpinned in requirements files so the runtime
  version wins; cap sqlalchemy<2.1 and evidently<0.7 for compatibility.
- Update README, CLAUDE.md and worker tests to match.
This commit is contained in:
2026-10-27 10:43:41 -03:00
parent c71e426b18
commit 34eaa01ca7
6 changed files with 63 additions and 222 deletions

View File

@@ -32,7 +32,6 @@ from temporalio.runtime import PrometheusConfig, Runtime, TelemetryConfig
with workflow.unsafe.imports_passed_through():
import asyncio
import importlib.metadata
import os
import sys
@@ -77,36 +76,6 @@ def _get_runtime(runtime: str | None) -> str:
return normalized_runtime or 'single'
def _loaded_distribution_versions() -> dict[str, str]:
"""Version on disk of every installed distribution whose top-level module is imported."""
# Iterate a copy: another thread importing while we iterate would raise RuntimeError.
loaded = {name.partition('.')[0] for name in sys.modules.copy()}
return {
dist: importlib.metadata.version(dist)
for module, dists in importlib.metadata.packages_distributions().items()
if module in loaded
for dist in dists
}
def _runtime_changed_loaded_packages(before: dict[str, str]) -> dict[str, str]:
"""
Packages that were imported before ``install_runtime`` and whose version changed on disk.
The process keeps the old code in memory, so a later lazy import mixes two versions
(SIENTIAPDE-2214: sklearn 1.9.1 in memory, 1.8.0 on disk, ``_cyutility`` import error).
"""
changed = {}
for dist, old in before.items():
try:
new = importlib.metadata.version(dist)
except importlib.metadata.PackageNotFoundError:
new = 'removed'
if new != old:
changed[dist] = f'{old} -> {new}'
return changed
async def main():
"""
Main entry point for the Model Manager worker application.
@@ -114,9 +83,9 @@ async def main():
This function initializes and starts all components of the worker:
1. Sets up logging and metadata
2. Starts Prometheus metrics server
3. Initializes notification handler, then installs the runtime; exits if the install
changed the version of a package the worker already imported, and only then sets
``app_up`` (what the probes check) to 1
3. Initializes notification handler and Plugin Store client, then sets ``app_up``
(what the probes check) to 1. The runtime libraries are already installed: the
sientia-module entrypoint merges them into the resolved requirements before boot
4. Creates and configures activities
5. Starts Temporal client and workers
6. Manages worker lifecycle and graceful shutdown
@@ -158,8 +127,6 @@ async def main():
metrics_controller = MetricsController(logger=logger)
logger.custom_info(f'Installing runtime {runtime}', metadata)
plugin_store_parameters = build_plugin_store_config()
plugin_store = PluginStore(
base_url=plugin_store_parameters['base_url'],
@@ -177,21 +144,7 @@ async def main():
metrics_controller=metrics_controller,
)
loaded_before_install = _loaded_distribution_versions()
await plugin_store.install_runtime(runtime_name=runtime)
changed = _runtime_changed_loaded_packages(loaded_before_install)
if changed:
logger.custom_critical(
f'Runtime {runtime} changed packages already imported by the worker: {changed}. '
'Pin them in requirements.txt to the runtime versions.',
metadata,
)
metrics.APP_UP.labels(pod_id=POD_ID).set(0)
sys.exit(1)
# The probes only check app_up, so it goes to 1 after the guard: a pod that would fail it
# never turns Ready, and a rollout keeps the old pod.
# The probes only check app_up, so it goes to 1 only after initialization.
metrics.APP_UP.labels(pod_id=POD_ID).set(1)
activities = Activities(
@@ -281,7 +234,7 @@ def start_prometheus_server(logger: SientiaLogger, metadata: dict[str, str | Non
Starts the Prometheus metrics server for monitoring and observability.
This function initializes the Prometheus HTTP server on the configured port.
``main`` sets ``app_up`` to 1 later, once the runtime install passed the version guard.
``main`` sets ``app_up`` to 1 later, once initialization is done.
The server exposes metrics that can be scraped by Prometheus for monitoring
the health and performance of the Model Manager worker.