QTZPOC-20

QTZPOC-20: Implement explicit joblib model flavor and enhance model import workflow with extensive E2E tests.

This commit introduces a first-class joblib model flavor for direct loading and saving of Python pickle artifacts, including custom code within the artifact's code/ directory, bypassing MLflow's flavor loaders. The model import workflow's status database configuration is now streamlined to default to the POSTGRES_* server, with explicit IMPORT_STATUS_DB_* overrides.

Comprehensive end-to-end tests have been added for the model import workflow, covering success paths, gate rejections, digest enforcement, provisioning failures, status record integrity, cleanup, and retry/recovery scenarios using a real-clock environment. New E2
This commit is contained in:
2026-08-26 15:45:33 -03:00
parent dac4f1718f
commit e95e65b72b
32 changed files with 4247 additions and 1040 deletions

View File

@@ -82,8 +82,8 @@ class Activities(Storage, MLFlow, Gates, OPC, ModelMetrics, API, ModelImport):
notification handler keeps its own client either way.
import_config: `build_import_config()`; read from the environment when absent
import_status_config: `build_import_status_config()`; read from the environment when
absent. This is the BFF database holding the import log, a second Postgres
connection that is never the one Storage holds.
absent. This is the BFF database holding the import log: another database on the
`POSTGRES_*` server, and a second connection that is never the one Storage holds.
Raises:
Exception: If any parent class initialization fails

View File

@@ -288,8 +288,10 @@ class ModelImport(SientiaMonitoring):
"""
Args:
status_config: `build_import_status_config()` — the BFF database holding the import log.
This is a **second** Postgres connection, independent of the one Laborious already
holds, which is what makes the transactional independence of design D7 literal.
This is a **second** Postgres connection — another database, on the `POSTGRES_*`
server unless the IMPORT_STATUS_DB_* overrides move it — and its separateness from
the one Laborious already holds is what makes design D7's transactional
independence literal.
import_config: `build_import_config()` — bucket, prefix, ceilings, work directory,
envelope key and the models collection name.
minio_repository: the shared MinIO repository, for the uploaded bundle.