gitea_admin/sientia-dataops-laborious-legacy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

QTZPOC-20

Browse Source 
QTZPOC-20: Unify artifact code directory handling for all model flavors and standardize joblib pickle path

Implemented a robust mechanism to ensure an artifact's code/ directory is prepended to sys.path before loading for sklearn, pyfunc, and joblib flavors. sklearn and pyfunc now resolve MLflow URIs to local directories first to facilitate this sys.path modification. Standardized the joblib flavor to explicitly read and write model.pkl within the artifact directory. Updated documentation and added comprehensive tests to reflect these changes.
This commit is contained in:
Vitor Pimentel dos Santos
2026-08-28 15:10:41 -03:00
parent f4c110cd8a
commit c1a44a9d7a
6 changed files with 475 additions and 273 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
README.md
View File

@@ -1003,10 +1003,7 @@ See [OPC UA Communication](#opc-ua-communication) for semantics, concurrency, an
| `IMPORT_MAX_COMPRESSION_RATIO` | Gate 4 compression ratio ceiling | `200` | No |
| `IMPORT_MODELS_COLLECTION` | MongoDB collection holding the model listing | `models` | No |
| `IMPORT_STATUS_DB_NAME` | Import log database name (the BFF's, on the `POSTGRES_*` server) | `sientia-core-mlops-bff` | If importing |
| `IMPORT_STATUS_DB_HOST` | Override: import log host, when it is not the `POSTGRES_*` server | `POSTGRES_HOST` | No |
| `IMPORT_STATUS_DB_PORT` | Override: import log port | `POSTGRES_PORT` | No |
| `IMPORT_STATUS_DB_USER` | Override: import log user | `POSTGRES_USER` | No |
| `IMPORT_STATUS_DB_PASSWORD` | Override: import log password | `POSTGRES_PASSWORD` | No |
| `IMPORT_STATUS_DB_{HOST,PORT,USER,PASSWORD}` | Override, only when the import log is not on the `POSTGRES_*` server | the `POSTGRES_*` value | No |
| `OPC_CONFIG` | OPC server configuration (JSON) | `{}` | No |
| `OPC_ID` | OPC server identifier | `1` | No |
| `OPC_URL` | OPC server URL | `opc.tcp://localhost:4840` | No |
@@ -1113,9 +1110,12 @@ Legacy MinIO object layout (relative key):
`joblib`. The flavor is configuration, never discovered from the artifact, and there is
no fallback between flavors: a failing loader propagates.
`joblib` reads the artifact directly — download, first top-level `.pkl`, `joblib.load` — with the
artifact's `code/` put on `sys.path` first, and it writes on retrain (`model.pkl` + `code/`, no
`MLmodel`, so only this runtime reads it back).
`joblib` reads the artifact directly — download, `<artifact>/model.pkl`, `joblib.load` — and writes on
retrain (`model.pkl` + `code/`, no `MLmodel`, so only this runtime reads it back).
Every flavor puts the artifact's own `code/` on `sys.path` before loading, **when that directory
exists**: MLflow's loaders only do it when the `MLmodel` manifest declares `code`, so `sklearn` and
`pyfunc` resolve the URI to a local directory first and load from there.
Full reference, including how to switch an existing model over: **[docs/model-flavors.md](docs/model-flavors.md)**.