Add docs

docs/model-flavors.md

@@ -0,0 +1,112 @@
+# Model flavors
+
+The *flavor* is how this runtime is told to deserialize a model. It is **configuration, never
+discovery**: the value comes from the model document in the `models` collection and the code never
+reads the artifact's `MLmodel` manifest to guess it.
+
+```json
+{
+  "model_config": {
+    "transform_flavor": "sklearn",
+    "predict_flavor": "joblib",
+    "retention_minutes": 60,
+    "target": "SE"
+  }
+}
+```
+
+`predict_flavor` drives `load_predict_model` (artifact `prediction_model`); `transform_flavor` drives
+`load_transform_model` (artifact `data_model`). Both default to `sklearn` when the key is absent.
+
+## Accepted values
+
+| Flavor | Read | Write (retrain) | How it loads |
+|---|---|---|---|
+| `sklearn` | yes | yes | `mlflow.sklearn.load_model(<uri>)` |
+| `pyfunc` | yes | yes | `mlflow.pyfunc.load_model(<uri>)`; the wrapper path also unwraps `._model_impl.python_model` |
+| `pytorch` | yes | yes | `mlflow.pytorch.load_model(<uri>)` |
+| `joblib` | yes | yes | downloads the artifact and `joblib.load`s its pickle — **no MLflow flavor loader involved** |
+
+Anything else raises `ValueError` with `INVALID_FLAVOR_MESSAGE`, **before** any artifact is resolved
+or downloaded. The same message and the same four values govern reading and writing.
+
+**There is no fallback between flavors.** A failing loader propagates: one attempt, one cause. A
+model configured with the wrong flavor fails visibly instead of being rescued silently.
+
+## The `joblib` flavor
+
+Some models the pipeline produces do not load through an MLflow flavor loader even though the
+artifact is intact: the `.pkl` is in the compressed form `joblib.dump` writes, which MLflow's
+`pickle.load` does not read. Declaring `joblib` is how such a model is served.
+
+**Reading.** Resolve the Production run, `client.download_artifacts(run_id, <artifact>)`, take the
+**first top-level `.pkl` in name order**, and `joblib.load` it. The listing does not recurse, so a
+nested wrapper artifact (`artifacts/*.pkl`) is never deserialized in place of the model. No
+`MLmodel` is parsed and no `models:/<name>/production` URI is built.
+
+**The artifact's `code/` goes on `sys.path` first.** A model whose class lives inside the artifact
+would otherwise raise `ModuleNotFoundError` at unpickle time. Before `joblib.load`, the flavor
+prepends `<artifact>/code` to `sys.path` when that directory exists — the directory itself, matching
+MLflow's own convention where `code/utils/…` imports as the package `utils`. The entry is not
+duplicated and is **not** removed after the load, because the deserialized object may import
+lazily. A missing `code/` is not an error, and no extra download is performed: `download_artifacts`
+already brings `code/` along.
+
+> `sys.path` is process-global. Two different models that embed a package with the same name
+> (`utils/`, typically) and are served by the same worker resolve to whichever entered first — the
+> second silently runs the first one's code. This is pre-existing behaviour of MLflow's own pyfunc
+> path with the same artifact layout; the mitigation, if it ever bites, is one worker per model.
+
+**Writing / retraining.** `joblib` is a write flavor too, so a joblib model retrains. `log_model`
+writes `joblib.dump(model, 'model.pkl', compress=3)` under the artifact path and re-logs the source
+artifact's `code/` to `<artifact>/code`, so the retrained model stays deserializable. Registration
+and promotion to `Production` are unchanged.
+
+**The written artifact has no `MLmodel` manifest.** Only this runtime's `joblib` flavor reads it:
+`mlflow.sklearn.load_model` and `models:/<name>/production` both fail on it. A model retrained in
+joblib **stays** joblib — `predict_flavor` needs no edit after a retrain, and no consumer outside
+this runtime (the MLflow UI, MLflow serving, another service) recognizes that artifact as a model.
+
+**What `joblib` is for.** A pickled object exposing `predict(data)` — and `fit(data)` if it is
+retrained. It is **not** for a bare pyfunc `PythonModel` wrapper: `get_prediction_data` and
+`get_cached_operation` call `predict(data)` for every non-pyfunc flavor, so a wrapper expecting
+`predict(context, model_input)` raises `TypeError` at predict and retrain time.
+
+**Observability.** A joblib load is an ordinary load: `MODEL_READ_LAG` / `MODEL_READ_COUNT` on
+success and `MODEL_READ_ERROR_COUNT` on failure, under the base `operation_type`
+(`load_predict_model` / `load_transform_model`). A joblib write emits the usual `MODEL_WRITE_*`
+under `log_model`. No suffixed operation type, no new metric, no new label.
+
+## Switching a model to `joblib`
+
+The runtime does not migrate documents. Editing the `models` collection is an operator step, and the
+order matters:
+
+1. **Inventory first, before the deploy.** Collect the models that fail in their configured flavor
+   loader — run predict/transform and note which ones raise. There is no automatic discovery
+   afterwards: once deployed, such a model simply fails, and the failure names the flavor loader,
+   not the fix.
+2. **Edit the document.** Set `model_config.predict_flavor: 'joblib'` and/or
+   `model_config.transform_flavor: 'joblib'` for each model in the inventory. Nothing else changes;
+   models already on a working flavor need no edit and behave identically.
+3. **Imported models arrive ready.** The `import_model` workflow writes
+   `predict_flavor: 'joblib'` itself — see [`model-import.md`](model-import.md) § 12.
+4. **Check the first retrain.** In MLflow, confirm the new run carries
+   `prediction_model/model.pkl` **and** `prediction_model/code/` when the source model had a
+   `code/` directory, and that the promoted version loads on the next predict.
+5. **Rolling back is not just a revert.** Reverting the code leaves any artifact a joblib retrain
+   already wrote in MLflow: that Production version stops being loadable, so the rollback also means
+   promoting the previous version back in the registry, and setting the documents' flavors back —
+   manual, in reverse order.
+
+## Known gap: how the run id is resolved
+
+`get_model_run_id` resolves the Production run with `source.split('/')[2]` on the registered
+version's `source`. **This has not been confirmed against the real tracking server.** No artifact
+scheme reproduced locally puts the run id in that position (a file store yields `''`, S3 yields the
+bucket, `mlflow-artifacts:/` yields the experiment id), yet predict works in production — so the
+real `source` has a shape none of the local probes produced.
+
+This is not specific to `joblib`: every flavor that resolves a run goes through the same line, and
+the joblib read path depends on it. Confirm it with a **read-only** query against the tracking
+server before relying on it, ideally before the next deploy.