QTZPOC-20
QTZPOC-20: Refactor model import workflow, introduce atomic model ID generation, and clarify MinIO expiry. Fuses bundle download, decryption, MLflow experiment creation, and artifact upload into a single activity to prevent cross-worker issues. Implements an atomic counter for MongoDB model IDs, replacing the max(id) + 1 scan to avoid collisions during concurrent imports. Adds comprehensive documentation for MinIO object expiry mechanisms and refines lifecycle rule application. Improves error handling with clearer responsibility classification and centralizes MLflow repository injection.
This commit is contained in:
@@ -69,7 +69,7 @@ def _assert_platform_failure(
|
||||
f"status is {row['status']!r}; a provisioning failure is not the user's to fix"
|
||||
)
|
||||
|
||||
code, reason = IMPORT_FAILURE_CATALOG[step]
|
||||
code, reason, _responsibility = IMPORT_FAILURE_CATALOG[step]
|
||||
detail = _as_dict(row['orchestrator_response_data'])['import']
|
||||
assert detail['step'] == step.value, detail
|
||||
assert detail['code'] == code.value, detail
|
||||
@@ -152,11 +152,13 @@ class TestExperimentCreation:
|
||||
object_key=object_key,
|
||||
)
|
||||
|
||||
# The bundle *was* opened before MLflow was reached, so the names are on the row — which is
|
||||
# what task 9.3 asks for, observed from a failure that stopped exactly there.
|
||||
# Download, decryption, experiment creation and artifact upload are one fused activity
|
||||
# attempt (`model-import-fix-bugs`, bug 8): the workflow only learns the bundle's names once
|
||||
# that whole attempt returns, and an experiment-creation failure means it never does. The
|
||||
# placeholder names inserted with the row are therefore still there — narrowed from the old
|
||||
# "names before MLflow is touched" invariant, per `model-import-status-record`'s updated spec.
|
||||
assert row is not None
|
||||
assert row['experiment_name'] == import_names['experiment_name']
|
||||
assert row['run_name'] == f'import-{import_names["model_name"]}-v2'
|
||||
assert row['experiment_name'] == 'import-e2e-placeholder'
|
||||
|
||||
# Nothing in Mongo: the document is the last step and never ran.
|
||||
documents = list(import_models_collection.find({}, {'_id': False}))
|
||||
|
||||
Reference in New Issue
Block a user