QTZPOC-20
QTZPOC-20: Refactor model import workflow, introduce atomic model ID generation, and clarify MinIO expiry. Fuses bundle download, decryption, MLflow experiment creation, and artifact upload into a single activity to prevent cross-worker issues. Implements an atomic counter for MongoDB model IDs, replacing the max(id) + 1 scan to avoid collisions during concurrent imports. Adds comprehensive documentation for MinIO object expiry mechanisms and refines lifecycle rule application. Improves error handling with clearer responsibility classification and centralizes MLflow repository injection.
This commit is contained in:
@@ -278,12 +278,10 @@ async def main():
|
||||
activities.record_import_terminal_status,
|
||||
activities.report_import_status_write_failure,
|
||||
activities.report_import_pipeline_failure,
|
||||
# Bundle
|
||||
activities.download_import_bundle,
|
||||
activities.open_import_bundle,
|
||||
# Bundle download/open, MLflow experiment creation and artifact upload — one
|
||||
# fused activity, so they never split across worker processes.
|
||||
activities.prepare_and_upload_import_bundle,
|
||||
# MLflow provisioning
|
||||
activities.create_import_experiment,
|
||||
activities.upload_import_artifacts,
|
||||
activities.register_import_model_version,
|
||||
# The model listing
|
||||
activities.write_import_model_document,
|
||||
|
||||
Reference in New Issue
Block a user