QTZPOC-20

QTZPOC-20: Implement model import workflow for .sientia bundles

Introduces the import_model workflow and its associated activities to enable importing .sientia bundles into MLflow and MongoDB. This includes:

*   New core workflow and activity logic for handling bundle download, decryption, validation, MLflow provisioning (experiments, runs, registered versions), and MongoDB model document creation.
*   Comprehensive E2E and unit test suites, including a dedicated test harness and a manual run script for development.
*   Updated .env.example and README.md with new configurations and usage instructions.
*   Ensures correct MLflow model source resolution for legacy loader compatibility and robust MongoDB document management.
*   Refactors existing utility scripts into a scripts/ directory.
This commit is contained in:
2026-08-25 17:07:42 -03:00
parent 0e445de386
commit dac4f1718f
41 changed files with 8168 additions and 1045 deletions

View File

@@ -77,6 +77,19 @@ with workflow.unsafe.imports_passed_through():
POD_ID = os.getenv('HOSTNAME')
SDK_METRICS_PORT = int(os.getenv('HTTP_SDK_METRICS_PORT', '9091'))
# The values that turn the import worker on. Anything else — unset, empty, `0`, `false`, a typo —
# leaves the four default workers alone and polls no import queue at all.
IMPORT_ENABLED_VALUES = ('1', 'true', 'yes', 'on')
def import_worker_enabled() -> bool:
"""Whether this runtime registers the fifth (import) worker.
Read at worker start, not per workflow: containment (QTZPOC-19) wants the importer in its own
runtime with its own credentials, and this flag is what makes that a deployment decision.
"""
return os.getenv('IMPORT_MODEL_ENABLED', '').strip().lower() in IMPORT_ENABLED_VALUES
async def main():
"""
@@ -250,7 +263,7 @@ async def main():
# its own credentials, and a flag lets that be a deployment decision instead of forcing every
# Laborious runtime to poll the import queue today. Unset, the four workers above are unaffected
# and no import queue is polled at all.
if os.getenv('IMPORT_MODEL_ENABLED', '').strip().lower() in ('1', 'true', 'yes', 'on'):
if import_worker_enabled():
logger.custom_info('Import model worker enabled', metadata_runtime)
workers.append(
prepare_worker(