vitor-aignosi
dac4f1718f
QTZPOC-20: Implement model import workflow for .sientia bundles Introduces the import_model workflow and its associated activities to enable importing .sientia bundles into MLflow and MongoDB. This includes: * New core workflow and activity logic for handling bundle download, decryption, validation, MLflow provisioning (experiments, runs, registered versions), and MongoDB model document creation. * Comprehensive E2E and unit test suites, including a dedicated test harness and a manual run script for development. * Updated .env.example and README.md with new configurations and usage instructions. * Ensures correct MLflow model source resolution for legacy loader compatibility and robust MongoDB document management. * Refactors existing utility scripts into a scripts/ directory.
37 lines
2.0 KiB
SQL
37 lines
2.0 KiB
SQL
-- V12 add_import_statuses_to_experiment_run, as a delta applied on top of `experiment_run.sql`.
|
|
--
|
|
-- Source of truth: sientia-dataops-database-migrations,
|
|
-- postgres/sientia-core-mlops-bff/migrations/V12__add_import_statuses_to_experiment_run.sql
|
|
--
|
|
-- Two things, and nothing else:
|
|
--
|
|
-- 1. the status vocabulary gains IMPORT_SUCCESS and IMPORT_ERROR;
|
|
-- 2. chk_experiment_run_terminal_status_run_type pairs each terminal value with its own
|
|
-- run_type, so a TRAIN row can never end IMPORT_SUCCESS and an IMPORT row can never end
|
|
-- TRAINING_ERROR. The two shared stage values (PENDING, ORCHESTRATOR_WAITING_PROC) stay out
|
|
-- of the pairing — both run types legitimately pass through them.
|
|
--
|
|
-- This file is applied separately because one e2e test runs an import against a table *without*
|
|
-- it: that failure shape — provisioned, then unrecordable, refused on the first attempt — is the
|
|
-- evidence for why this migration has to land before IMPORT_MODEL_ENABLED is turned on anywhere.
|
|
|
|
ALTER TABLE public.experiment_run
|
|
DROP CONSTRAINT IF EXISTS chk_experiment_run_status;
|
|
|
|
ALTER TABLE public.experiment_run
|
|
ADD CONSTRAINT chk_experiment_run_status CHECK (status IN ('PENDING',
|
|
'ORCHESTRATOR_REQUEST_ERROR',
|
|
'ORCHESTRATOR_WAITING_PROC',
|
|
'ORCHESTRATOR_VALIDATION_ERROR',
|
|
'TRAINING_SUCCESS', 'TRAINING_ERROR',
|
|
'IMPORT_SUCCESS', 'IMPORT_ERROR'));
|
|
|
|
ALTER TABLE public.experiment_run
|
|
DROP CONSTRAINT IF EXISTS chk_experiment_run_terminal_status_run_type;
|
|
|
|
ALTER TABLE public.experiment_run
|
|
ADD CONSTRAINT chk_experiment_run_terminal_status_run_type CHECK (
|
|
(run_type = 'IMPORT' OR status NOT IN ('IMPORT_SUCCESS', 'IMPORT_ERROR'))
|
|
AND (run_type = 'TRAIN' OR status NOT IN ('TRAINING_SUCCESS', 'TRAINING_ERROR'))
|
|
);
|