sientia-dataops-model-manager/docs/scenarios.md

E2E Test Scenarios

This document maps the workflow scenarios tested in the E2E suite to their corresponding JSON input files and expected behaviors.

Infrastructure (second-pass review)

• Containers: PostgreSQL, MinIO, MongoDB, and Gitea via testcontainers; real clients and Activities code paths.
• MLflow: file:// tracking URI (real SDK, no remote server).
• Temporal: WorkflowEnvironment.start_time_skipping() — official temporalio test runtime; workflows and activities are not stubbed.
• Logging/metrics: get_logger + MetricsController (sientia_do); no unittest.mock for observability in e2e/conftest.py.
• Unit tests under tests/ may still use mocks where appropriate; that policy is separate from this E2E suite.

1. TrainModel Workflow (test_train_model_workflow.py)

1.1 Happy Paths (Successful execution)

Test Function	Input JSON	Expected Status	Description
test_scenario_1_1_1_linear_regression_basic	01-linear-regression-basic.json	TRAINING_SUCCESS	Basic linear regression without scaler. Verifies end-to-end pipeline.
test_scenario_1_1_2_linear_regression_with_scaler	02-linear-regression-with-scaler.json	TRAINING_SUCCESS	Linear regression with Standard Scaler.
test_scenario_1_1_3_polynomial_regression_degree2_with_scaler	03-polynomial-regression-degree2.json	TRAINING_SUCCESS	Polynomial regression (degree 2) with Standard Scaler.
test_scenario_1_1_4_polynomial_regression_degree3_with_scaler	04-polynomial-regression-degree3.json	TRAINING_SUCCESS	Polynomial regression (degree 3) with Standard Scaler.
test_scenario_1_1_5_linear_regression_with_lags	05-linear-regression-with-lags.json	TRAINING_SUCCESS	Linear regression with lag_train/lag_val per variable.
test_scenario_1_1_6_linear_regression_nan_interpolation	06-linear-regression-nan-interpolation.json	TRAINING_SUCCESS	Linear regression with nan_treatment='linear interpolation'.
test_scenario_1_1_7_linear_regression_static_window_removal	07-linear-regression-static-window-removal.json	TRAINING_SUCCESS	rem_static_win=true with default static_threshold.
test_scenario_1_1_8_linear_regression_with_limits	08-linear-regression-with-limits.json	TRAINING_SUCCESS	support_filters with min/max per variable.
test_scenario_1_1_9_polynomial_degree2_scaler_and_lags	09-polynomial-degree2-with-scaler-and-lags.json	TRAINING_SUCCESS	Polynomial (degree 2), Standard Scaler, and lags.
test_scenario_1_1_10_linear_regression_with_ar_opt_params	10-linear-regression-with-ar.json	TRAINING_SUCCESS	opt_params.include_ar=true (placeholder for future AR behavior).
test_scenario_1_1_11_linear_regression_static_threshold_custom	11-linear-regression-static-threshold-custom.json	TRAINING_SUCCESS	rem_static_win=true with custom static_threshold.
test_scenario_1_1_12_alternate_date_format_dd_mm_yyyy	12-angular-test-date-format.json	TRAINING_SUCCESS	date_column=DATA, dd/MM/yyyy format, object training_data_dd_mm_yyyy.csv.
test_scenario_1_1_13_alternate_csv_narrow_date_window	13-angular-test-double-date-column.json	TRAINING_SUCCESS	Same alternate CSV with a bounded start_date/end_date window.
test_scenario_1_1_14_polynomial_with_support_filters	14-angular-test-polynomial-support-filters.json	TRAINING_SUCCESS	Polynomial (degree 4), scaler, upper_line/lower_line support filters.
test_scenario_1_1_15_linear_regression_custom_target_column_name	15-linear-regression-custom-target-column.json	TRAINING_SUCCESS	Custom target_variable column name (not literal target); Evidently/report columns must match.
test_scenario_1_1_16_naive_timestamp_header_column	16-linear-regression-naive-timestamp-header.json	TRAINING_SUCCESS	date_column=Timestamp, naive CSV training_data_timestamp_naive.csv.
test_scenario_1_1_17_linear_regression_blank_timestamp_row_dropped	17-linear-regression-blank-timestamp-row.json	TRAINING_SUCCESS	One empty timestamp cell; row dropped before index.

1.2 Error Paths

Test Function	Input JSON	Expected Status	Description
test_scenario_1_2_1_minio_file_not_found	01-linear-regression-basic.json	TRAINING_ERROR	MinIO file does not exist. Workflow fails during file download.
test_scenario_1_2_2_experiment_run_id_not_in_db	01-linear-regression-basic.json	N/A (raises Exception)	experiment_run_id does not exist in DB. Workflow fails immediately on status update attempt.

2. Parameter Validation (test_train_model_validation.py)

These scenarios test the business rule validations inside validate_train_params. All are expected to terminate with ORCHESTRATOR_VALIDATION_ERROR.

Test Function	Modification	Expected Error Substring
test_scenario_2_1_1_train_size_out_of_range	train_size = 5	'train_size'
test_scenario_2_1_2_empty_variable_columns	variable_columns = []	'variable_columns'
test_scenario_2_1_3_invalid_date_format	date_format = 'INVALID'	'date_format'
test_scenario_2_1_4_whitespace_only_model_name	model_name = ' '	'model_name'
test_scenario_2_1_5_unknown_model_type	model_type = 'totally_unknown_model'	'totally_unknown_model'
test_scenario_2_1_6_missing_target_variable	target_variable = ''	'target_variable'
test_scenario_2_1_7_missing_experiment_run_id	Missing experiment_run_id	N/A (raises ValueError immediately)
test_scenario_2_1_8_missing_date_column	Missing date_column	N/A (raises ValueError immediately)
test_scenario_2_1_9_whitespace_date_column	date_column = ' '	'date_column'

3. CleanupFiles Workflow (test_cleanup_files_workflow.py)

Test Function	Description
test_scenario_3_1_1_cleanup_with_no_temp_dirs	Temp directory is empty. Activity completes without error.
test_scenario_3_1_2_cleanup_removes_old_temp_dirs	Two stale directories matching name_YYYYMMDD_HHMMSS_microseconds are removed when older than retention.
test_scenario_3_1_3_cleanup_nonexistent_temp_path	Target path does not exist. Handled gracefully without error.

4. PluginStore Wiring (test_plugin_store_wiring.py)

Activities receives plugin_store_config and builds the PluginStore itself. These scenarios run 01-linear-regression-basic.json on a worker built from a per-test config (activities_factory) against the Gitea model-store repo.

Test Function	Config Change	Expected Status	Description
test_scenario_4_1_1_plugin_store_from_env_config	STORE_* env vars → build_plugin_store_config()	TRAINING_SUCCESS	Same path as the worker. With STORE_BRANCH unset the store resolves the branch to main.
test_scenario_4_1_2_explicit_branch	branch = 'main'	TRAINING_SUCCESS	Explicit branch skips resolution and is used for every read.
test_scenario_4_1_3_explicit_branch_without_model	branch = 'e2e-no-linear-regression'	ORCHESTRATOR_VALIDATION_ERROR	Branch forked from main without models/linear_regression/index.yaml; error mentions linear_regression/index.yaml.
test_scenario_4_1_4_nonexistent_repository	repo = 'no-such-store'	ORCHESTRATOR_VALIDATION_ERROR	Activities still builds (branch falls back to main); fails when loading model metadata.
test_scenario_4_1_5_wrong_credentials	password = 'wrong-password'	ORCHESTRATOR_VALIDATION_ERROR	Gitea rejects the read; error mentions 401.