SIENTIAPDE-2226: Add plugin store e2e docs and cover repo edge cases
- Document PluginStore wiring e2e scenarios (4.1.1-4.1.5) covering env config, explicit branch, missing model, bad repo and bad credentials. - Add unit tests for DataManagerRepository date parsing, timestamp dropping, numeric coercion and timezone handling to keep coverage at 100%. - Mark the cleanup guard in TrainModel as `no branch`, since the false branch always exits through the re-raise.
This commit is contained in:
@@ -64,3 +64,15 @@ These scenarios test the business rule validations inside `validate_train_params
|
||||
| `test_scenario_3_1_1_cleanup_with_no_temp_dirs` | Temp directory is empty. Activity completes without error. |
|
||||
| `test_scenario_3_1_2_cleanup_removes_old_temp_dirs` | Two stale directories matching `name_YYYYMMDD_HHMMSS_microseconds` are removed when older than retention. |
|
||||
| `test_scenario_3_1_3_cleanup_nonexistent_temp_path` | Target path does not exist. Handled gracefully without error. |
|
||||
|
||||
## 4. PluginStore Wiring (`test_plugin_store_wiring.py`)
|
||||
|
||||
`Activities` receives `plugin_store_config` and builds the `PluginStore` itself. These scenarios run `01-linear-regression-basic.json` on a worker built from a per-test config (`activities_factory`) against the Gitea `model-store` repo.
|
||||
|
||||
| Test Function | Config Change | Expected Status | Description |
|
||||
|---|---|---|---|
|
||||
| `test_scenario_4_1_1_plugin_store_from_env_config` | `STORE_*` env vars → `build_plugin_store_config()` | `TRAINING_SUCCESS` | Same path as the worker. With `STORE_BRANCH` unset the store resolves the branch to `main`. |
|
||||
| `test_scenario_4_1_2_explicit_branch` | `branch = 'main'` | `TRAINING_SUCCESS` | Explicit branch skips resolution and is used for every read. |
|
||||
| `test_scenario_4_1_3_explicit_branch_without_model` | `branch = 'e2e-no-linear-regression'` | `ORCHESTRATOR_VALIDATION_ERROR` | Branch forked from `main` without `models/linear_regression/index.yaml`; error mentions `linear_regression/index.yaml`. |
|
||||
| `test_scenario_4_1_4_nonexistent_repository` | `repo = 'no-such-store'` | `ORCHESTRATOR_VALIDATION_ERROR` | `Activities` still builds (branch falls back to `main`); fails when loading model metadata. |
|
||||
| `test_scenario_4_1_5_wrong_credentials` | `password = 'wrong-password'` | `ORCHESTRATOR_VALIDATION_ERROR` | Gitea rejects the read; error mentions `401`. |
|
||||
|
||||
@@ -131,7 +131,9 @@ class TrainModel:
|
||||
training_succeeded = True
|
||||
finally:
|
||||
try:
|
||||
if train_result is not None:
|
||||
# train_result is only None when training raised, so the false branch
|
||||
# always leaves through the re-raise and never reaches the return below.
|
||||
if train_result is not None: # pragma: no branch
|
||||
await self._cleanup_resources(
|
||||
run_dir=train_result.get('run_dir'),
|
||||
metadata=metadata,
|
||||
|
||||
@@ -92,6 +92,52 @@ def test_ensure_date_column_parsed_invalid_raises():
|
||||
dmr._ensure_date_column_parsed(df, p)
|
||||
|
||||
|
||||
def test_ensure_date_column_parsed_unmapped_format_raises():
|
||||
df = pd.DataFrame({'ts': ['2024-01-01 10:00:00']})
|
||||
p = _params(date_column='ts')
|
||||
# Bypasses validate_business_rules, which would reject it earlier
|
||||
p.date_format = 'dd/MM/yyyy HH:mm'
|
||||
with pytest.raises(ValueError, match='is not mapped to a strftime pattern'):
|
||||
dmr._ensure_date_column_parsed(df, p)
|
||||
|
||||
|
||||
def test_drop_rows_with_missing_timestamp_without_date_column_returns_input():
|
||||
repo = dmr.DataManagerRepository(MagicMock())
|
||||
df = pd.DataFrame({'v1': [1.0, 2.0]})
|
||||
assert repo._drop_rows_with_missing_timestamp(df, _params(), {}) is df
|
||||
|
||||
|
||||
def test_drop_rows_with_missing_timestamp_non_object_dtype_drops_nat():
|
||||
repo = dmr.DataManagerRepository(MagicMock())
|
||||
df = pd.DataFrame(
|
||||
{'timestamp': pd.to_datetime(['2024-01-01', None, '2024-01-03']), 'v1': [1, 2, 3]}
|
||||
)
|
||||
out = repo._drop_rows_with_missing_timestamp(df, _params(), {})
|
||||
assert list(out['v1']) == [1, 3]
|
||||
|
||||
|
||||
def test_coerce_non_timestamp_columns_to_numeric_skips_date_column():
|
||||
repo = dmr.DataManagerRepository(MagicMock())
|
||||
df = pd.DataFrame({'timestamp': ['2024-01-01 10:00:00'], 'v1': ['1.5']})
|
||||
out = repo._coerce_non_timestamp_columns_to_numeric(df, _params(), {})
|
||||
assert out['timestamp'].iloc[0] == '2024-01-01 10:00:00'
|
||||
assert float(out['v1'].iloc[0]) == 1.5
|
||||
|
||||
|
||||
def test_set_timezone_on_index_converts_aware_index_to_utc():
|
||||
repo = dmr.DataManagerRepository(MagicMock())
|
||||
idx = pd.DatetimeIndex(['2024-01-01 00:00'], tz='America/Sao_Paulo')
|
||||
out = repo._set_timezone_on_index(pd.DataFrame({'v1': [1]}, index=idx))
|
||||
assert str(out.index.tz) == 'UTC'
|
||||
assert out.index[0] == pd.Timestamp('2024-01-01 03:00', tz='UTC')
|
||||
|
||||
|
||||
def test_set_timezone_on_index_non_datetime_index_raises():
|
||||
repo = dmr.DataManagerRepository(MagicMock())
|
||||
with pytest.raises(ValueError, match='Index is not a DatetimeIndex'):
|
||||
repo._set_timezone_on_index(pd.DataFrame({'v1': [1]}))
|
||||
|
||||
|
||||
def test_prepare_training_data_csv_load_failure():
|
||||
repo = dmr.DataManagerRepository(MagicMock())
|
||||
p = _params()
|
||||
|
||||
Reference in New Issue
Block a user