SIENTIAPDE-2226: Add plugin store e2e docs and cover repo edge cases

- Document PluginStore wiring e2e scenarios (4.1.1-4.1.5) covering env
  config, explicit branch, missing model, bad repo and bad credentials.
- Add unit tests for DataManagerRepository date parsing, timestamp
  dropping, numeric coercion and timezone handling to keep coverage at
  100%.
- Mark the cleanup guard in TrainModel as `no branch`, since the false
  branch always exits through the re-raise.
This commit is contained in:
2026-10-27 12:22:14 -03:00
parent 51c34350f7
commit 3266d34620
3 changed files with 61 additions and 1 deletions

View File

@@ -64,3 +64,15 @@ These scenarios test the business rule validations inside `validate_train_params
| `test_scenario_3_1_1_cleanup_with_no_temp_dirs` | Temp directory is empty. Activity completes without error. |
| `test_scenario_3_1_2_cleanup_removes_old_temp_dirs` | Two stale directories matching `name_YYYYMMDD_HHMMSS_microseconds` are removed when older than retention. |
| `test_scenario_3_1_3_cleanup_nonexistent_temp_path` | Target path does not exist. Handled gracefully without error. |
## 4. PluginStore Wiring (`test_plugin_store_wiring.py`)
`Activities` receives `plugin_store_config` and builds the `PluginStore` itself. These scenarios run `01-linear-regression-basic.json` on a worker built from a per-test config (`activities_factory`) against the Gitea `model-store` repo.
| Test Function | Config Change | Expected Status | Description |
|---|---|---|---|
| `test_scenario_4_1_1_plugin_store_from_env_config` | `STORE_*` env vars → `build_plugin_store_config()` | `TRAINING_SUCCESS` | Same path as the worker. With `STORE_BRANCH` unset the store resolves the branch to `main`. |
| `test_scenario_4_1_2_explicit_branch` | `branch = 'main'` | `TRAINING_SUCCESS` | Explicit branch skips resolution and is used for every read. |
| `test_scenario_4_1_3_explicit_branch_without_model` | `branch = 'e2e-no-linear-regression'` | `ORCHESTRATOR_VALIDATION_ERROR` | Branch forked from `main` without `models/linear_regression/index.yaml`; error mentions `linear_regression/index.yaml`. |
| `test_scenario_4_1_4_nonexistent_repository` | `repo = 'no-such-store'` | `ORCHESTRATOR_VALIDATION_ERROR` | `Activities` still builds (branch falls back to `main`); fails when loading model metadata. |
| `test_scenario_4_1_5_wrong_credentials` | `password = 'wrong-password'` | `ORCHESTRATOR_VALIDATION_ERROR` | Gitea rejects the read; error mentions `401`. |

View File

@@ -131,7 +131,9 @@ class TrainModel:
training_succeeded = True
finally:
try:
if train_result is not None:
# train_result is only None when training raised, so the false branch
# always leaves through the re-raise and never reaches the return below.
if train_result is not None: # pragma: no branch
await self._cleanup_resources(
run_dir=train_result.get('run_dir'),
metadata=metadata,

View File

@@ -92,6 +92,52 @@ def test_ensure_date_column_parsed_invalid_raises():
dmr._ensure_date_column_parsed(df, p)
def test_ensure_date_column_parsed_unmapped_format_raises():
df = pd.DataFrame({'ts': ['2024-01-01 10:00:00']})
p = _params(date_column='ts')
# Bypasses validate_business_rules, which would reject it earlier
p.date_format = 'dd/MM/yyyy HH:mm'
with pytest.raises(ValueError, match='is not mapped to a strftime pattern'):
dmr._ensure_date_column_parsed(df, p)
def test_drop_rows_with_missing_timestamp_without_date_column_returns_input():
repo = dmr.DataManagerRepository(MagicMock())
df = pd.DataFrame({'v1': [1.0, 2.0]})
assert repo._drop_rows_with_missing_timestamp(df, _params(), {}) is df
def test_drop_rows_with_missing_timestamp_non_object_dtype_drops_nat():
repo = dmr.DataManagerRepository(MagicMock())
df = pd.DataFrame(
{'timestamp': pd.to_datetime(['2024-01-01', None, '2024-01-03']), 'v1': [1, 2, 3]}
)
out = repo._drop_rows_with_missing_timestamp(df, _params(), {})
assert list(out['v1']) == [1, 3]
def test_coerce_non_timestamp_columns_to_numeric_skips_date_column():
repo = dmr.DataManagerRepository(MagicMock())
df = pd.DataFrame({'timestamp': ['2024-01-01 10:00:00'], 'v1': ['1.5']})
out = repo._coerce_non_timestamp_columns_to_numeric(df, _params(), {})
assert out['timestamp'].iloc[0] == '2024-01-01 10:00:00'
assert float(out['v1'].iloc[0]) == 1.5
def test_set_timezone_on_index_converts_aware_index_to_utc():
repo = dmr.DataManagerRepository(MagicMock())
idx = pd.DatetimeIndex(['2024-01-01 00:00'], tz='America/Sao_Paulo')
out = repo._set_timezone_on_index(pd.DataFrame({'v1': [1]}, index=idx))
assert str(out.index.tz) == 'UTC'
assert out.index[0] == pd.Timestamp('2024-01-01 03:00', tz='UTC')
def test_set_timezone_on_index_non_datetime_index_raises():
repo = dmr.DataManagerRepository(MagicMock())
with pytest.raises(ValueError, match='Index is not a DatetimeIndex'):
repo._set_timezone_on_index(pd.DataFrame({'v1': [1]}))
def test_prepare_training_data_csv_load_failure():
repo = dmr.DataManagerRepository(MagicMock())
p = _params()