sientia-dataops-scouter_temporal/e2e/scenarios.md

# Test Scenarios for PI Web API Scouter Workflow

This document describes all possible test scenarios for the `pi_web_api_scouter` workflow and its child workflow `core_scouter`.

## Workflow Overview

The `pi_web_api_scouter` workflow:
1. Retrieves tag values from PI Web API
2. Delegates processing to `core_scouter` child workflow which:
   - Applies data quality gates
   - Aggregates data
   - Groups and holds data in Redis
   - Exports to PostgreSQL
   - Writes metrics
   - Optionally stores debug data package

---

## 1. PI Web API Scouter - Main Workflow Scenarios

### 1.1 Success Scenarios

#### Scenario 1.1.1: Happy Path - Complete Success
**Description**: Workflow completes successfully with valid data from PI Web API

**Input**:
- Valid `model_name`, `model_id`, `schedule_name`
- Valid `pi_web_api_query` with endpoint, period, max_count, api_timeout
- Valid `model_tags` with webids and configurations
- Valid filters, schema, table_name, retention_time

**Expected Behavior**:
- `get_tag_values` returns non-empty list of records
- Workflow proceeds to `core_scouter`
- All activities execute successfully
- Data is stored in PostgreSQL
- Metrics are written
- Workflow completes without errors

**Assertions**:
- PI Web API client called once with correct parameters
- Data exists in SQLite (PostgreSQL substitute)
- Data cached in Redis
- Metrics written
- No errors raised

---

#### Scenario 1.1.2: Success with Multiple Tags
**Description**: Workflow processes multiple tags successfully

**Input**:
- Multiple tags in `model_tags` (3+ tags)
- Each tag has valid webid, aggr_function, data_range, frequency

**Expected Behavior**:
- All tags retrieved from PI Web API
- All tags processed through quality gates
- All tags aggregated correctly
- All tags stored in database

**Assertions**:
- Number of records matches number of tags
- All tags present in final data
- Aggregation applied per tag configuration

---

#### Scenario 1.1.3: Success with Debug Data Package Enabled
**Description**: Workflow completes with `debug_data_package=True`

**Input**:
- All standard input
- `debug_data_package: True`

**Expected Behavior**:
- Normal workflow execution
- `store_data_package` activity called
- Data package stored in Redis

**Assertions**:
- `store_data_package` called once
- Data package key exists in Redis
- Package contains both `data` and `held_data`

---

### 1.2 Early Exit Scenarios

#### Scenario 1.2.1: Empty Data from PI Web API
**Description**: PI Web API returns empty data

**Input**:
- Valid configuration
- PI Web API returns empty DataFrame or empty list

**Expected Behavior**:
- `get_tag_values` returns empty list `[]`
- Workflow checks `if not data:` and returns early
- `core_scouter` is NOT called
- Workflow completes without error

**Assertions**:
- PI Web API called once
- `core_scouter` NOT called
- No data in PostgreSQL
- No data in Redis (except possibly from previous runs)

---

#### Scenario 1.2.2: None Returned from PI Web API
**Description**: PI Web API returns None

**Input**:
- Valid configuration
- PI Web API returns None

**Expected Behavior**:
- `get_tag_values` returns None
- Workflow checks `if not data:` and returns early
- `core_scouter` is NOT called

**Assertions**:
- PI Web API called once
- `core_scouter` NOT called
- Workflow completes without error

---

### 1.3 Error Scenarios

#### Scenario 1.3.1: PI Web API Connection Error
**Description**: PI Web API client raises connection error

**Input**:
- Valid configuration
- PI Web API client raises `PIMSRequestError` or connection exception

**Expected Behavior**:
- `get_tag_values` catches exception
- Sends notification with `PI_WEB_API_REQUEST_ERROR`
- Raises exception (workflow fails after retries)

**Assertions**:
- Notification sent with correct error details
- Exception propagated to workflow
- Workflow fails (after retry policy exhausted)
- `core_scouter` NOT called

---

#### Scenario 1.3.2: PI Web API Timeout
**Description**: PI Web API request times out

**Input**:
- Valid configuration
- `api_timeout` set to low value
- PI Web API takes longer than timeout

**Expected Behavior**:
- Request times out
- Exception raised
- Notification sent
- Workflow fails after retries

**Assertions**:
- Timeout exception caught
- Notification sent
- Workflow fails

---

#### Scenario 1.3.3: Invalid Endpoint
**Description**: Invalid PI Web API endpoint provided

**Input**:
- Invalid endpoint path in `pi_web_api_query`

**Expected Behavior**:
- PI Web API client raises error
- Notification sent
- Workflow fails

**Assertions**:
- Error notification sent
- Workflow fails

---

## 2. CoreScouter - Child Workflow Scenarios

### 2.1 Success Scenarios

#### Scenario 2.1.1: Complete Processing Success
**Description**: All stages complete successfully

**Input**:
- Valid data from parent workflow
- Valid filters, model_tags, schema, table_name
- `fill_missing_tags: False`
- `debug_data_package: False`

**Expected Behavior**:
- `data_quality_gate` filters data
- `aggregate_data` aggregates by tag
- `group_and_hold_data` stores in Redis
- `export_data_to_postgres` writes to database
- `write_metrics` records metrics
- Workflow completes

**Assertions**:
- All activities called in correct order
- Data in PostgreSQL
- Data in Redis
- Metrics written
- `store_data_package` NOT called

---

#### Scenario 2.1.2: Success with Data Quality Filters
**Description**: Data quality filters applied successfully

**Input**:
- Data with some quality issues
- Filters configured with `NULL_VALUES_FILTER` or `OUT_OF_BOUNDS_FILTER`
- Policy set to `DISCARD` or `WARN`

**Expected Behavior**:
- Quality gate identifies issues
- Notification sent (WARNING level)
- If policy is `DISCARD`, bad rows removed
- Remaining data processed normally

**Assertions**:
- Quality issues detected
- Notification sent
- Bad data discarded if policy is `DISCARD`
- Good data processed

---

#### Scenario 2.1.3: Success with Different Aggregation Functions
**Description**: Different aggregation functions applied correctly

**Input**:
- Multiple tags with different `aggr_function`: `avg`, `mdn`, `max`, `min`, `lts`
- Time-series data with multiple points per tag

**Expected Behavior**:
- Each tag aggregated with its configured function
- Aggregated values correct for each function type

**Assertions**:
- `avg` calculates mean correctly
- `mdn` calculates median correctly
- `max` returns maximum value
- `min` returns minimum value
- `lts` returns latest value

---

#### Scenario 2.1.4: Success with Fill Missing Tags
**Description**: Missing tags filled with None

**Input**:
- `fill_missing_tags: True`
- Some tags missing from data

**Expected Behavior**:
- Missing tags added to `data_hold` with value `None`
- All expected tags present in final data

**Assertions**:
- Missing tags present with `None` value
- All model_tags represented in output

---

### 2.2 Early Exit Scenarios

#### Scenario 2.2.1: Empty Data After Grouping
**Description**: `group_and_hold_data` returns empty dict

**Input**:
- Data that results in empty `held_data` after grouping

**Expected Behavior**:
- `group_and_hold_data` returns `{}`
- Workflow checks `if held_data == {}:` and returns early
- `export_data_to_postgres` NOT called
- `write_metrics` NOT called
- `store_data_package` NOT called

**Assertions**:
- Early return after grouping
- No database export
- No metrics written
- Workflow completes without error

---

### 2.3 Error Scenarios

#### Scenario 2.3.1: Redis Connection Error
**Description**: Redis unavailable during `group_and_hold_data`

**Input**:
- Valid data
- Redis connection fails

**Expected Behavior**:
- `redis_repository.get()` or `redis_repository.set()` raises exception
- Notification sent with `REDIS_GET_ERROR` or `REDIS_SET_ERROR`
- Exception propagated (workflow fails after retries)

**Assertions**:
- Error notification sent
- Workflow fails

---

## 3. Activity-Specific Scenarios

> **Note**: Activity-specific scenarios are better suited for unit tests rather than e2e tests.
> These scenarios are covered indirectly through workflow e2e tests. For detailed activity testing,
> refer to the unit test suite in `tests/activities/`.

---

## 5. Test Data Requirements

### 5.1 Valid Test Data Structure

```python
{
    'model_name': 'test_model',
    'model_id': 'test_model_id',
    'schedule_name': 'test_schedule',
    'model_tags': {
        'tag1': {
            'webid': 'webid1',
            'aggr_function': 'avg',
            'data_range': [0, 100],
            'frequency': 60000,
        },
    },
    'trigger_laborious': False,
    'filters': {},
    'schema': 'test_schema',
    'table_name': 'test_table',
    'retention_time': 3600,
    'fill_missing_tags': False,
    'debug_data_package': False,
    'pi_web_api_query': {
        'endpoint': '/streamsets/recorded',
        'period': '*-1d',
        'max_count': 10,
        'api_timeout': 30,
    },
}
```

### 5.2 Mock PI Web API Response

```python
DataFrame({
    'timestamp': ['2024-01-01 12:00:00+0000', ...],
    'name': ['tag1', 'tag2', ...],
    'value': [10.5, 20.3, ...],
    'tag': ['webid1', 'webid2', ...],
})
```

---

## 6. Test Implementation Notes

### 6.1 Test Organization

- Group tests by scenario category
- Use descriptive test names matching scenario IDs
- Share fixtures for common setup
- Use parametrized tests for similar scenarios

### 6.2 Assertions Checklist

For each scenario, verify:
- [ ] Correct activities called
- [ ] Correct parameters passed
- [ ] Expected data in storage (SQLite/Redis)
- [ ] Expected notifications sent
- [ ] Expected metrics written
- [ ] No unexpected errors
- [ ] Workflow state correct

### 6.3 Mock Configuration

- Mock PI Web API client responses
- Use fake Redis (fakeredis)
- Use fake MongoDB (mongomock)
- Use SQLite for PostgreSQL
- Mock notification handler
- Mock metrics controller

---

## 7. Priority Scenarios

### High Priority (Must Test)
1. Scenario 1.1.1: Happy Path
2. Scenario 1.2.1: Empty Data
3. Scenario 1.3.1: API Connection Error
4. Scenario 2.1.1: Complete Processing
5. Scenario 2.2.1: Empty After Grouping
6. Scenario 2.3.4: PostgreSQL Error

### Medium Priority (Should Test)
1. Scenario 1.1.3: Debug Package
2. Scenario 2.1.2: Quality Filters
3. Scenario 2.1.3: Different Aggregations
4. Scenario 3.3.5: Invalid Aggregation

### Low Priority (Nice to Have)
1. Scenario 4.1.2: Retry Success
2. Scenario 5.1.1: Large Dataset
3. Scenario 5.3.1: Concurrent Execution