gitea_admin/sientia-dataops-scouter_temporal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SIENTIAPDE-1445

Browse Source 
Update pyproject.toml to enable automatic asyncio mode, modify pytest_asyncio fixture scopes in conftest.py for better isolation, and streamline e2e scenarios documentation in scenarios.md by removing outdated scenarios and reorganizing sections for clarity.
This commit is contained in:
vitor-aignosi
2025-12-30 16:18:52 -03:00
parent 4f076956cc
commit 90e801bdcd
9 changed files with 1546 additions and 353 deletions
Download Patch File Download Diff File Expand all files Collapse all files

364
e2e/scenarios.md
View File

@@ -187,22 +187,6 @@ The `pi_web_api_scouter` workflow:
---
#### Scenario 1.3.4: Missing Required Input Fields
**Description**: Missing required input fields
**Input**:
- Missing `model_id`, `model_name`, `schedule_name`, or `pi_web_api_query`
**Expected Behavior**:
- KeyError raised when accessing missing fields
- Workflow fails immediately
**Assertions**:
- KeyError or similar exception
- Workflow fails before any activity execution
---
## 2. CoreScouter - Child Workflow Scenarios
### 2.1 Success Scenarios
@@ -336,43 +320,7 @@ The `pi_web_api_scouter` workflow:
### 2.3 Error Scenarios
#### Scenario 2.3.1: Data Quality Gate Error
**Description**: Error during quality gate processing
**Input**:
- Invalid filter configuration
- Filter function raises exception
**Expected Behavior**:
- Exception caught in quality gate
- Notification sent with `DATA_QUALITY_GATE_ISSUES`
- Exception propagated (workflow fails after retries)
**Assertions**:
- Error notification sent
- Workflow fails
---
#### Scenario 2.3.2: Aggregation Error
**Description**: Error during data aggregation
**Input**:
- Invalid aggregation function
- Data format issues
**Expected Behavior**:
- Invalid function sends notification with `AGGREGATION_ISSUES`
- Returns `'continue'` for invalid function (skips that tag)
- Other errors raise exception
**Assertions**:
- Invalid function handled gracefully
- Other errors cause workflow failure
---
#### Scenario 2.3.3: Redis Connection Error
#### Scenario 2.3.1: Redis Connection Error
**Description**: Redis unavailable during `group_and_hold_data`
**Input**:
@@ -390,25 +338,7 @@ The `pi_web_api_scouter` workflow:
---
#### Scenario 2.3.4: PostgreSQL Connection Error
**Description**: PostgreSQL unavailable during export
**Input**:
- Valid data
- PostgreSQL connection fails
**Expected Behavior**:
- `export_data_to_postgres` raises exception
- Notification sent with `ERROR_EXPORTING_DATA_TO_POSTGRES`
- Exception propagated (workflow fails after retries)
**Assertions**:
- Error notification sent
- Workflow fails
---
#### Scenario 2.3.5: PostgreSQL Unique Constraint Violation
#### Scenario 2.3.2: PostgreSQL Unique Constraint Violation
**Description**: Duplicate data violates unique constraint
**Input**:
@@ -429,283 +359,15 @@ The `pi_web_api_scouter` workflow:
## 3. Activity-Specific Scenarios
### 3.1 get_tag_values Activity
#### Scenario 3.1.1: Success with Valid WebIds
**Input**: All webids valid and present
**Expected**: Returns list of records with timestamp, name, value, tag
#### Scenario 3.1.2: Some WebIds are None
**Input**: Some webids in `model_tags` are `None`
**Expected**: None webids filtered out, only valid webids queried
#### Scenario 3.1.3: DataFrame with NaN Values
**Input**: PI Web API returns DataFrame with NaN values
**Expected**: NaN values handled, data normalized correctly
#### Scenario 3.1.4: Timestamp Normalization
**Input**: Multiple timestamps in response
**Expected**: All timestamps normalized to max timestamp value
> **Note**: Activity-specific scenarios are better suited for unit tests rather than e2e tests.
> These scenarios are covered indirectly through workflow e2e tests. For detailed activity testing,
> refer to the unit test suite in `tests/activities/`.
---
### 3.2 data_quality_gate Activity
## 5. Test Data Requirements
#### Scenario 3.2.1: No Filters Configured
**Input**: Empty `filters: {}`
**Expected**: Data passes through unchanged, filtered by model_tags only
#### Scenario 3.2.2: NULL_VALUES_FILTER with DISCARD Policy
**Input**: Data with null values, policy `DISCARD`
**Expected**: Null rows removed, notification sent
#### Scenario 3.2.3: OUT_OF_BOUNDS_FILTER with WARN Policy
**Input**: Data outside range, policy `WARN`
**Expected**: Notification sent, data kept
#### Scenario 3.2.4: Unknown Filter Type
**Input**: Filter name not in `quality_gate_filters`
**Expected**: Warning logged, filter skipped, processing continues
#### Scenario 3.2.5: Filter Removes All Data
**Input**: Filter that removes all rows
**Expected**: Empty DataFrame returned, processing continues
---
### 3.3 aggregate_data Activity
#### Scenario 3.3.1: Single Value Per Tag
**Input**: One data point per tag
**Expected**: Fast path returns value directly
#### Scenario 3.3.2: Multiple Values - Latest (lts)
**Input**: Multiple points, `aggr_function: 'lts'`
**Expected**: Returns last value in sorted order
#### Scenario 3.3.3: Multiple Values with NaN
**Input**: Some NaN values in series
**Expected**: NaN values dropped before aggregation
#### Scenario 3.3.4: All NaN Values
**Input**: All values are NaN
**Expected**: Returns `None`, tag skipped
#### Scenario 3.3.5: Invalid Aggregation Function
**Input**: Unknown `aggr_function`
**Expected**: Notification sent, returns `'continue'`, tag skipped
#### Scenario 3.3.6: Empty DataFrame After Filtering
**Input**: No data after quality gate
**Expected**: Returns empty DataFrame dict
---
### 3.4 group_and_hold_data Activity
#### Scenario 3.4.1: First Run - No Existing Data
**Input**: No existing data in Redis for key
**Expected**: Creates new `data_hold` dict, stores in Redis
#### Scenario 3.4.2: Subsequent Run - Existing Data
**Input**: Existing `data_hold` in Redis
**Expected**: Merges new data with existing, updates timestamp
#### Scenario 3.4.3: Removed Tags Cleanup
**Input**: Tags removed from `model_tags`
**Expected**: Removed tags deleted from `data_hold`
#### Scenario 3.4.4: Empty Input Data
**Input**: Empty DataFrame
**Expected**: Returns empty dict, warning logged
#### Scenario 3.4.5: Redis Get Error
**Input**: Redis get operation fails
**Expected**: Notification sent, exception raised
#### Scenario 3.4.6: Redis Set Error
**Input**: Redis set operation fails
**Expected**: Notification sent, exception raised
---
### 3.5 export_data_to_postgres Activity
#### Scenario 3.5.1: Successful Insert
**Input**: Valid data, no conflicts
**Expected**: Data inserted, `affected_rows > 0`
#### Scenario 3.5.2: Conflict with Ignore Policy
**Input**: Duplicate data, `on_conflict: 'ignore'`
**Expected**: Duplicates ignored, `affected_rows` may be less than total
#### Scenario 3.5.3: Conflict with Replace Policy
**Input**: Duplicate data, `on_conflict: 'replace'`
**Expected**: Duplicates updated, `affected_rows` includes updates
#### Scenario 3.5.4: Timestamp Conversion
**Input**: String timestamps in data
**Expected**: Timestamps converted to datetime format
#### Scenario 3.5.5: Database Connection Error
**Input**: Database unavailable
**Expected**: Exception raised, notification sent
---
### 3.6 write_metrics Activity
#### Scenario 3.6.1: Success with Valid Values
**Input**: Data with non-None values
**Expected**: Metrics written for all non-None values
#### Scenario 3.6.2: Some None Values
**Input**: Some values are None
**Expected**: None values skipped, only non-None values written
#### Scenario 3.6.3: All None Values
**Input**: All values are None
**Expected**: No metrics written, activity completes
---
### 3.7 store_data_package Activity
#### Scenario 3.7.1: Success
**Input**: Valid data and held_data
**Expected**: Package stored in Redis with TTL 120
#### Scenario 3.7.2: Redis Error
**Input**: Redis set fails
**Expected**: Notification sent, exception raised
---
## 4. Integration Scenarios
### 4.1 End-to-End Scenarios
#### Scenario 4.1.1: Complete Happy Path
**Description**: Full workflow from API to database
**Flow**:
1. PI Web API returns data
2. Quality gate passes
3. Aggregation succeeds
4. Redis storage succeeds
5. PostgreSQL export succeeds
6. Metrics written
7. Debug package stored (if enabled)
**Assertions**:
- All activities called
- Data in all storage layers
- No errors
---
#### Scenario 4.1.2: Partial Failure with Retry
**Description**: Activity fails, retries succeed
**Flow**:
1. First attempt fails (e.g., Redis timeout)
2. Retry policy triggers
3. Second attempt succeeds
4. Workflow continues
**Assertions**:
- Retry policy applied
- Workflow eventually succeeds
- Error logged but not fatal
---
#### Scenario 4.1.3: Complete Failure After Retries
**Description**: Activity fails after all retries exhausted
**Flow**:
1. Activity fails repeatedly
2. Retry policy exhausted
3. Workflow fails
**Assertions**:
- All retries attempted
- Workflow fails with error
- Error notification sent
---
## 5. Edge Cases and Boundary Conditions
### 5.1 Data Edge Cases
#### Scenario 5.1.1: Very Large Dataset
**Input**: Thousands of data points
**Expected**: Handles efficiently, all processed
#### Scenario 5.1.2: Single Data Point
**Input**: One tag, one data point
**Expected**: Processes correctly
#### Scenario 5.1.3: Extreme Values
**Input**: Very large or very small numeric values
**Expected**: Handled correctly, no overflow
#### Scenario 5.1.4: Special Characters in Tag Names
**Input**: Tag names with special characters
**Expected**: Handled correctly
---
### 5.2 Configuration Edge Cases
#### Scenario 5.2.1: Very Short Retention Time
**Input**: `retention_time: 1` (1 second)
**Expected**: Data expires quickly but workflow completes
#### Scenario 5.2.2: Very Long Retention Time
**Input**: `retention_time: 86400` (1 day)
**Expected**: Data persists for full duration
#### Scenario 5.2.3: Max Count = 1
**Input**: `max_count: 1`
**Expected**: Only latest value retrieved
#### Scenario 5.2.4: Max Count = Large Number
**Input**: `max_count: 10000`
**Expected**: Many values retrieved and processed
---
### 5.3 Concurrent Execution Scenarios
#### Scenario 5.3.1: Multiple Workflows Same Schedule
**Input**: Two workflows with same `schedule_name` running concurrently
**Expected**: Both complete, data merged correctly in Redis
#### Scenario 5.3.2: Multiple Workflows Different Schedules
**Input**: Multiple workflows with different `schedule_name`
**Expected**: Each uses separate Redis keys, no interference
---
## 6. Performance Scenarios
### 6.1 Load Scenarios
#### Scenario 6.1.1: High Throughput
**Input**: Many tags, frequent execution
**Expected**: Handles load efficiently
#### Scenario 6.1.2: Large Payload
**Input**: Large amount of data per tag
**Expected**: Processes within timeout limits
---
## 7. Test Data Requirements
### 7.1 Valid Test Data Structure
### 5.1 Valid Test Data Structure
```python
{
@@ -736,7 +398,7 @@ The `pi_web_api_scouter` workflow:
}
```
### 7.2 Mock PI Web API Response
### 5.2 Mock PI Web API Response
```python
DataFrame({
@@ -749,16 +411,16 @@ DataFrame({
---
## 8. Test Implementation Notes
## 6. Test Implementation Notes
### 8.1 Test Organization
### 6.1 Test Organization
- Group tests by scenario category
- Use descriptive test names matching scenario IDs
- Share fixtures for common setup
- Use parametrized tests for similar scenarios
### 8.2 Assertions Checklist
### 6.2 Assertions Checklist
For each scenario, verify:
- [ ] Correct activities called
@@ -769,7 +431,7 @@ For each scenario, verify:
- [ ] No unexpected errors
- [ ] Workflow state correct
### 8.3 Mock Configuration
### 6.3 Mock Configuration
- Mock PI Web API client responses
- Use fake Redis (fakeredis)
@@ -780,7 +442,7 @@ For each scenario, verify:
---
## 9. Priority Scenarios
## 7. Priority Scenarios
### High Priority (Must Test)
1. Scenario 1.1.1: Happy Path