38 lines
1.4 KiB
Markdown
38 lines
1.4 KiB
Markdown
# Scouter end-to-end tests
|
|
|
|
Production-faithful E2E tests for `Scouter`, `PIWebAPIScouter`, and `CoreScouter` against real backing services.
|
|
|
|
## Requirements
|
|
|
|
- Docker (for testcontainers: MongoDB, Redis, PostgreSQL)
|
|
- Python 3.11+ with dev dependencies: `pip install -r requirements-dev.txt`
|
|
|
|
## Run locally
|
|
|
|
```bash
|
|
pytest e2e/ --override-ini testpaths=e2e -m e2e -v
|
|
```
|
|
|
|
Stop on first failure:
|
|
|
|
```bash
|
|
pytest e2e/ --override-ini testpaths=e2e -m e2e -x
|
|
```
|
|
|
|
## Coverage (separate from unit tests)
|
|
|
|
```bash
|
|
COVERAGE_FILE=.coverage.e2e pytest e2e/ --override-ini testpaths=e2e -m e2e --cov=scouter --cov-branch
|
|
coverage combine .coverage .coverage.e2e && coverage report
|
|
```
|
|
|
|
## Scenario catalog
|
|
|
|
See [scenarios.md](scenarios.md) for numbered scenarios and how they map to `test_scenario_*` functions. Section `## 0` of that file lists the harness smoke tests in `test_harness_smoke.py` (infra liveness checks, not business scenarios).
|
|
|
|
## Production code is not mocked
|
|
|
|
E2E uses real testcontainers, an in-process PI Web API HTTP server, `WorkflowEnvironment.start_local()`, and production `Activities` wiring. The only stand-ins are `mock_logger` and the optional `notification_inserts` spy. If a scenario fails due to a production defect, it is marked `xfail(strict=True)` and tracked in `openspec/changes/standardize-and-complete-e2e-tests/notes.md` when applicable.
|
|
|
|
Unit tests under `tests/` remain Docker-free and run with the default `pytest` invocation.
|