SIENTIAPDE-1974

Add e2e tag-database scenarios and cover worker/email/formatter gaps

- Add e2e scenarios 1.9.1-1.9.4 for PI Web API through the tag
  database: scouter and output tag resolution, mixed-connector schedule
  hold, and OPC slot unchanged by PI connectors. Seed connectors and
  connector_tags from a new scenario input.
- Tighten the happy-path e2e to assert both schedules and that no PI
  output config is produced when the tag database is unused.
- Drop dead `if self.server` / `if notification_handler` guards in the
  email reconnect and worker shutdown paths, since both are always set.
- Log the traceback inside the message on worker failure, as the
  sientia_do Logger.error does not accept exc_info.
- Add unit tests for the worker failure exit, Prometheus startup
  failure, email reconnect without a password, and formatter, mongo,
  temporal manager and orchestrator edge cases.

e2e/scenarios.md

@@ -26,7 +26,9 @@ Source: `e2e/test_orchestrator_main_workflow.py`
 
 ### 1.1.1 Happy path
 
-Pipelines in Mongo → Temporal schedule created in the `laborious` namespace, `orchestrated_schedules` entries written.
+Pipelines in Mongo → both Temporal schedules created in the `laborious` namespace, one
+`orchestrated_schedules` entry per pipeline, and no PI Web API output config when no pipeline uses
+the tag database.
 
 ### 1.3.1 Create-only
 
@@ -64,6 +66,38 @@ Two `drift` pipelines on `frequency='1m'`: one without `interval_minutes` and wi
 `interval_minutes=60` and `chunk_period='s'` → both forwarded unchanged, no warning
 (SIENTIAPDE-2212).
 
+### 1.9 PI Web API through the tag database (connectors-pi-web-api)
+
+Scenario input: `orchestrator_pi_web_api.json` (two `pi_web_api` connectors, one `opc_ua`
+connector, their `connector_tags`, PI Web API pipelines by `tag_id` and legacy ones, and an OPC
+scouter by `tag_id`).
+
+#### 1.9.1 PI scouter through the tag database
+
+`pi_web_api_scouter` with `tag_id` read tags → `model_tags` WebID from `connector_tags.address`,
+`pi_web_api_query` with the connector `period` under the pipeline `max_count` (`api_timeout` capped
+at the frequency), and `connection` with base URL and the secret reference only. An unknown
+`tag_id` is left out and notified (`ORCHESTRATOR_PI_TAG_NOT_FOUND_DURING_SCHEDULE_CONFIGURATION`).
+A legacy pipeline (inline `webid`) gets the pre-change input, without `connection`.
+
+#### 1.9.2 PI output tags through the tag database
+
+`predictions_batch` `pi_web_api_output_config` given as `tag_id` lists → `{tag_name: WebID}` from
+`Write` / `Prediction Confidence` tags; a `Read` tag given as a confidence tag is left out and
+notified (`ORCHESTRATOR_PI_TAG_WRONG_CATEGORY_DURING_SCHEDULE_CONFIGURATION`). A legacy
+`{tag_name: webid}` mapping passes through unchanged.
+
+#### 1.9.3 Mixed PI connectors hold the schedule
+
+A pipeline updated to read tags of two PI connectors → its existing scouter schedule is neither
+updated nor deleted, and the conflict is notified
+(`ORCHESTRATOR_PI_MIXED_CONNECTORS_DURING_SCHEDULE_CONFIGURATION`).
+
+#### 1.9.4 OPC slot unchanged by PI connectors
+
+The OPC slot v2 written to Redis for an OPC scouter by `tag_id` is the expected one with the PI
+connectors and tags present, and identical after removing them.
+
 ---
 
 ## 2. Alerts workflow