SIENTIAPDE-1973

SIENTIAPDE-1973: Refactor unit tests for improved readability, primarily by splitting chained assertions into individual statements and minor fixture import adjustments.

.env.example

@@ -21,7 +21,6 @@ NATS_WRITE_SUBJECT="opc.write"
 NATS_QUEUE="gateway-writers"
 GRPC_ENABLED="false"
 GRPC_PORT="50051"
-CERT_BUCKET="sientia-dataops"
 CERT_CACHE_DIR="/tmp/opc-certs"
 WRITE_TIMESTAMP="true"
 WRITE_QUALITY="true"

.github/workflows/deploy.yml

@@ -1,10 +1,15 @@
 name: Deploy DataOps Worker
 
 on:
-  pull_request:
-    types: [closed]
-    branches: [main, 'release/**', 'feature/**']
   workflow_dispatch:
+    inputs:
+      chart_ref:
+        description: >
+          sientia-helm-chart branch to bump GITHUB_BRANCH on AND deploy — the literal
+          deploy target. Validated via git ls-remote; a branch that does not exist in
+          sientia-helm-chart is a hard error, not a silent fallback to main.
+        required: true
+        type: string
 
 permissions:
   contents: write
@@ -14,10 +19,10 @@ permissions:
 jobs:
   deploy:
     name: Deploy
-    if: github.event_name == 'workflow_dispatch' || github.event.pull_request.merged == true
     uses: Aignosi/github_workflow_templates/.github/workflows/reusable-deploy-dataops.yml@main
     with:
-      branch_name: ${{ github.event.pull_request.head.ref }}
+      branch_name: ${{ github.ref_name }}
+      chart_ref: ${{ inputs.chart_ref }}
       project_name: 'ingestor'
       chart_name: 'opc-ingestor'
       env_list_path: '[".env"]'

.github/workflows/quality-gate.yml

@@ -5,7 +5,6 @@ on:
     branches:
       - main
       - 'release/**'
-      - 'feature/**'
   pull_request:
     branches:
       - main

.github/workflows/require-chart-ref-label.yml

@@ -1,16 +0,0 @@
-name: Require chart-ref label
-
-on:
-  pull_request:
-    types: [opened, synchronize, reopened, labeled, unlabeled]
-    branches:
-      - main
-      - 'release/**'
-      - 'feature/**'
-
-permissions:
-  pull-requests: write
-
-jobs:
-  require-chart-ref-label:
-    uses: Aignosi/github_workflow_templates/.github/workflows/require-chart-ref-label.yml@main

.github/workflows/version-bump.yml

@@ -0,0 +1,17 @@
+name: Version Bump
+
+permissions:
+  contents: write
+
+on:
+  pull_request:
+    types: [closed]
+    branches: [main, 'release/**', 'feature/**']
+
+jobs:
+  bump:
+    if: github.event.pull_request.merged == true
+    uses: Aignosi/github_workflow_templates/.github/workflows/reusable-version-bump.yml@main
+    with:
+      update_version_in: '["pyproject"]'
+    secrets: inherit

CLAUDE.md

@@ -41,7 +41,6 @@ app.py (asyncio loop, signals, Prometheus server)
        -> IngestorManager (managers/ingestor_manager.py) — central coordinator
             -> ResourceManager (managers/resource_manager.py) — Redis slot leases, heartbeats, active-ingestor registry
             -> ServerReader (managers/server_reader.py), one per OPC server — SubscriptionEngine over the shared OpcConnectionPool
-               (READ_ENGINE=legacy: OpcManager (managers/opc_manager.py), the rc3 reader kept as plan B until the end of SIENTIAPDE-1974)
             -> OpcConnectionPool (connection/pool.py) + CertProvider (connection/certs.py) — pooled connections, cert from MinIO
             -> DataManager (managers/data_manager.py) — MongoDB persistence + optional Kafka publish
 ```
@@ -58,7 +57,9 @@ app.py (asyncio loop, signals, Prometheus server)
 
 Any change to slot/lease counts should be traced through `manage_leases`/`manage_no_slots` — that's where the rebalancing math lives, not in `ResourceManager` itself (which just does Redis primitives).
 
-**ServerReader** (SIENTIAPDE-1974, default `READ_ENGINE=v2`; the legacy `OpcManager` stays as plan B with `READ_ENGINE=legacy` — slot v1 only, certificate files from the Gitea clone — until it is unused and the tests pass the same without it) reads one OPC server through a `SubscriptionEngine`: the connection comes from the shared `OpcConnectionPool` (also used by NATS writes and gRPC), which fetches the platform certificate from MinIO via `CertProvider` and uses `SecurityPolicyBasic256Sha256` — no certificate files in the repo/image. Slots are parsed by `io/slots.py` (`parse_slot`), which accepts slot v2 (`names[]`) and converts the legacy v1 (`topics`). `datachange_notification` fans each value out to every `names` entry via `DataSink`. Subscribed connections are pinned in the pool so the idle sweep never drops them. Tracks `non_receive_count` to detect silently-dead subscriptions (`check_cycles` / `check_opc_listenning`); a lost server is shut down (pool connection evicted) and re-initialized on the next loop.
+**ServerReader** (SIENTIAPDE-1974) reads one OPC server through a `SubscriptionEngine`: the connection comes from the shared `OpcConnectionPool` (also used by NATS writes and gRPC). Slots are slot v2 only, parsed by `io/slots.py` (`parse_slot`): each server carries `security` (`mode`, `policy`) and `certificate` (certificate_id, application_uri, MinIO `bucket`/`object_name` of cert and private_key, from `opc_certificates` via the Orchestrator; `None` = connect without security, D5). The pool applies security only with a certificate (`connection/security.py`: `is_secured`, `apply_security` maps mode/policy to asyncua, unknown → error), and `CertProvider` downloads each certificate into `<CERT_CACHE_DIR>/<certificate_id>/` (cache, lock and `invalidate` per certificate) — no certificate files in the repo/image, no default certificate. A change of `security`/`certificate` in the slot re-initializes the reader (pool connection evicted). `datachange_notification` fans each value out to every `names` entry via `DataSink`. Subscribed connections are pinned in the pool so the idle sweep never drops them. Tracks `non_receive_count` to detect silently-dead subscriptions (`check_cycles` / `check_opc_listenning`); a lost server is shut down (pool connection evicted) and re-initialized on the next loop.
+
+**Writes** (SIENTIAPDE-1975): `NatsWriteConsumer` → `WriteEngine` → pool, then `DataSink.update_write_status`. Commands (C2) carry only `server_id`; `ServerConfigResolver` (`connection/servers.py`) resolves the connection: live pooled connection → `connectors` aggregation with a `$lookup` on `opc_certificates`, in the slot format. The certificate rule (active + client, `connection/certificates.py`) mirrors the Orchestrator's `tag_database.resolve_certificate` — keep both in sync (SIENTIAPDE-1974 D1; the Mongo fallback goes away when writes are routed to the pod that owns the connection). `write_status` keeps the worst outcome (`ok` < `error` < `session_bad`); the gateway is the only one that updates it (the Laborious inserts the row as `pending` and never records OPC outcomes). The NATS first connect is bounded (it runs on the main loop); reconnection after it is unbounded.
 
 **Metrics** (`ingestor/metrics.py`): all Prometheus metrics defined here, always labeled by `pod_id`. Emitted via `SientiaMonitoring.emit_metric(...)` (from `sientia_do`), not the prometheus client directly — new metrics should follow that pattern for consistency with logging/notifications.
 

README.md

@@ -56,20 +56,52 @@ Gateway for on-demand and command-driven interactions with OPC UA servers:
 
 - **NATS write consumer**: consumes write commands from `opc.write` (see
   `NATS_WRITE_SUBJECT`) and executes them against the target OPC server
-  (`ingestor/adapters/nats_consumer.py`, `ingestor/engines/write.py`).
+  (`ingestor/adapters/nats_consumer.py`, `ingestor/engines/write.py`). See
+  [Write commands](#write-commands-c2-and-write_status-c3) below.
 - **On-demand gRPC `ReadTag`**: a gRPC server (`ingestor/adapters/grpc_server.py`,
   `ingestor/proto/opc_gateway.proto`) for synchronous, single-tag reads.
 - **Scheduled active reads**: a polling read engine (`ingestor/engines/active_read.py`)
   for tags that need periodic reads outside the subscription pipeline;
   orchestrator-side wiring lands in SIENTIAPDE-1974.
-- **Certificate management**: client certs for secure OPC connections are
-  fetched from MinIO + a Mongo `platform_certificates` collection
+- **Certificate management**: each OPC connector has its own client certificate
+  (`opc_certificates`), named in the slot and downloaded from MinIO
   (`ingestor/connection/certs.py`) rather than shipped in the image.
 
 See `docs/deploy-notes.md` for the deployment-facing details (env vars,
 health probe, NetworkPolicy, cert cache) and the `# OPC Gateway` block in
 `.env.example` for the full list of new environment variables.
 
+### Write commands (C2) and `write_status` (C3)
+
+The Laborious publishes one command per tag (SIENTIAPDE-1975):
+
+```json
+{"server_id": "1", "addr": "ns=2;s=Pred", "value": 1.25, "data_type": "float",
+ "model_id": "2", "model_name": "sin-approx", "schema": "sientia_data",
+ "table_name": "predictions", "timestamp": "2026-10-13T12:00:00-03:00",
+ "write_timestamp": false, "write_quality": false}
+```
+
+- **Server connection**: the command carries only `server_id`. The connection is
+  resolved by `ServerConfigResolver` (`ingestor/connection/servers.py`): the live
+  pooled connection opened by the reads, else the `connectors` document (`type: opc_ua`,
+  `status: Active`) joined with `opc_certificates`, with the orchestrator's rule for the
+  read slots (`ingestor/connection/certificates.py`), in the slot format (`url`,
+  `security`, `certificate`). Lookups are cached for 60 s. An unknown or incomplete
+  server is a failed write (`server_not_found`).
+- **Outcome**: `UPDATE ... SET write_status` on the prediction row (by `timestamp` +
+  `model_id`), keeping the worst outcome: `ok` < `error` < `session_bad`, and
+  `pending`/NULL are replaced by any answer, so the order of the answers does not
+  matter. The gateway is the only one that updates the column: the Laborious inserts the
+  row as `pending` (merged with its PI Web API outcome) and never records OPC outcomes.
+  The UPDATE never raises; its outcome (`updated`, `kept`, `row_not_found`,
+  `invalid`, `error`) is logged and counted in `opc_write_status_update_total`.
+  The reason of a failed write is logged by `WriteEngine`.
+- **NATS connection**: the first connection gives up after 5 s (it runs on the main
+  loop: nats-py's defaults would block it for minutes while NATS is down); once
+  connected, it reconnects forever. A connection closed for good is started again
+  by `GatewayRuntime.idle_sweep`.
+
 ## 📋 Prerequisites
 
 - Python 3.11+
@@ -195,48 +227,42 @@ The feeder creates sample OPC tag configurations in Redis that the ingestor can
 
 ## 🔐 OPC Client Certificate
 
-OPC UA servers that require secure connections validate the gateway with a single,
-shared platform X.509 client certificate. Every OPC connection — subscription reads,
-NATS writes and gRPC reads — uses it with `Basic256Sha256` (SIENTIAPDE-1974).
+Each OPC connector has its own X.509 client certificate (SIENTIAPDE-1974): the
+connector's `security.client_certificate_id` points to an `opc_certificates` document
+(`kind: client`, `status: active`). The Orchestrator joins them and writes, for each
+server of the slot, its `security` (`mode`, `policy`) and its `certificate`:
+
+```json
+"certificate": {
+  "certificate_id": 1,
+  "application_uri": "urn:sientia:opc-ingestor",
+  "cert": {"bucket": "opc-certificates", "object_name": "client/sientia-platform/opc-ingestor-cert.pem"},
+  "private_key": {"bucket": "opc-certificates", "object_name": "client/sientia-platform/opc-ingestor-key.pem"}
+}
+```
 
 The certificate is **never** read from the repository or the image. On connect, the
-`CertProvider` (`ingestor/connection/certs.py`):
-1. reads the single document of the Mongo collection `platform_certificates`:
-   `{"client_cert": {"minio_object_name": "..."}, "private_key": {"minio_object_name": "..."}}`;
-2. downloads both objects from the MinIO bucket `CERT_BUCKET` (default
-   `streamlit-connectors`) into `CERT_CACHE_DIR` (default `/tmp/opc-certs`).
+`CertProvider` (`ingestor/connection/certs.py`) downloads both objects from MinIO into
+`CERT_CACHE_DIR/<certificate_id>/` (default `/tmp/opc-certs`), and caches them per
+certificate. When the slot names other objects for the same certificate (it was
+replaced), the server config changes, the connection is re-opened and the files are
+downloaded again. A failed secured connection also forces a new download.
 
-Requirements for a server to accept the connection:
-- the server's slot config has `use_security: true` (a slot v1 server with a
-  `cert_path` is treated the same way; the path itself is ignored);
-- the certificate's SAN URI equals the server's `server_uri` in the slot — it is sent
-  as the client `application_uri` (KEPServer rejects a mismatch with
-  `BadCertificateUriInvalid`);
-- the server exposes a `Basic256Sha256` endpoint and trusts the certificate (e.g.
-  KEPServer: *OPC UA Configuration Manager → Trusted Clients → Trust*, then restart
-  its runtime).
+- **Security only with a certificate.** With a `certificate`, the connection uses the
+  slot's `security.mode` (`Sign` / `SignAndEncrypt`) and `security.policy`
+  (`Basic256Sha256`, `Aes128Sha256RsaOaep`, `Aes256Sha256RsaPss`, `Basic256`,
+  `Basic128Rsa15`); an unknown one fails the connection. Without a certificate
+  (`null`), or with `mode: None`, it connects without security, and a server that
+  requires a certificate refuses the connection (the error shows in the log and the
+  connection metrics). There is no default certificate.
+- The certificate's `application_uri` (its SAN URI) is sent as the client application
+  URI (KEPServer rejects a mismatch with `BadCertificateUriInvalid`).
+- The server must trust the certificate (e.g. KEPServer: *OPC UA Configuration Manager
+  → Trusted Clients → Trust*, then restart its runtime).
 
-MinIO is only contacted when a certificate is first needed, so a MinIO outage only
-affects secured servers, which are retried on the next main-loop iteration.
-
-### Plan B: legacy read engine (`READ_ENGINE=legacy`)
-
-Until the end of SIENTIAPDE-1974 the legacy reader is kept for validation and as a
-fallback. `READ_ENGINE` selects the read path (default `v2`; an invalid value falls
-back to `v2` with a warning):
-
-| `READ_ENGINE` | Reader | Slot | Certificate |
-|---|---|---|---|
-| `v2` (default) | `ServerReader` + `SubscriptionEngine` over the shared pool | v1 or v2 | MinIO + Mongo (`CertProvider`), `Basic256Sha256` |
-| `legacy` | the rc3 `OpcManager`, unchanged, with its own connection | **v1 only** (a v2 slot is rejected and notified) | files from the repository the pod clones (`cert_path` / `private_key_path` in the slot), `Basic256` |
-
-Writes and gRPC always use the pool (MinIO certificate), whatever the read engine.
-The legacy engine needs the certificate files in the Gitea branch the pod clones;
-`scripts/generate-and-push-opc-cert.sh` generates them and pushes them to
-`certs/` (`GITEA_PASSWORD='<senha>' BRANCH=<branch> ./scripts/generate-and-push-opc-cert.sh`).
-Each version deploy creates a new Gitea branch without `certs/`, so they have to be
-copied again for the legacy engine. Remove the legacy engine (OpcManager, its tests
-and the script) once it is unused and the test suite passes the same without it.
+MinIO is only contacted when a certificate is first needed (one client per bucket), so
+a MinIO outage only affects secured servers, which are retried on the next main-loop
+iteration.
 
 ## 🧪 Testing
 
@@ -290,6 +316,9 @@ The OPC Ingestor exposes comprehensive Prometheus metrics for monitoring:
 ### OPC Gateway Metrics
 - `opc_write_total`: Active write attempts (NATS-driven), by result
 - `opc_write_latency_seconds`: Active write latency
+- `opc_write_status_by_model`: Last write outcome per model (1 = ok, 0 = failed; label `status`)
+- `opc_write_status_update_total`: `write_status` UPDATEs, by result
+  (`updated`, `kept`, `row_not_found`, `invalid`, `error`)
 - `opc_active_read_total`: Scheduled/on-demand read attempts, by result
 - `opc_error_by_type_total`: OPC errors by type (connection, conversion, session, ...)
 
@@ -314,7 +343,7 @@ The OPC Ingestor exposes comprehensive Prometheus metrics for monitoring:
 | `HTTP_METRICS_PORT` | Prometheus metrics port | `9090` | No |
 
 See the `# OPC Gateway` block in `.env.example` (`NATS_URL`, `GRPC_ENABLED`,
-`CERT_BUCKET`, `WRITE_TIMESTAMP`, ...) and `docs/deploy-notes.md` for the
+`CERT_CACHE_DIR`, `WRITE_TIMESTAMP`, ...) and `docs/deploy-notes.md` for the
 gateway-specific environment variables.
 
 ### OPC Server Configuration
@@ -322,8 +351,8 @@ gateway-specific environment variables.
 OPC servers are configured by the Orchestrator through Redis, one key per slot
 (`slot:opc_tags:<n>`). The gateway reads the **slot v2** structure (contract C1):
 one entry per address, with one `names` item per collection / tag name / model, so
-the same address can feed several models under different names. For secured
-connections set `use_security: true` (the certificate comes from MinIO, see
+the same address can feed several models under different names. Each server carries
+its `security` and its `certificate` (`null` = no security, see
 [OPC Client Certificate](#-opc-client-certificate)):
 
 ```json
@@ -331,11 +360,16 @@ connections set `use_security: true` (the certificate comes from MinIO, see
   "slot:opc_tags:1": {
     "server1": {
       "name": "server1",
-      "server_id": "1",
+      "server_id": "kepserver",
       "url": "opc.tcp://localhost:4841",
-      "server_uri": "urn:sientia:opc-ingestor",
-      "use_security": true,
       "subscription_period_ms": 7500,
+      "security": {"mode": "SignAndEncrypt", "policy": "Basic256Sha256"},
+      "certificate": {
+        "certificate_id": 1,
+        "application_uri": "urn:sientia:opc-ingestor",
+        "cert": {"bucket": "opc-certificates", "object_name": "client/sientia-platform/opc-ingestor-cert.pem"},
+        "private_key": {"bucket": "opc-certificates", "object_name": "client/sientia-platform/opc-ingestor-key.pem"}
+      },
       "tags": {
         "ns=2;i=2": {
           "frequency": "15000",
@@ -351,11 +385,10 @@ connections set `use_security: true` (the certificate comes from MinIO, see
 }
 ```
 
-The legacy **slot v1** (each tag with `tag_name` + `topics`, the server with
-`cert_path`) is still accepted and converted on read (`ingestor/io/slots.py`): one
-`names` item per topic, and `use_security` is true when `cert_path` is set (the path
-itself is ignored). This lets the gateway and the Orchestrator be deployed in any
-order. Documents written from a v1 slot have no `model_id`, exactly as before.
+Only the slot v2 is accepted (`ingestor/io/slots.py`): a slot without `security`, a
+tag without `names`, or a name without `collection`, `tag_name` or `model_id` is
+rejected and notified (`OPC_SLOT_CONFIG_ERROR_<slot>`), and the last good config keeps
+being read. The gateway and the Orchestrator of SIENTIAPDE-1974 are deployed together.
 
 ## 🔧 Development
 
@@ -366,14 +399,13 @@ sientia-dataops-opc-ingestor/
 │   ├── managers/            # Component managers
 │   │   ├── data_manager.py # Data persistence and export
 │   │   ├── ingestor_manager.py # Main coordination
-│   │   ├── opc_manager.py  # Legacy reader, plan B (READ_ENGINE=legacy)
 │   │   ├── resource_manager.py # Resource coordination
 │   │   └── server_reader.py # OPC UA server reads (SubscriptionEngine + pool)
 │   ├── app.py              # Main application entry point
 │   ├── ingestor.py         # Core ingestor logic
 │   └── metrics.py          # Prometheus metrics definitions
 ├── simulator/               # OPC simulation and testing tools
-├── scripts/                 # Ops scripts (e2e seed; legacy OPC cert generation)
+├── scripts/                 # Ops scripts (e2e seed)
 ├── tests/                   # Test suite
 └── requirements.txt         # Python dependencies
 ```

docs/planning/SIENTIAPDE-1972-banco-de-tags-e-certificados.md

@@ -0,0 +1,354 @@
+# SIENTIAPDE-1972 — Estado atual, pendências e plano: banco de tags + servidores OPC ↔ certificados (MinIO)
+
+> Levantamento em 28/10/2026, a partir do Jira (1972 e subtasks, 1568/1569/159x), do código do
+> `sientia-dataops-opc-ingestor` (`release/SIENTIAPDE-1973` @ `d68e842`, 2.0.0-rc4), do
+> `sientia-dataops-orchestrator_temporal` (`release/SIENTIAPDE-1974`), do Streamlit
+> (`sientia-projects-templates` `rc/SIENTIAPDE-1588`), do `sientia-dataops-database-migrations`
+> e do design `aig-docs/07 - Projects & R&D/sientia/opc-gateway/design.md`.
+
+---
+
+## 1. Resumo executivo
+
+- **O gateway funciona no dev** (leitura v2, escrita via NATS, `write_status`) e as subtasks 1973, 1974,
+  1975 e 1985 estão **Concluídas**. Mas **nada disso está na `main`/produção**: o PR #48
+  (`release/SIENTIAPDE-1973` → `main`) está aberto, e a `release/SIENTIAPDE-1974` do orchestrator
+  nem PR para a `main` tem (5 commits à frente, 30 atrás).
+- **Risco imediato**: a `main` do Laborious (1.6.0) já escreve OPC **só via NATS**. Se ela for para
+  um ambiente com o ingestor 1.4.x, toda escrita OPC fica em `pending`.
+- **O banco de tags não existe em nenhum ambiente.** `connectors`/`connector_tags` só existiram no dev
+  durante o teste A2 da 1974 e foram apagadas. O orchestrator e o gateway já sabem ler o banco de tags
+  (por `tag_id`), mas sem dado, tudo roda pelo caminho legado (`opc-servers` + `server_id/tag_address`).
+- **Certificado hoje é um só para a plataforma** (`platform_certificates`, `find_one({})`). Não existe
+  correlação servidor OPC ↔ certificado no Mongo. O `server_cert_path` (cert do servidor) é ignorado
+  com aviso. Bucket em uso: `streamlit-connectors` (o design pede `sientia-dataops`).
+- **O CRUD (Streamlit 1568/1569) está On Hold** desde 04/2026 para migrar ao Angular; não há card
+  Angular aberto para isso (só a idea `SIENTIAPDI-84`, em parking lot).
+
+O que falta para fechar o 1972 é pouco (merges + 2202 + 2270). O que vocês tinham pensado — **banco de
+tags + servidores OPC estruturados com certificados no MinIO** — é praticamente uma **fase 2 inteira**
+que não tem card hoje. Este documento propõe o modelo e quebra em cards.
+
+---
+
+## 2. Estado das subtasks do 1972
+
+| Card | Status | O que entregou / o que falta |
+|---|---|---|
+| 1973 Gateway | ✅ Concluído (02/10) | Escrita NATS, gRPC ReadTag, pool, CertProvider. PRs empilhados #31–#33 fechados, código na `release/SIENTIAPDE-1973` |
+| 1974 Orchestrator + leitura v2 | ✅ Concluído (09/10) | Gateway #41 (leitura v2, cert do MinIO), #42 (KeyError) mergeados na release. Orchestrator #45 (slot v2, `names[]`, banco de tags opcional por `tag_id`) mergeado na **`release/SIENTIAPDE-1974`**, não na `main`. G6/G7 (A6/A7) não serão feitos: não existe leitura ativa programada; leitura ativa é só on-demand via gRPC, sem persistir (G-AR) |
+| 1975 Laborious NATS + `write_status` | ✅ Concluído (23/10) | migrations #37 (V16), gateway #43/#47, laborious #5 (main 1.6.0). Validado até o ProSYS (M0–M4) |
+| 1985 NATS no cluster | ✅ Concluído (30/09) | NATS 2.14.6 via helm. `#108` (release/1.0.0) e `#126` (development) — conferir se ambos já foram mergeados |
+| **2202** timestamp/qualidade na escrita | 🔄 Em andamento (Eduardo) | Decidir onde fica a config (sugestão: por servidor). **Encaixa direto no modelo de `connectors` deste plano** |
+| **2270** mlops-bff `write_status` no Model Sustain | 🔄 Em andamento (Carla) | Depende de decisão do PO (falha de escrita deixa o modelo CRITICAL ou não) e da V16 aplicada antes do deploy do BFF |
+
+---
+
+## 3. Pendências (consolidado dos comentários dos cards)
+
+### 3.1 Promoção para produção (bloqueia tudo o mais)
+
+| # | Pendência | Onde | Situação |
+|---|---|---|---|
+| P1 | PR #48 `release/SIENTIAPDE-1973` → `main` (gateway 2.x) | opc-ingestor | Aberto. Combinar número de versão (bump pode não gerar 2.0.0) |
+| P2 | PR `release/SIENTIAPDE-1974` → `main` | orchestrator | **Não existe.** Release está 30 commits atrás da `main` (0.8.2), com conflito em 4 arquivos (`orchestrator_functions.py`, teste dele, `README.md`, `pyproject.toml`). **Desde 19/10 o dev roda o orchestrator 0.8.1 (`main`, sem slot v2)**: o deploy do PR #49 fez o bump de `GITHUB_BRANCH` 0.7.0-rc1 → 0.8.1 no chart (`development`, commit `07ffc99`) |
+| P3 | Card do helm (H1–H4 do 1975): migrations com V16, Grafana sem 12/13 e com `write_status`, `docs/nats-broker.md` §11. NATS + envs já estão em todo profile com `opc-ingestor` ligado (desligado só em galvani, quartz-homologacao, gcp-hom, que não têm `opc-ingestor` nem `laborious-worker`) | sientia-helm-chart | **Card não criado** |
+| P4 | Ordem de deploy por ambiente: migrations V16 → gateway 2.x → laborious 1.6.0 | todos | Documentar no runbook de release |
+| P5 | Correção da `main` do gateway (SQLAlchemy 2.1 + senha do Mongo no log), branch `fix/sqlalchemy-21-mongo-uri-log` | opc-ingestor | Fica obsoleta se o #48 entrar; senão, abrir PR de hotfix |
+| P6 | Servidores OPC de produção aceitarem `Basic256Sha256` e confiarem no certificado da plataforma (SAN URI = `server_uri`) | infra/cliente | Pré-requisito do deploy da leitura v2 em cada cliente |
+| P7 | Certificado da plataforma cadastrado (`platform_certificates` + MinIO) em cada ambiente | por ambiente | Hoje só no dev, cadastrado à mão |
+
+### 3.2 Gateway (código)
+
+| # | Pendência | Referência |
+|---|---|---|
+| G-AR | **Leitura ativa = unitária e atômica: lê e devolve o valor, nada mais** (decisão 28/10). Sem leitura programada e sem persistência no gRPC. Hoje já se comporta assim (`slot_lookup=None` → `persisted=false`), falta remover o código morto: `scheduled_tick`/`_tick_tag`/`_should_read`/`_ticks` e o parâmetro `names` + `sink.publish_reading` de `read_now`/`_build_result` (`engines/active_read.py`); `slot_lookup` de `ReadTagServicer`/`GrpcServer` (`adapters/grpc_server.py`); `ponytail` de `gateway_runtime.py:10,93`; campo `persisted` do `ReadTagResponse` (marcar `reserved 5` no proto); `active_read` do slot v2 no orchestrator e no `parse_slot`; testes correspondentes | decisão 28/10 |
+| G-DOC | Atualizar design §6 C4 e §14 ("Reads, persists to the pipeline, and returns the value") e os requisitos G3/G4/A6/A7 do 1974 para refletir a decisão | aig-docs `opc-gateway/design.md` |
+| G-PB | Remover plano B `READ_ENGINE=legacy` (`OpcManager`, `scripts/generate-and-push-opc-cert.sh`) | critério do 1974; pode sair no #48 ou logo depois |
+| G-RC | Reconectar ao receber `status_change_notification` (KEPServer derruba sessão a cada 2 h → 2,5–3 min sem dado) | achado do 1974 |
+| G-SC | Suporte a `server_cert_path` (validar cert do servidor) — hoje só aviso | `ingestor/managers/ingestor_manager.py:708` — resolvido pela Fase 2 abaixo |
+
+### 3.3 Fora do gateway
+
+| # | Pendência | Situação |
+|---|---|---|
+| C1 | Consumidores dos códigos 12/13/14: `sientia-dataops-report`, `aignosi-scripts` (`sientia_status_report.sh`) | **Sem card** (só o BFF tem, 2270) |
+| C2 | Laborious: remover `write_opc_data` antigo quando não houver execução anterior ao patch | sem card |
+| C3 | Limpeza no dev: tabela `sientia_data.e2e_predictions`, pipeline `laborious-test-1975-write`, branches `test-SIENTIAPDE-197x-*` no Gitea | pendente de ok |
+| C4 | Melhoria de baixa prioridade do scouter (timeout Redis, retries, catch-up) | sugerido, sem card |
+| C5 | Usuário `aignosi` da VM ProSYS + senha em texto no README do `sientia-iac` | sem card |
+
+### 3.4 Banco de tags e certificados (bloqueios B1–B6 do 1974 — **nenhum resolvido**)
+
+| # | Bloqueio | Situação hoje |
+|---|---|---|
+| B1 | `connectors`/`connector_tags` não existem em nenhum ambiente | **escopo do orchestrator** (decisão 28/10) |
+| B2 | Pipelines não gravam `tag_id` em `read_tags`/`write_tags` | **escopo do orchestrator** |
+| B3 | ~~Onde fica `active_read`~~ | não se aplica: leitura ativa é só on-demand, sem persistir |
+| B4 | Certificado por ambiente + bucket (`sientia-dataops` do design × `streamlit-connectors` do chart/D7) e correlação servidor ↔ certificado | **escopo do orchestrator**; bucket indefinido |
+| B5 | Tela de certificados: 1599 entregou no **Streamlit** (rc, não em produção) | não existe no Angular |
+| B6 | ~~gRPC persist / timestamp nulo~~ | não se aplica: gRPC não persiste; `source_timestamp` vazio só volta vazio na resposta |
+| — | Renomear tag em `connector_tags` não reimplanta o scouter (só `pipelines.updated_at`) | limitação conhecida |
+| — | Restos do 1569: 1598 (`tag_id` sequencial sem corrida), 1600 (`metadata`), 1602 (bucket por servidor), 1603 (Server URI), 1604, 1605 (CSV) | On Hold, "passíveis de cancelamento" |
+
+---
+
+## 4. Diagnóstico do modelo atual (Mongo + MinIO)
+
+Hoje coexistem **dois registros de servidor OPC** e **um certificado global**:
+
+```
+opc-servers            { id, name, url, uri, cert_path, private_key_path, server_cert_path, ... }   ← legado, usado em todos os ambientes
+connectors (opc_ua)    { connector_id, type, name, url, server_uri, use_security, subscription_period_ms, status, description }
+connector_tags         { tag_id, connector_id, address, tag_name, category, data_type, unit, status, description, <custom flat keys> }
+platform_certificates  { client_cert: {minio_object_name, sha256_hash, expiration_date, ...}, private_key: {...} }   ← 1 documento só
+MinIO                  <bucket>/platform/certificates/client_cert_<arquivo>, private_key_<arquivo>
+```
+
+Problemas concretos:
+
+1. **Sem correlação servidor ↔ certificado.** `CertProvider._download_all` faz `find_one('platform_certificates', {})`
+   (`ingestor/connection/certs.py:47`). Um cliente com dois servidores que exigem certificados diferentes
+   (ou um servidor que exija validação do cert do servidor) não é atendível.
+2. **`server_uri` é dado do certificado, não do servidor.** O `application_uri` do cliente precisa ser
+   igual à SAN URI do certificado da plataforma (comentário 06/10 do 1974). Hoje ele é digitado por
+   conector (`connectors.server_uri` / `opc-servers.uri`) — e o 1603 queria "chumbar" no backend.
+   Erro de digitação = conexão segura recusada sem pista clara.
+3. **`server_cert_path` ignorado.** O cert do servidor não tem lugar no MinIO/Mongo.
+4. **Dois espaços de id.** `ServerConfigResolver` procura `connectors.connector_id` e depois `opc-servers.id`
+   com o mesmo `server_id` (`ingestor/connection/servers.py:105-115`); o orchestrator trata "mesmos campos"
+   como mesmo servidor. Se os ids divergirem, um `server_id` aponta para servidores diferentes conforme a fonte.
+5. **Ids por `$max + 1`** (Streamlit `tag_monitor.py:141`) — corrida em criação simultânea/import em lote.
+   O repo de migrations já tem o padrão certo (`model_id_counters` com `$inc` atômico).
+6. **Sem schema/índices.** Nada impede `(connector_id, tag_name)` duplicado nem conector sem `url`.
+7. **Sem visibilidade de expiração** do certificado (o Streamlit calcula `expiration_date`, ninguém alerta).
+
+---
+
+## 5. Modelo proposto
+
+Princípio: **`connectors` vira o registro canônico de servidores OPC**, `opc-servers` vira legado
+(somente leitura, depois removido); **certificados viram entidade própria** referenciada pelo conector.
+
+### 5.1 `opc_certificates` (nova; substitui `platform_certificates`)
+
+```js
+{
+  certificate_id: 1,                         // int, counter atômico
+  kind: "client",                            // client | server | ca
+  name: "sientia-platform-2026",
+  is_default: true,                          // no máximo 1 client default (índice parcial único)
+  status: "active",                          // active | retired
+  application_uri: "urn:sientia:opc-ingestor",  // SAN URI extraída do cert no upload (kind=client)
+  subject: "CN=sientia-opc-ingestor,...",
+  thumbprint_sha1: "…",                      // o que aparece no console do KEPServer/servidor
+  not_before: ISODate(), not_after: ISODate(),
+  cert:        { bucket, object_name, sha256, size, content_type, original_filename },
+  private_key: { bucket, object_name, sha256, size },   // só kind=client
+  created_at, created_by, updated_at
+}
+```
+
+- Índices: `certificate_id` único; `{is_default:1}` único parcial (`kind:"client", is_default:true`);
+  `not_after` (para alerta).
+- `application_uri`, `thumbprint`, `not_after` **extraídos do arquivo no upload**, nunca digitados.
+
+### 5.2 `connectors` (opc_ua) — campos novos
+
+```js
+{
+  connector_id: 1,                           // = opc-servers.id no backfill (mantém server_id estável)
+  type: "opc_ua", name: "kepserver", url: "opc.tcp://…", status: "Active",
+  subscription_period_ms: 1000,
+  security: {
+    mode: "SignAndEncrypt",                  // None | Sign | SignAndEncrypt
+    policy: "Basic256Sha256",
+    client_certificate_id: null,             // null → certificado default da plataforma
+    server_certificate_id: 7                 // opcional: valida o cert do servidor (resolve G-SC)
+  },
+  write_options: { write_timestamp: true, write_quality: true },   // SIENTIAPDE-2202 (Kepware: false/false)
+  legacy_opc_server_id: "1",                 // rastreio do backfill
+  created_at, updated_at
+}
+```
+
+- `use_security` vira derivado (`security.mode != "None"`); manter o campo booleano durante a transição
+  para não quebrar orchestrator/gateway atuais.
+- `server_uri` **sai do conector**: o `application_uri` vem do certificado cliente resolvido.
+  (Manter leitura do campo antigo como fallback na transição.)
+- Índices: `connector_id` único; `{type, name}` único.
+
+### 5.3 `connector_tags` — ajustes
+
+```js
+{
+  tag_id: 9001, connector_id: 1, address: "ns=2;s=…", tag_name: "TargetSin",
+  category: "Read",                          // Read | Write | Prediction Confidence
+  data_type: "Double", unit: "", status: "Active", description: "",
+  metadata: {},                              // objeto aninhado (decisão do design §14; fecha a dúvida do 1600)
+  created_at, updated_at
+}
+```
+
+- Índices: `tag_id` único; `{connector_id, tag_name}` único; `{connector_id, address}` (não único —
+  multi-nome é permitido); `updated_at` (orchestrator detectar mudança).
+
+### 5.4 `connector_id_counters` (padrão `model_id_counters`)
+
+Documentos `{_id: "connectors"|"connector_tags"|"opc_certificates", seq}`, semeados com o `max` atual,
+incremento por `find_one_and_update($inc)`. Resolve o 1598 sem `_get_next_tag_id_batch` com corrida
+(batch = `$inc: N` e usa a faixa).
+
+### 5.5 MinIO
+
+```
+<CERT_BUCKET>/opc-ua/certificates/<certificate_id>/cert.der
+<CERT_BUCKET>/opc-ua/certificates/<certificate_id>/private_key.pem
+```
+
+- Chave por `certificate_id` (não por nome de conector: renomear conector não move objeto; um cert serve vários conectores).
+- Bucket: **decidir** (ver D1). Policies: escrita só para o usuário do front/BFF; leitura para o gateway.
+- O `sha256` no Mongo é conferido pelo gateway no download (detecta objeto trocado/corrompido).
+
+### 5.6 Como o gateway resolve uma conexão (alvo)
+
+```
+server_id ──► connectors (connector_id) ──► security.client_certificate_id ?? default
+                     │                              │
+                     │                              └─► opc_certificates ─► MinIO (cert+key) ─► cache por certificate_id
+                     ├─► security.server_certificate_id ─► opc_certificates (kind=server) ─► MinIO
+                     └─► application_uri := opc_certificates.application_uri
+fallback (transição): opc-servers.id + certificado default
+```
+
+---
+
+## 6. Plano em fases
+
+### Fase 0 — Orchestrator primeiro, depois levar a 2.x para produção (curto prazo)
+
+Ordem: **orchestrator antes do gateway**. Sem o slot v2 na `main` do orchestrator, a 2.x em produção só
+roda o caminho legado (slot v1 convertido): sem multi-nome, sem banco de tags. O orchestrator também é dono
+de popular o banco de tags, gravar `tag_id` nas pipelines e correlacionar servidores ↔ certificados (Fase 1).
+
+1. Atualizar `release/SIENTIAPDE-1974` com a `main` (0.8.2), resolver os 4 conflitos, rodar testes + e2e,
+   abrir PR → `main`, deploy no dev e **refazer A1, A2 e A8** (o dev perdeu o slot v2 em 19/10; os testes
+   da 1975 de 19–23/10 rodaram com slot v1).
+2. Só então revisar/mergear **#48** (gateway → `main`); definir versão `2.0.0`. Decidir se o plano B sai aqui (G-PB).
+3. **Criar card do helm (P3)** com H1–H4.
+4. Runbook de deploy por ambiente (P4, P6, P7): checklist V16 → gateway → laborious; cert cadastrado; servidor
+   confia no cert; Kepware com `write_timestamp/quality=false`.
+5. Concluir **2202** já usando `connectors.write_options` (fallback env `WRITE_TIMESTAMP/WRITE_QUALITY`) — é
+   o primeiro campo do modelo novo e não depende do resto.
+6. **2270**: obter decisão do PO; criar cards de C1 (report, aignosi-scripts).
+7. Limpeza do dev (C3).
+
+**Saída:** 1972 pode ser fechado. Tudo abaixo vira **nova história/épico** (sugestão: "Banco de tags e
+certificados OPC estruturados", ligada à `SIENTIAPDI-84`).
+
+### Fase 1 — Modelo de dados + backfill — escopo do orchestrator (collections/índices via migrate-mongo no `sientia-dataops-database-migrations`)
+
+1. Migration `…-opc-tag-database.js`: cria `connectors`, `connector_tags`, `opc_certificates`,
+   `connector_id_counters` com `$jsonSchema` (validationLevel `moderate` na transição) e os índices de §5.
+2. Migration `…-backfill-tag-database.js` (idempotente, só insere o que não existe):
+   - `opc-servers` → `connectors` com **`connector_id = int(opc-servers.id)`**, `security.mode` derivado de
+     `cert_path`, `legacy_opc_server_id`.
+   - `platform_certificates` → `opc_certificates` (id 1, `is_default: true`), copiando os metadados MinIO
+     existentes (sem mover objeto ainda).
+   - `pipelines.read_tags[]`/`write_tags[]` (`server_id`, `tag_address`, `tag_name`) → `connector_tags`
+     (dedupe por `(connector_id, tag_name)`), e **grava `tag_id` de volta na pipeline** (resolve B1 + B2 de uma vez).
+   - semeia os counters com os máximos.
+3. Dry-run com dump do dev e de um cliente (relatório de conflitos: mesmo `tag_name` com endereços diferentes,
+   `server_id` sem servidor).
+
+### Fase 2 — Gateway (`sientia-dataops-opc-ingestor`)
+
+1. `CertProvider` → `CertificateStore`: `fetch(certificate_id | None)`, cache e lock **por `certificate_id`**,
+   verifica `sha256`; `None` = default (`is_default`), fallback `platform_certificates` durante a transição.
+2. `ServerConfigResolver` + slot: conexão passa a carregar `client_certificate_id`, `server_certificate_id`,
+   `application_uri`, `write_options`. Pool usa esses campos em `apply_security` (passa `server_certificate`).
+3. Validar no carregamento: SAN do cert == `application_uri`; cert não expirado → erro claro + notificação
+   (`OPC_CERTIFICATE_*`), em vez de falha opaca do asyncua.
+4. Métrica `opc_certificate_expiry_seconds{certificate_id}` + alerta (ex.: < 30 dias).
+5. `WriteEngine`: `write_timestamp/quality` = comando → `connectors.write_options` → env (fecha 2202 de vez).
+6. G-AR: limpar leitura ativa para leitura unitária sem persistência (pode ir junto com o #48 ou logo depois) + G-DOC.
+
+### Fase 3 — Orchestrator (`sientia-dataops-orchestrator_temporal`)
+
+1. Fonte de servidor: `connectors` primeiro, `opc-servers` fallback (hoje é o contrário: só usa `connectors`
+   quando a pipeline tem `tag_id`).
+2. Slot v2 por servidor: `connector_id`, `client_certificate_id`, `server_certificate_id`, `application_uri`
+   (no lugar de `use_security`/`server_uri` soltos — manter os antigos até o gateway novo estar em todo lugar).
+3. Remover `active_read` do slot v2 (sem leitura ativa programada).
+4. Reagir a mudança em `connector_tags.updated_at`/`connectors.updated_at` (reimplantar scouter/slot) — fecha a
+   limitação de renomear tag.
+
+### Fase 4 — CRUD no Angular (front `sientia` + BFF)
+
+1. Novo épico/história substituindo 1568/1569: telas de **Conectores**, **Tags** (com import CSV — escopo do 1605)
+   e **Certificados** (upload extrai SAN/thumbprint/validade; marca default; lista conectores que usam).
+2. BFF é quem escreve no Mongo/MinIO (front não fala com MinIO direto); usa os counters.
+3. Tela de pipeline passa a escolher tag por `tag_id` (pipelines novas já nascem com `tag_id`).
+4. Cancelar no Jira os restos Streamlit não iniciados (1600, 1604, 1605, 1603) com link para o novo card.
+
+### Fase 5 — Desligar o legado
+
+1. Orchestrator e gateway sem fallback para `opc-servers`/`platform_certificates`; `server_id/tag_address`
+   nas pipelines viram só leitura histórica.
+2. Remover `opc-servers`, `platform_certificates` (migration com backup) e o caminho Streamlit.
+3. `$jsonSchema` em `validationLevel: strict`.
+
+---
+
+## 7. Decisões necessárias
+
+| # | Decisão | Recomendação | Dono |
+|---|---|---|---|
+| D1 | Bucket dos certificados: `sientia-dataops` (design §7/§14, Streamlit rc) × `streamlit-connectors` (chart, D7 da 1974) | `sientia-dataops` (bucket por serviço, estratégia MinIO); migrar objetos na Fase 1 e trocar `CERT_BUCKET` no chart | Vitor / platform |
+| D2 | Cert por servidor ou único da plataforma | Modelo suporta os dois; **default único** + override por conector só quando necessário | Vitor |
+| D3 | `connector_id` = `opc-servers.id` no backfill | Sim — mantém `server_id` de slots, `write_tags` e métricas estáveis | Eduardo |
+| D6 | `metadata` aninhado × chaves soltas (1600) | Aninhado (`metadata: {}`), como no design §14 | PO |
+| D7 | `write_options` por servidor (2202) | Por servidor, padrão `true`; Kepware configurado `false` antes do deploy | Eduardo |
+| D8 | Quem escreve banco de tags em produção até o Angular existir | Backfill por migration + edição via migration/script revisado (sem Streamlit em produção) | Bruno / PO |
+| D9 | Plano B `READ_ENGINE=legacy` sai no #48? | Sim, se A1–A8 seguem ok no dev com a 2.0.0-rc4 | Eduardo |
+
+---
+
+## 8. Cards a criar / ajustar
+
+| Card | Tipo | Repo | Fase |
+|---|---|---|---|
+| Helm: V16, NATS+gateway 2.x nos profiles, `OPC_WRITE_MODE`, Grafana `write_status`, docs (H1–H4) | Subtask do 1972 | sientia-helm-chart | 0 |
+| PR `release/SIENTIAPDE-1974` → `main` (orchestrator) | Subtask do 1972 | orchestrator | 0 |
+| Consumidores 12/13/14: report + aignosi-scripts | Subtask | report, aignosi-scripts | 0 |
+| **[Nova história] Banco de tags e certificados OPC estruturados** | História (épico 1274 ou novo) | — | 1–5 |
+| ↳ Orchestrator: popular banco de tags, gravar `tag_id` nas pipelines, correlação servidor ↔ certificado (collections/índices via migrate-mongo) | Subtask | orchestrator, database-migrations | 1 |
+| ↳ Gateway: `CertificateStore` por `certificate_id`, validação SAN/expiração, métrica de expiração | Subtask | opc-ingestor | 2 |
+| ↳ Gateway: leitura ativa unitária sem persistência (G-AR) + design (G-DOC) | Subtask | opc-ingestor, orchestrator, aig-docs | 2 |
+| ↳ Gateway: reconexão por `status_change_notification` | Subtask | opc-ingestor | 2 |
+| ↳ Orchestrator: `connectors` como fonte, slot com certificados, sem `active_read`, reação a `updated_at` | Subtask | orchestrator | 3 |
+| ↳ Angular + BFF: CRUD conectores/tags/certificados (substitui 1568/1569) | História própria | sientia, mlops-bff | 4 |
+| ↳ Remoção do legado (`opc-servers`, `platform_certificates`, `READ_ENGINE=legacy`) | Subtask | vários | 5 |
+
+---
+
+## 9. Riscos
+
+- **Laborious 1.6.0 antes do gateway 2.x** em qualquer ambiente → escritas presas em `pending` (P4).
+- **Backfill com ids divergentes** entre `opc-servers` e um `connectors` já existente em algum cliente (o
+  Streamlit rc pode ter rodado em algum lugar) → checar antes, migration aborta em conflito.
+- **Troca de bucket/objeto do certificado** derruba conexões seguras se o gateway não invalidar o cache —
+  já existe `invalidate()` em falha; manter e testar por `certificate_id`.
+- **Servidores de cliente com Basic256 apenas** (asyncua marca deprecated) → leitura v2 não conecta; levantar por cliente antes.
+
+---
+
+## 10. Referências
+
+- Jira: SIENTIAPDE-1972, 1973, 1974, 1975, 1985, 2202, 2270, 1568, 1569, 1598, 1599, 1603, 1605; idea SIENTIAPDI-84.
+- Código: `ingestor/connection/certs.py`, `ingestor/connection/servers.py`, `ingestor/gateway_runtime.py`,
+  `ingestor/managers/ingestor_manager.py:708`, `ingestor/io/slots.py`.
+- Orchestrator: `release/SIENTIAPDE-1974` — `orchestrator/activities/formatters.py`, `orchestrator/utils/orchestrator_functions.py`.
+- Streamlit: `sientia-projects-templates` `rc/SIENTIAPDE-1588` — `app/pages/connectors.py` (`_save_platform_certificate`), `app/pages/tag_monitor.py`.
+- Migrations: `mongodb/migrations/20260908170557-model_id_counters_collection.js` (padrão de counter).
+- Design: `aig-docs/07 - Projects & R&D/sientia/opc-gateway/design.md` §3, §7, §14, §15.

docs/planning/SIENTIAPDE-1974-certificado-por-conector.md

@@ -0,0 +1,241 @@
+# SIENTIAPDE-1974 — Certificado OPC por conector (`opc_certificates`)
+
+> Plano de implementação. Atualizado em 2026-10-29.
+> Repositórios: `sientia-dataops-database-migrations`, `sientia-helm-chart`,
+> `sientia-dataops-orchestrator_temporal`, `sientia-dataops-opc-ingestor`.
+
+## Objetivo
+
+Ligar cada servidor OPC (`connectors`) ao certificado que ele usa (`opc_certificates`) por **um único caminho**,
+sem transição, sem fallback e sem legado:
+
+```
+connectors.security.client_certificate_id ──► opc_certificates.certificate_id
+                                                     │
+orchestrator: faz o join e grava no slot v2, por servidor, security + certificado (paths no MinIO + application_uri)
+                                                     │
+ingestor: lê o slot, baixa cert + chave do MinIO e conecta. Sem certificado no slot → conecta sem segurança
+```
+
+O ingestor não usa certificado "default", `platform_certificates`, `opc-servers`, nem o caminho legado
+(`READ_ENGINE=legacy`, `OpcManager`, slot v1).
+
+**Exceção consciente (D1):** a escrita mantém um fallback no Mongo. Quando o pod que recebe o comando não tem a
+conexão no pool, ele resolve o servidor em `connectors` → `opc_certificates`, com a mesma regra do orchestrator. São
+duas fontes da mesma regra, aceitas por enquanto. A otimização fica num card próprio (ver "Fora do escopo").
+
+**Fora do escopo:** migração e correção de dados. As migrations deste plano tratam só de estrutura.
+
+---
+
+## Estado atual (verificado)
+
+| Item | Estado | Evidência |
+|---|---|---|
+| `platform_certificates` | Criada pelo front Streamlit, sem migration | `sientia-projects-templates` `origin/rc/SIENTIAPDE-1588` `app/pages/connectors.py:27` |
+| `opc_certificates` | Criada pela migration `20261028170000-opc_tag_database_collections.js`, com schema parcial. Tem dados no dev | Migration, consulta no dev |
+| Vínculo connector → certificado | O campo existe no schema (`connectors.security.client_certificate_id`). No dev está `null` | Migration, consulta no dev |
+| `security.mode` / `security.policy` | Estão no schema e nos dados do dev, mas o ingestor ignora os dois: usa `use_security` e fixa `Basic256Sha256` no código | `ingestor/connection/security.py`, `pool.py:130` |
+| `application_uri` | Vem de `connectors.server_uri`, mas é dado do certificado (tem que bater com a SAN URI do `.pem`) | `pool.py:126-132`, `security.py:9` |
+| Leitura do certificado | `CertProvider` faz `find_one('platform_certificates', {})` e baixa do `CERT_BUCKET` | `ingestor/connection/certs.py:10,47` |
+| Bucket `opc-certificates` | Citado em `opc_certificates.cert.bucket`, mas não provisionado. O chart aponta `CERT_BUCKET=streamlit-connectors` | `sientia-helm-chart/values/minio.yaml:516-523`, `opc-ingestor/values.yaml:116` |
+| Mudança de config de um servidor no slot | O ingestor compara a config inteira do servidor (sem `tags`) e reconecta se algo mudou | `ingestor/managers/ingestor_manager.py:293-307` |
+| Escrita (NATS) | Core NATS, sem JetStream, entrega *at-most-once*. Queue group `gateway-writers`: cada comando vai para um só pod, que não tem como devolvê-lo. O Laborious publica e esquece; o resultado vai para `write_status` no Postgres | `sientia-helm-chart/docs/nats-broker.md:29-39`, `ingestor/adapters/nats_consumer.py`, `laborious/activities/nats_write.py` |
+
+---
+
+## Decisões
+
+| # | Decisão | Status |
+|---|---|---|
+| D0 | `opc_certificates` é a oficial; o ingestor deixa de ler a `platform_certificates` (a collection fica) | **Decidido** |
+| D0.1 | Migration da `opc_certificates` completa, documentada e auditável | **Decidido** (Etapa 1) |
+| D0.2 | Bucket `opc-certificates` provisionado no `sientia-helm-chart`, branch `feature/SIENTIAPDE-1974-connectors-pi-web-api` | **Decidido** (Etapa 2) |
+| D0.3 | Sem transição. Caminho único na leitura; a escrita tem a exceção de D1 | **Decidido** |
+| D1 | Escrita: pool primeiro; sem conexão no pool, resolve no Mongo (`connectors` → `opc_certificates`) com a mesma regra do orchestrator. Roteamento ao pod dono vira card novo | **Decidido** (ver abaixo) |
+| D2 | Zero legado: `READ_ENGINE=legacy`, `OpcManager`, slot v1, `opc-servers` | **Decidido** |
+| D3 | Dados fora do plano; migrations só de estrutura; sem runbook | **Decidido** |
+| D4 | `security.mode` + `security.policy` são a fonte. `use_security` e `server_uri` saem | **Decidido** |
+| D5 | Sem certificado registrado, o servidor é tratado como um que não exige certificado: conecta sem segurança (e falha se o servidor exigir) | **Decidido** |
+| D6 | Branch do ingestor: `release/SIENTIAPDE-1973` | **Decidido** |
+| D7 | `created_at`, `updated_at`, `created_by`, `updated_by` obrigatórios. Rastro do backfill não é validado. Sem `additionalProperties: false` | **Decidido** |
+
+### D1 — Escrita: pool primeiro, Mongo como fallback (decidido)
+
+**Como funciona hoje:** o Laborious publica o comando (C2) em `opc.write` só com o `server_id`. No ingestor, o
+`ServerConfigResolver` (`ingestor/connection/servers.py`) usa a conexão do pool, se existir. Senão, consulta o Mongo
+(`connectors` e depois `opc-servers`), com cache de 60 s (10 s quando não encontra), e o pool abre a conexão.
+
+**O que muda:**
+- Sai `opc-servers` (D2) e sai o override por dados de conexão no comando (o Laborious nunca envia).
+- O lookup no Mongo passa a fazer o join `connectors` → `opc_certificates` e monta a config **no mesmo formato do
+  servidor no slot** (`server_id`, `url`, `security`, `certificate`). O pool e o `CertProvider` só conhecem esse formato.
+- A regra do certificado (D5: certificado `active` e do tipo `client`, senão `certificate: None`) fica isolada numa
+  função do ingestor, com testes que espelham os do orchestrator.
+
+**Por que não só o pool:** o Core NATS entrega cada comando a um único pod, sem devolver nem reentregar. Rotear o
+comando ao pod dono da conexão (subject por servidor, republicação ou JetStream) não cobre um servidor que nenhum pod
+tem no pool (servidor só de escrita, ou pod recém-reiniciado) e ainda exige uma regra de expiração do `pending`.
+O fallback no Mongo cobre todos esses casos hoje.
+
+**Custo aceito:**
+- Duas fontes da mesma regra (orchestrator e ingestor).
+- Sessões duplicadas: o pod que recebe uma escrita sem ter a conexão abre a sua própria sessão (já acontece hoje).
+
+### D5 — Sem certificado registrado (decidido)
+
+- O orchestrator grava `certificate: None` no slot quando `client_certificate_id` é `null`, aponta para um id que não
+  existe, ou aponta para um certificado que não é `active` ou não é `client`. Nos dois últimos casos (id preenchido,
+  mas sem certificado válido), o orchestrator registra um aviso em `issues`, porque provavelmente é erro de cadastro.
+  Mesmo assim o servidor entra no slot.
+- O ingestor aplica segurança **só quando o slot tem `certificate`**. Nesse caso usa `security.mode` e
+  `security.policy`. Sem `certificate`, conecta sem segurança, e um servidor que exige certificado recusa a conexão.
+  O erro aparece no log e nas métricas de conexão do ingestor.
+- Não existe certificado "default": `is_default` e o índice parcial `ux_opc_certificates_default_client` saem do schema.
+
+---
+
+## Etapa 1 — Migrations de estrutura (`sientia-dataops-database-migrations`)
+
+**Nova migration** `mongodb/migrations/<timestamp>-opc_certificates_schema.js` (SIENTIAPDE-1974):
+
+1. **Schema da `opc_certificates`**, com o cabeçalho do arquivo descrevendo cada campo:
+
+   | Campo | Tipo | Obrigatório | Descrição |
+   |---|---|---|---|
+   | `certificate_id` | int/long | sim | Id estável, referenciado por `connectors.security.client_certificate_id` |
+   | `kind` | `client` \| `server` \| `ca` | sim | `client`: cert + chave do ingestor. `server`/`ca`: só cert |
+   | `name` | string | sim | Nome legível, único por `kind` |
+   | `status` | `active` \| `retired` | sim | Só `active` entra no slot |
+   | `application_uri` | string | sim se `client` | SAN URI do certificado, usada como `client.application_uri` |
+   | `cert` | `{bucket, object_name, sha256}` | sim (`bucket`, `object_name`) | Objeto no MinIO. `sha256`: string (64 hex) ou null |
+   | `private_key` | `{bucket, object_name, sha256}` | sim se `client`, proibido nos outros | Chave privada no MinIO |
+   | `thumbprint_sha1` | string (40 hex) \| null | não | O que aparece no console do servidor OPC |
+   | `not_before` / `not_after` | date \| null | não | Validade (`not_after` já tem índice) |
+   | `created_at` / `updated_at` | date | sim | Auditoria |
+   | `created_by` / `updated_by` | string | sim | Auditoria: quem cadastrou ou trocou o certificado |
+
+   - "Obrigatório se `client`" fica num `anyOf` do `$jsonSchema`.
+   - Campos fora da tabela (como o rastro do backfill) não são descritos nem validados: sem `additionalProperties: false`.
+   - Índices: novo `ux_opc_certificates_kind_name` em `{kind, name}`, único. Sai `ux_opc_certificates_default_client` (D5). Ficam `ux_opc_certificates_certificate_id` e `ix_opc_certificates_not_after`.
+   - `validationLevel: strict`.
+   - `down`: volta ao validator e aos índices da `20261028170000`.
+
+2. **Schema dos `connectors`** (parte do schema da `20261028190000-connectors_pi_web_api_types.js`):
+   - Saem `use_security` e `server_uri` (D4).
+   - `security` passa a ser obrigatório para `type: 'opc_ua'`, com `mode` e `policy` obrigatórios.
+   - `security.client_certificate_id` continua opcional/nullable (D5).
+   - `down` restaura o schema anterior.
+
+3. **Testes** em `tests/`, no padrão de `test_connectors_types.py` e `test_head.py`:
+   - Documentos válidos e inválidos por `kind` (com e sem `private_key` e `application_uri`).
+   - Campos de auditoria ausentes.
+   - Unicidade de `{kind, name}`.
+   - Connector `opc_ua` com e sem `security`, e com `client_certificate_id` nulo.
+   - Roundtrip `up` → `down`.
+   - Head esperado.
+
+## Etapa 2 — Provisionamento (`sientia-helm-chart`, branch `feature/SIENTIAPDE-1974-connectors-pi-web-api`)
+
+1. `values/minio.yaml`: bucket `opc-certificates` em `provisioning.buckets` (linhas 516-523).
+2. Acesso: o ingestor usa as credenciais root do MinIO (`opc-ingestor/values.yaml:106-115`). Avaliar com quem cuida
+   da plataforma um usuário e uma policy de **somente leitura** em `opc-certificates`, como `model-training-user`.
+3. `src/charts-internal/opc-ingestor/values.yaml:98-117`: remover a env `CERT_BUCKET` e o comentário sobre
+   `platform_certificates`. Remover `READ_ENGINE`, se estiver definida em algum profile (D2).
+4. `docs/nats-broker.md:373`: trocar a menção à `platform_certificates` pelo fluxo novo. O NATS não muda (D1).
+
+## Etapa 3 — Orchestrator (`sientia-dataops-orchestrator_temporal`)
+
+1. **`orchestrator/workflows/orchestrator.py` (`DEFAULT_CONNECTORS_QUERY`)**:
+   - Projetar `security.mode`, `security.policy` e `security.client_certificate_id`. Saem `server_uri` e `use_security`.
+   - `$lookup` de `security.client_certificate_id` para `opc_certificates.certificate_id`, com `status: 'active'` e `kind: 'client'`.
+   - Projetar `certificate_id`, `application_uri`, `cert.{bucket,object_name}` e `private_key.{bucket,object_name}`. As datas não entram (BSON que o payload do Temporal não carrega).
+2. **`orchestrator/utils/opc_slots.py`**: servidor do slot v2 (contrato C1):
+   ```python
+   {
+       'server_id': ..., 'name': ..., 'url': ...,
+       'subscription_period_ms': ...,
+       'security': {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'},
+       'certificate': {                   # None: sem certificado registrado (D5)
+           'certificate_id': 1,
+           'application_uri': 'urn:sientia:opc-ingestor',
+           'cert': {'bucket': 'opc-certificates', 'object_name': 'client/sientia-platform/opc-ingestor-cert.pem'},
+           'private_key': {'bucket': 'opc-certificates', 'object_name': 'client/sientia-platform/opc-ingestor-key.pem'},
+       },
+       'tags': {...},
+   }
+   ```
+3. **`orchestrator/utils/tag_database.py`**:
+   - `CONNECTOR_FIELDS` sem `server_uri`.
+   - `client_certificate_id` preenchido, mas sem certificado ativo do tipo `client` → `certificate: None` mais aviso em `issues` e em `formatters.py` (D5).
+4. **Testes**:
+   - `test_opc_slots.py`, `test_tag_database.py`, `tag_database_fixtures.py`, `test_formatters.py`.
+   - `test_orchestrator.py` e a history de replay, se o input do workflow mudar.
+   - Cenários e2e.
+5. **Docs**: README (seção "OPC slots", contrato C1) e CLAUDE.md.
+
+## Etapa 4 — Ingestor (`sientia-dataops-opc-ingestor`, branch `release/SIENTIAPDE-1973`)
+
+**Certificado e segurança**
+
+| Arquivo | Mudança |
+|---|---|
+| `ingestor/connection/certs.py` | Sai o Mongo (`COLLECTION_NAME`, `find_one`, o `mongo` do construtor). `fetch(certificate)` baixa o `bucket/object_name` vindo do slot. Cache, lock e pasta (`<cache_dir>/<certificate_id>/`) separados por `certificate_id`. `invalidate(certificate_id)` |
+| `ingestor/config.py` | Sai o `cert_bucket`; fica o `cert_cache_dir` |
+| `ingestor/connection/security.py` | `apply_security(client, security, certificate, cert_path, key_path)`, com o `application_uri` do certificado e mode e policy do slot (mapa de string para classe do asyncua; desconhecida → erro) |
+| `ingestor/connection/pool.py` | `_connect`: aplica segurança só quando `certificate` não é `None` (D5). Sai o `server_uri`. `_open` invalida o certificado da conexão. `_check_cfg` passa a comparar `security` e `certificate` (só aviso; a reconexão vem do `ingestor_manager`) |
+| `ingestor/io/slots.py` | `SERVER_FIELDS`: `server_id`, `url`, `subscription_period_ms`, `security`, `certificate`. Sai a conversão do slot v1 |
+
+**Escrita (D1)**
+
+| Arquivo | Mudança |
+|---|---|
+| `ingestor/connection/servers.py` | `ServerConfigResolver`: pool primeiro (`config_for`); senão, Mongo, com o cache de TTL atual. `_lookup` faz uma aggregation em `connectors` (`type: 'opc_ua'`, `status: 'Active'`) com `$lookup` em `opc_certificates`, e devolve a config no formato do servidor no slot (`server_id`, `url`, `security`, `certificate`). Saem `opc-servers` (`OPC_SERVERS`, `_opc_server_cfg`, `_id_forms`, já que `connector_id` é slug string) e o override pelo comando |
+| `ingestor/connection/certificates.py` (novo) ou dentro de `servers.py` | Função que, a partir do documento do connector e do certificado do `$lookup`, devolve `certificate` ou `None` com a regra de D5 (certificado `active` e `client`; id preenchido sem certificado válido → aviso no log). Testes que espelham os do orchestrator |
+| `ingestor/engines/write.py` | `_server_cfg` sem o ramo de conexão vinda do comando. Sem mudança de comportamento: servidor não encontrado continua indo para erro no `write_status` |
+
+**Remoção do legado (D2)**
+
+| Arquivo | Mudança |
+|---|---|
+| `ingestor/managers/opc_manager.py` | Removido |
+| `ingestor/managers/ingestor_manager.py` | Saem `READ_ENGINES`/`READ_ENGINE`, os ramos `legacy`, a checagem de slot v1 (`:698`) e `_warn_server_cert_path` (`:708`) |
+| `ingestor/io/data_sink.py` | Sai o tratamento de slot v1 (`:56`) |
+| `ingestor/gateway_runtime.py` | `CertProvider` sem Mongo. `ServerConfigResolver` continua com Mongo (D1) |
+| `tests/` | Remover os testes do legado; adaptar `certs`, `pool`, `slots`, `servers`, `ingestor_manager` e `write` |
+| `README.md`, `CLAUDE.md` | Sem plano B e sem `platform_certificates` (README:67 e :236) |
+
+## Etapa 5 — Deploy (por ambiente)
+
+1. Migrations (Etapa 1). Os dados precisam estar migrados antes (fora do escopo): o `strict` rejeita documento fora do schema.
+2. Helm (Etapa 2).
+3. Ingestor e orchestrator **juntos**. Sem transição, um slot antigo não tem `security`/`certificate`.
+4. Validação no dev:
+   - O slot no Redis tem `security` e `certificate`.
+   - O ingestor conecta ao `kepserver` com o mode e a policy do banco.
+   - A troca de `client_certificate_id` reconecta com o certificado novo.
+   - Um servidor sem certificado conecta sem segurança.
+   - Um id de certificado inválido gera aviso.
+   - Uma escrita para um servidor que está no pool usa a conexão do pool.
+   - Uma escrita para um servidor fora do pool resolve pelo Mongo, com o mesmo certificado que o slot teria.
+   - Uma escrita para um `server_id` inexistente termina em erro no `write_status`.
+
+## Fora do escopo
+
+- Migração e correção de dados (`opc_certificates`, `connectors`, objetos no MinIO).
+- Remoção da collection `platform_certificates` (o ingestor só deixa de lê-la).
+- Validação do certificado do servidor (`security.server_certificate_id`, `kind: server`).
+- Métrica e alerta de expiração (`not_after`).
+- Conferência de `sha256` no download.
+- Upload e CRUD de certificados no front Angular/BFF (quem preenche `created_by`/`updated_by`).
+- **Card novo — Escrita OPC roteada ao pod dono da conexão:** subject por servidor (`opc.write.<server_id>`), slot
+  cobrindo servidores só de escrita e tratamento de comando sem destino (expiração do `pending` ou `request` com
+  *no responders*). Quando estiver pronto, remove o fallback no Mongo do `ServerConfigResolver` e a segunda fonte da
+  regra de certificado (D1).
+
+## Referências
+
+- Plano geral: `docs/planning/SIENTIAPDE-1972-banco-de-tags-e-certificados.md` (§5.1, §5.2, Fases 1–3).
+- Migrations: `mongodb/migrations/20261028170000-opc_tag_database_collections.js`, `20261028190000-connectors_pi_web_api_types.js`.
+- Escrita: `sientia-dataops-laborious` `laborious/activities/nats_write.py`; ingestor `ingestor/connection/servers.py`, `ingestor/engines/write.py:148-160`; `sientia-helm-chart/docs/nats-broker.md`.
+- Streamlit (origem da `platform_certificates`): `sientia-projects-templates` `origin/rc/SIENTIAPDE-1588` `app/pages/connectors.py`.

ingestor/adapters/nats_consumer.py

@@ -3,12 +3,14 @@
 Thin adapter: decode the message JSON and hand the dict to WriteEngine,
 which is already raise-safe against malformed commands. No metrics of its
 own — the engine records write metrics; a malformed-JSON drop is logged
-here. No reconnect logic — nats-py handles reconnection internally.
+here. Once connected, nats-py reconnects by itself, forever.
 """
 
+import asyncio
 import json
+import math
+import traceback
 
-import nats
 from nats.aio.client import Client
 from nats.aio.subscription import Subscription
 from sientia_do.observability.logger import Logger
@@ -24,18 +26,55 @@ class NatsWriteConsumer:
         logger: Logger,
         subject: str = 'opc.write',
         queue: str = 'gateway-writers',
+        connect_timeout_s: float = 5.0,
     ):
         self.nats_url = nats_url
         self.write_engine = write_engine
         self.logger = logger
         self.subject = subject
         self.queue = queue
+        self.connect_timeout_s = connect_timeout_s
         self.nc: Client | None = None
         self.sub: Subscription | None = None
 
+    @property
+    def closed(self) -> bool:
+        """True when there is no connection, or it was closed for good."""
+        return self.nc is None or self.nc.is_closed
+
     async def start(self) -> None:
-        self.nc = await nats.connect(servers=self.nats_url)
-        self.sub = await self.nc.subscribe(self.subject, queue=self.queue, cb=self._handle)
+        """Connect and subscribe, giving up after ``connect_timeout_s``.
+
+        It runs from the main loop (startup and ``GatewayRuntime.idle_sweep``), so it
+        must never block it: nats-py's defaults retry the first connection 60 times,
+        about 2 minutes, longer than the heartbeat TTL, and with
+        ``max_reconnect_attempts=-1`` they would retry it forever. Once connected,
+        reconnection is unbounded, so an outage never closes the connection for good.
+
+        Raises:
+            Exception: When the connection or the subscription failed; nothing is left
+                open in that case.
+        """
+        nc = Client()
+        try:
+            await asyncio.wait_for(
+                nc.connect(
+                    servers=self.nats_url,
+                    connect_timeout=math.ceil(self.connect_timeout_s),
+                    max_reconnect_attempts=-1,
+                    name='opc-gateway',
+                    # One log line per event instead of nats-py's traceback per attempt.
+                    error_cb=self._on_error,
+                    disconnected_cb=self._on_disconnected,
+                    reconnected_cb=self._on_reconnected,
+                ),
+                timeout=self.connect_timeout_s,
+            )
+            self.sub = await nc.subscribe(self.subject, queue=self.queue, cb=self._handle)
+        except (Exception, asyncio.CancelledError):
+            await self._discard(nc)
+            raise
+        self.nc = nc
         self.logger.info(f'NatsWriteConsumer started: subject={self.subject} queue={self.queue}')
 
     async def _handle(self, msg) -> None:
@@ -45,11 +84,41 @@ class NatsWriteConsumer:
             preview = msg.data[:200]
             self.logger.error(f'NatsWriteConsumer dropped malformed JSON ({e}): {preview!r}')
             return
-        await self.write_engine.write(payload)
+        try:
+            await self.write_engine.write(payload)
+        except Exception as e:
+            # The engine is raise-safe; this only keeps an unexpected bug from going to
+            # nats-py's error callback without context.
+            self.logger.error(
+                f'NatsWriteConsumer: unexpected error handling a write command ({e}):\n'
+                f'{traceback.format_exc()}'
+            )
+
+    # These stay async with nothing to await: nats-py rejects callbacks that are not
+    # coroutine functions (InvalidCallbackTypeError on connect).
+    async def _on_error(self, e: Exception) -> None:
+        self.logger.warning(f'NATS error: {type(e).__name__} {e}')
+
+    async def _on_disconnected(self) -> None:
+        self.logger.warning('Disconnected from NATS; reconnecting')
+
+    async def _on_reconnected(self) -> None:
+        self.logger.info('Reconnected to NATS')
+
+    async def _discard(self, nc: Client) -> None:
+        try:
+            await nc.close()
+        except Exception as e:
+            self.logger.warning(f'Error closing a failed NATS connection: {e}')
 
     async def stop(self) -> None:
         if self.nc is None:
             return
+        if self.nc.is_closed:
+            # Closed for good already: unsubscribe/drain would raise ConnectionClosedError
+            # and abort the shutdown sequence.
+            self.nc = None
+            return
         if self.sub is not None:
             await self.sub.unsubscribe()
         await self.nc.drain()

ingestor/config.py

@@ -22,9 +22,8 @@ class GatewayConfig:
     nats_queue: str = 'gateway-writers'
     grpc_enabled: bool = False
     grpc_port: int = 50051
-    # Legacy frontend bucket the platform already provisions (SIENTIAPDE-1974 D7);
-    # the design's `sientia-dataops` bucket does not exist in the platform MinIO.
-    cert_bucket: str = 'streamlit-connectors'
+    # The certificate bucket and objects come from the slot / the write lookup
+    # (opc_certificates, SIENTIAPDE-1974); only the local cache dir is configured here.
     cert_cache_dir: str = '/tmp/opc-certs'  # noqa: S108 # nosec B108 - default is spec'd, overridable via env
     write_timestamp: bool = True
     write_quality: bool = True
@@ -36,10 +35,7 @@ class GatewayConfig:
         """Only the certificate settings — what the read path needs. Plain strings,
         no number parsing, so a malformed write/gRPC env var (GRPC_PORT,
         WRITE_MIN_PAUSE_S, ...) can only disable the runtime, never the reads."""
-        return cls(
-            cert_bucket=getenv('CERT_BUCKET', cls.cert_bucket),
-            cert_cache_dir=getenv('CERT_CACHE_DIR', cls.cert_cache_dir),
-        )
+        return cls(cert_cache_dir=getenv('CERT_CACHE_DIR', cls.cert_cache_dir))
 
     @classmethod
     def from_env(cls) -> 'GatewayConfig':
@@ -49,7 +45,6 @@ class GatewayConfig:
             nats_queue=getenv('NATS_QUEUE', cls.nats_queue),
             grpc_enabled=_env_bool('GRPC_ENABLED', cls.grpc_enabled),
             grpc_port=int(getenv('GRPC_PORT', cls.grpc_port)),
-            cert_bucket=getenv('CERT_BUCKET', cls.cert_bucket),
             cert_cache_dir=getenv('CERT_CACHE_DIR', cls.cert_cache_dir),
             write_timestamp=_env_bool('WRITE_TIMESTAMP', cls.write_timestamp),
             write_quality=_env_bool('WRITE_QUALITY', cls.write_quality),

ingestor/connection/certificates.py

@@ -0,0 +1,92 @@
+"""Client certificate of a tag database connector (SIENTIAPDE-1974 D5).
+
+Mirrors the Orchestrator's `tag_database.resolve_certificate`, which puts the
+certificate in the read slot: the write lookup (ServerConfigResolver, D1) resolves a
+server that is not in the pool with the same rule, so a write uses the certificate the
+slot would have. Keep both in sync.
+"""
+
+from typing import Any
+
+# security fields of an OPC UA connector: the source of its OPC security (D4)
+SECURITY_FIELDS = ('mode', 'policy')
+CERTIFICATE_FIELDS = ('certificate_id', 'application_uri')
+CERTIFICATE_OBJECTS = ('cert', 'private_key')
+STORED_OBJECT_FIELDS = ('bucket', 'object_name')
+
+# opc_certificates lookup of a connectors aggregation: the certificate the connector's
+# security.client_certificate_id points to, only when it is active and of kind client.
+CERTIFICATE_LOOKUP = {
+    '$lookup': {
+        'from': 'opc_certificates',
+        'let': {'certificate_id': '$security.client_certificate_id'},
+        'pipeline': [
+            {
+                '$match': {
+                    '$expr': {'$eq': ['$certificate_id', '$$certificate_id']},
+                    'status': 'active',
+                    'kind': 'client',
+                }
+            },
+            {
+                '$project': {
+                    '_id': 0,
+                    'certificate_id': 1,
+                    'application_uri': 1,
+                    'cert.bucket': 1,
+                    'cert.object_name': 1,
+                    'private_key.bucket': 1,
+                    'private_key.object_name': 1,
+                }
+            },
+        ],
+        'as': 'certificates',
+    }
+}
+
+
+def _is_complete(document: Any, fields: tuple[str, ...]) -> bool:
+    return isinstance(document, dict) and all(
+        document.get(field) not in (None, '') for field in fields
+    )
+
+
+def _is_usable_certificate(certificate: Any) -> bool:
+    return _is_complete(certificate, CERTIFICATE_FIELDS) and all(
+        _is_complete(certificate.get(name), STORED_OBJECT_FIELDS) for name in CERTIFICATE_OBJECTS
+    )
+
+
+def client_certificate_id(connector: dict[str, Any]) -> Any:
+    """The connector's security.client_certificate_id (None when it has none)."""
+    security = connector.get('security')
+    return security.get('client_certificate_id') if isinstance(security, dict) else None
+
+
+def has_security(connector: dict[str, Any]) -> bool:
+    """Whether the connector has security.mode and security.policy (D4)."""
+    return _is_complete(connector.get('security'), SECURITY_FIELDS)
+
+
+def resolve_certificate(connector: dict[str, Any]) -> dict[str, Any] | None:
+    """
+    The client certificate of a connector from CERTIFICATE_LOOKUP, or None (D5).
+
+    None means no certificate: the connection is opened without security. A connector
+    whose client_certificate_id is set but has no usable certificate is most likely a
+    registration error; the caller reports it (see client_certificate_id).
+
+    Returns:
+        dict[str, Any] | None: certificate_id, application_uri, and cert and private_key
+            as {bucket, object_name}, as in the slot
+    """
+    for certificate in connector.get('certificates') or []:
+        if _is_usable_certificate(certificate):
+            return {
+                **{field: certificate[field] for field in CERTIFICATE_FIELDS},
+                **{
+                    name: {field: certificate[name][field] for field in STORED_OBJECT_FIELDS}
+                    for name in CERTIFICATE_OBJECTS
+                },
+            }
+    return None

ingestor/connection/certs.py

@@ -7,63 +7,88 @@ from sientia_do.repository.minio_repository import MinioRepository
 
 from ingestor.config import _env_bool
 
-COLLECTION_NAME = 'platform_certificates'
+# The two objects of a client certificate, and the local file each one is saved as
+CERTIFICATE_OBJECTS = (('cert', 'cert'), ('private_key', 'key'))
 
 
 class CertificateNotConfiguredError(RuntimeError):
-    """No usable platform OPC client certificate is registered (Mongo + MinIO)."""
+    """The certificate of a server does not name its MinIO objects (opc_certificates)."""
+
+
+def _objects(certificate: dict) -> tuple[tuple[str, str], ...]:
+    """(bucket, object_name) of the cert and the private key, in CERTIFICATE_OBJECTS order."""
+    try:
+        return tuple(
+            (str(certificate[field]['bucket']), str(certificate[field]['object_name']))
+            for field, _ in CERTIFICATE_OBJECTS
+        )
+    except (KeyError, TypeError) as e:
+        raise CertificateNotConfiguredError(
+            f'OPC client certificate {certificate!r} does not name the MinIO bucket and '
+            f'object of its cert and private_key (opc_certificates)'
+        ) from e
+
+
+def _certificate_id(certificate: dict) -> str:
+    """certificate_id as the name of its cache folder: an integer, never a path."""
+    try:
+        return str(int(certificate['certificate_id']))
+    except (KeyError, TypeError, ValueError) as e:
+        raise CertificateNotConfiguredError(
+            f'OPC client certificate {certificate!r} has no integer certificate_id'
+        ) from e
 
 
 class CertProvider:
-    """Downloads the platform's client cert + private key (Mongo metadata,
-    MinIO bytes) into a writable cache dir — pod root FS is read-only (§12),
-    so `cache_dir` must be an emptyDir/tmpfs mount.
+    """Downloads the client cert + private key of a connection (the MinIO objects its
+    slot server or write lookup names, SIENTIAPDE-1974) into a writable cache dir — pod
+    root FS is read-only (§12), so `cache_dir` must be an emptyDir/tmpfs mount.
 
-    The certificate is single and shared by every connection (reads, writes,
-    gRPC), so it is downloaded once and cached; `invalidate()` forces a fresh
-    download (the pool calls it when a secured connection fails, e.g. after the
-    certificate was replaced in MinIO). The lock serializes downloads so two
-    connections never rewrite the cache files at the same time."""
+    Each certificate (`certificate_id`) has its own folder (`<cache_dir>/<certificate_id>/`),
+    cache entry and lock, so two connections never rewrite the same files at the same
+    time. A cached certificate is reused while it names the same objects; when the slot
+    names other objects for the same id (the certificate was replaced), it is downloaded
+    again. `invalidate(certificate_id)` forces a fresh download (the pool calls it when a
+    secured connection fails, e.g. after the objects were replaced in MinIO)."""
 
-    def __init__(self, mongo, minio, cache_dir: str, bucket: str, metadata: dict | None = None):
-        self.mongo = mongo
+    def __init__(self, minio, cache_dir: str, metadata: dict | None = None):
         self.minio = minio
         self.cache_dir = cache_dir
-        self.bucket = bucket
         self.metadata = metadata or {}
-        self._lock = asyncio.Lock()
-        self._paths: tuple[str, str] | None = None
+        # ponytail: one Lock per certificate, never cleaned up — the set is small.
+        self._locks: dict[str, asyncio.Lock] = {}
+        self._paths: dict[str, tuple[tuple[tuple[str, str], ...], tuple[str, str]]] = {}
 
-    async def fetch(self) -> tuple[str, str]:
-        async with self._lock:
-            if self._paths is None:
-                self._paths = await self._download_all()
-            return self._paths
+    async def fetch(self, certificate: dict) -> tuple[str, str]:
+        """Local paths of the cert and private key of `certificate` (slot format)."""
+        certificate_id = _certificate_id(certificate)
+        objects = _objects(certificate)
+        async with self._locks.setdefault(certificate_id, asyncio.Lock()):
+            cached = self._paths.get(certificate_id)
+            if cached is None or cached[0] != objects:
+                self._paths[certificate_id] = (
+                    objects,
+                    await self._download_all(certificate_id, objects),
+                )
+            return self._paths[certificate_id][1]
 
-    def invalidate(self) -> None:
-        self._paths = None
+    def invalidate(self, certificate_id) -> None:
+        self._paths.pop(str(certificate_id), None)
 
-    async def _download_all(self) -> tuple[str, str]:
-        doc = await self.mongo.find_one(COLLECTION_NAME, {}, self.metadata)
-        try:
-            cert_object = doc['client_cert']['minio_object_name']
-            key_object = doc['private_key']['minio_object_name']
-        except (KeyError, TypeError) as e:
-            raise CertificateNotConfiguredError(
-                f'No OPC client certificate configured: the Mongo collection '
-                f"'{COLLECTION_NAME}' needs a document with client_cert.minio_object_name "
-                f'and private_key.minio_object_name (objects in the MinIO bucket '
-                f"'{self.bucket}'). Register the platform certificate before using "
-                f'secured OPC servers.'
-            ) from e
-        Path(self.cache_dir).mkdir(parents=True, exist_ok=True)
-        cert_path = await self._download(cert_object)
-        key_path = await self._download(key_object)
+    async def _download_all(
+        self, certificate_id: str, objects: tuple[tuple[str, str], ...]
+    ) -> tuple[str, str]:
+        folder = Path(self.cache_dir) / certificate_id
+        folder.mkdir(parents=True, exist_ok=True)
+        (cert_bucket, cert_object), (key_bucket, key_object) = objects
+        cert_path = await self._download(cert_bucket, cert_object, folder / 'cert')
+        key_path = await self._download(key_bucket, key_object, folder / 'key')
         return cert_path, key_path
 
-    async def _download(self, object_name: str) -> str:
-        data = await self.minio.download_file(object_name, self.bucket, self.metadata)
-        local_path = Path(self.cache_dir) / Path(object_name).name
+    async def _download(self, bucket: str, object_name: str, stem: Path) -> str:
+        data = await self.minio.download_file(object_name, bucket, self.metadata)
+        # The extension tells asyncua the format (.pem / .der)
+        local_path = stem.with_suffix(Path(object_name).suffix)
         # Atomic replace: a connection loading the previous file never sees it half-written.
         tmp_path = local_path.with_name(local_path.name + '.tmp')
         tmp_path.write_bytes(data)
@@ -72,43 +97,43 @@ class CertProvider:
 
 
 class LazyMinio:
-    """Builds the MinIO client on first use. MinioRepository's constructor hits the
-    network (bucket check), so building it eagerly would let a MinIO outage at pod
-    start take the whole read path down. Built lazily, only the servers that need a
-    certificate fail — and they are retried on the next loop."""
+    """Builds one MinIO client per bucket on first use. MinioRepository's constructor
+    hits the network (bucket check), so building it eagerly would let a MinIO outage at
+    pod start take the whole read path down. Built lazily, only the servers that need a
+    certificate fail — and they are retried on the next loop. One client per bucket,
+    because the constructor creates its bucket when missing: it is only ever given a
+    bucket a certificate names."""
 
     def __init__(self, factory):
         self._factory = factory
-        self._client = None
+        self._clients: dict[str, MinioRepository] = {}
 
     async def download_file(self, object_name: str, bucket: str, metadata: dict):
-        if self._client is None:
-            self._client = self._factory()
-        return await self._client.download_file(object_name, bucket, metadata)
+        if bucket not in self._clients:
+            self._clients[bucket] = self._factory(bucket)
+        return await self._clients[bucket].download_file(object_name, bucket, metadata)
 
 
 def build_cert_provider(
-    mongo, config, logger, notification_handler, metrics_controller, metadata=None
+    config, logger, notification_handler, metrics_controller, metadata=None
 ) -> CertProvider:
     """CertProvider shared by the read path (IngestorManager) and the write path
-    (GatewayRuntime): one certificate source — MinIO + Mongo — for every connection."""
+    (GatewayRuntime): one certificate cache for every connection."""
 
-    def minio_factory() -> MinioRepository:
+    def minio_factory(bucket: str) -> MinioRepository:
         return MinioRepository(
             endpoint=getenv('MINIO_ENDPOINT', 'localhost:9000'),
             access_key=getenv('MINIO_ACCESS_KEY', 'minioadmin'),
             secret_key=getenv('MINIO_SECRET_KEY', 'minioadmin'),
             secure=_env_bool('MINIO_SECURE', True),
-            bucket=config.cert_bucket,
+            bucket=bucket,
             logger=logger,
             notification_handler=notification_handler,
             metrics_controller=metrics_controller,
         )
 
     return CertProvider(
-        mongo=mongo,
         minio=LazyMinio(minio_factory),
         cache_dir=config.cert_cache_dir,
-        bucket=config.cert_bucket,
         metadata=metadata,
     )

ingestor/connection/pool.py

@@ -9,7 +9,7 @@ from sientia_do.observability.sientia_monitoring import SientiaMonitoring
 import ingestor.metrics as metrics
 from ingestor.connection.certs import CertProvider
 from ingestor.connection.clock import MonotonicClock
-from ingestor.connection.security import apply_security
+from ingestor.connection.security import apply_security, is_secured
 
 
 class OpcConnectionPool(SientiaMonitoring):
@@ -65,17 +65,26 @@ class OpcConnectionPool(SientiaMonitoring):
         """Warn when a caller asks for a server with other connection settings than
         the pooled client was opened with. The client is reused as is: reconnecting
         on every mismatch would let one malformed write command drop the reads.
-        Config changes from the slot evict the client first (ServerReader.shutdown)."""
+        Config changes from the slot (security, certificate) evict the client first:
+        the IngestorManager re-initializes the reader (ServerReader.shutdown)."""
         opened = self._configs.get(server_id, {})
-        diff = [
-            k for k in ('url', 'server_uri', 'use_security') if opened.get(k) != server_cfg.get(k)
-        ]
+        diff = [k for k in ('url', 'security', 'certificate') if opened.get(k) != server_cfg.get(k)]
         if diff:
             self.logger.warning(
                 f'OPC server {server_id}: reusing the pooled connection, but {diff} differ '
                 f'from the settings it was opened with'
             )
 
+    def config_for(self, server_id: str) -> dict | None:
+        """Connection settings of the pooled client for `server_id`, or None when the
+        server has no live pooled connection. Lets a writer reuse exactly the settings
+        the read path opened the connection with."""
+        server_id = str(server_id)
+        if server_id not in self._connections:
+            return None
+        cfg = self._configs.get(server_id)
+        return dict(cfg) if cfg is not None else None
+
     def _cached(self, server_id: str) -> Client | None:
         entry = self._connections.get(server_id)
         if entry is None:
@@ -88,10 +97,10 @@ class OpcConnectionPool(SientiaMonitoring):
         try:
             client = await self._connect(server_cfg)
         except Exception:
-            if server_cfg.get('use_security'):
+            if is_secured(server_cfg):
                 # The cached certificate may be the cause (replaced in MinIO,
                 # rejected by the server): download it again on the next attempt.
-                self.cert_provider.invalidate()
+                self.cert_provider.invalidate(server_cfg['certificate']['certificate_id'])
             await self.emit_metric(
                 metric_object=metrics.OPC_CONNECTIONS_FAILED,
                 tags={'pod_id': self.pod_id, 'server_name': server_id},
@@ -113,13 +122,12 @@ class OpcConnectionPool(SientiaMonitoring):
         pod_uri = self.pod_id.replace('-', ':')
         client.product_uri = pod_uri
         client.application_uri = pod_uri
-        server_uri = server_cfg.get('server_uri')
-        if server_uri:
-            # Must match the certificate's SAN URI (KEPServer: BadCertificateUriInvalid).
-            client.application_uri = server_uri
-        if server_cfg.get('use_security'):
-            cert, key = await self.cert_provider.fetch()
-            await apply_security(client, server_cfg.get('server_uri'), cert, key, None)
+        # Security only with a client certificate (SIENTIAPDE-1974 D5); apply_security
+        # sets the certificate's application URI, which must match its SAN URI.
+        if is_secured(server_cfg):
+            certificate = server_cfg['certificate']
+            cert, key = await self.cert_provider.fetch(certificate)
+            await apply_security(client, server_cfg['security'], certificate, cert, key)
         await client.connect()
         return client
 

ingestor/connection/security.py

@@ -1,17 +1,65 @@
-from asyncua.crypto.security_policies import SecurityPolicyBasic256Sha256
+from asyncua import ua
+from asyncua.crypto.security_policies import (
+    SecurityPolicyAes128Sha256RsaOaep,
+    SecurityPolicyAes256Sha256RsaPss,
+    SecurityPolicyBasic128Rsa15,
+    SecurityPolicyBasic256,
+    SecurityPolicyBasic256Sha256,
+)
 
 SESSION_TIMEOUT_MS = 600_000  # 10 min; tune per server (Samarco: long sessions degrade)
 
+# connectors.security.policy / .mode (SIENTIAPDE-1974 D4) to the asyncua classes
+POLICIES = {
+    'Basic256Sha256': SecurityPolicyBasic256Sha256,
+    'Aes128Sha256RsaOaep': SecurityPolicyAes128Sha256RsaOaep,
+    'Aes256Sha256RsaPss': SecurityPolicyAes256Sha256RsaPss,
+    'Basic256': SecurityPolicyBasic256,
+    'Basic128Rsa15': SecurityPolicyBasic128Rsa15,
+}
+MODES = {
+    'Sign': ua.MessageSecurityMode.Sign,
+    'SignAndEncrypt': ua.MessageSecurityMode.SignAndEncrypt,
+}
 
-async def apply_security(client, server_uri, cert_path, private_key_path, server_cert_path):
-    """Apply Basic256Sha256 + certificate auth (1879 §1) and set channel > session
-    lifetime (1879 §3: secure_channel_timeout = session_timeout * 1.25)."""
-    client.application_uri = server_uri
+
+class UnsupportedSecurityError(ValueError):
+    """The server's security mode or policy has no asyncua counterpart."""
+
+
+def is_secured(server_cfg: dict) -> bool:
+    """Whether a connection applies security: only with a client certificate (D5), and
+    not when its mode is 'None'. Without one, it connects without security, and a server
+    that requires a certificate refuses the connection."""
+    security = server_cfg.get('security') or {}
+    return server_cfg.get('certificate') is not None and security.get('mode') != 'None'
+
+
+async def apply_security(client, security, certificate, cert_path, private_key_path):
+    """Apply the server's mode + policy with the connector's client certificate, and set
+    channel > session lifetime (1879 §3: secure_channel_timeout = session_timeout * 1.25).
+
+    The application URI is the certificate's (its SAN URI): the server rejects a
+    different one (KEPServer: BadCertificateUriInvalid).
+
+    Raises:
+        UnsupportedSecurityError: When the mode or policy is unknown.
+    """
+    policy = POLICIES.get(security.get('policy'))
+    mode = MODES.get(security.get('mode'))
+    if policy is None or mode is None:
+        raise UnsupportedSecurityError(
+            f'Unsupported OPC security mode={security.get("mode")!r} '
+            f'policy={security.get("policy")!r} (modes: {", ".join(MODES)}; '
+            f'policies: {", ".join(POLICIES)})'
+        )
+    client.application_uri = certificate['application_uri']
     await client.set_security(
-        SecurityPolicyBasic256Sha256,
+        policy,
         certificate=cert_path,
         private_key=private_key_path,
-        server_certificate=server_cert_path,
+        server_certificate=None,
+        mode=mode,
     )
     client.session_timeout = SESSION_TIMEOUT_MS
     client.secure_channel_timeout = int(SESSION_TIMEOUT_MS * 1.25)

ingestor/connection/servers.py

@@ -0,0 +1,126 @@
+"""Connection settings of the OPC server a write command targets (SIENTIAPDE-1975 GW1).
+
+The Laborious only knows the `server_id` of a write tag (pipeline `write_tags`); the
+connection (url, security, certificate) belongs to the gateway, which owns every OPC
+session. A server already in the pool is written through that connection. Otherwise it
+is resolved from the tag database (`connectors` → `opc_certificates`) with the rule the
+Orchestrator uses to build the read slots (SIENTIAPDE-1974 D1), in the slot format.
+"""
+
+from typing import Any
+
+from sientia_do.observability.logger import Logger
+
+from ingestor.connection.certificates import (
+    CERTIFICATE_LOOKUP,
+    client_certificate_id,
+    has_security,
+    resolve_certificate,
+)
+from ingestor.connection.clock import MonotonicClock
+from ingestor.connection.pool import OpcConnectionPool
+
+CONNECTORS = 'connectors'
+
+
+def _connector_pipeline(server_id: str) -> list[dict[str, Any]]:
+    """The active OPC UA connector `server_id` (connector_id is its slug), with its
+    certificate. Only the fields used are projected."""
+    return [
+        {'$match': {'connector_id': server_id, 'type': 'opc_ua', 'status': 'Active'}},
+        {'$limit': 1},
+        CERTIFICATE_LOOKUP,
+        {
+            '$project': {
+                '_id': 0,
+                'connector_id': 1,
+                'url': 1,
+                'subscription_period_ms': 1,
+                'security.mode': 1,
+                'security.policy': 1,
+                'security.client_certificate_id': 1,
+                'certificates': 1,
+            }
+        },
+    ]
+
+
+class ServerConfigResolver:
+    """Resolve and cache the connection settings of an OPC server by `server_id`.
+
+    Order: the live pooled connection (what the read path opened, so writes never fight
+    the reads over settings), then MongoDB. Lookups are cached for `ttl_s`; a server
+    that is not found, for `missing_ttl_s`, so a pipeline pointing at a removed server
+    does not query MongoDB on every command.
+
+    # ponytail: the MongoDB fallback goes away once write commands are routed to the pod
+    # that owns the connection (SIENTIAPDE-1974 D1, new card).
+    """
+
+    def __init__(
+        self,
+        pool: OpcConnectionPool,
+        mongo,
+        logger: Logger,
+        ttl_s: float = 60.0,
+        missing_ttl_s: float = 10.0,
+        metadata: dict | None = None,
+    ):
+        self.pool = pool
+        self.mongo = mongo
+        self.logger = logger
+        self.ttl_s = ttl_s
+        self.missing_ttl_s = missing_ttl_s
+        self.metadata = metadata or {}
+        self.clock = MonotonicClock()
+        self._cache: dict[str, tuple[dict[str, Any] | None, float]] = {}
+
+    async def resolve(self, server_id: str) -> dict[str, Any] | None:
+        """Connection settings for `server_id`, or None when no source knows it.
+
+        Raises:
+            Exception: When MongoDB cannot be read (the caller reports the write as
+                failed).
+        """
+        server_id = str(server_id)
+        pooled = self.pool.config_for(server_id)
+        if pooled is not None:
+            return pooled
+        now = self.clock.elapsed()
+        cached = self._cache.get(server_id)
+        if cached is not None:
+            cfg, cached_at = cached
+            if now - cached_at < (self.ttl_s if cfg is not None else self.missing_ttl_s):
+                return dict(cfg) if cfg is not None else None
+        cfg = await self._lookup(server_id)
+        self._cache[server_id] = (cfg, now)
+        return dict(cfg) if cfg is not None else None
+
+    async def _lookup(self, server_id: str) -> dict[str, Any] | None:
+        connectors = await self.mongo.aggregate(
+            CONNECTORS, _connector_pipeline(server_id), self.metadata
+        )
+        connector = connectors[0] if connectors else None
+        if connector is None or not connector.get('url') or not has_security(connector):
+            self.logger.warning(
+                f'OPC server {server_id} not found, not active or incomplete (url, '
+                f'security.mode, security.policy) in {CONNECTORS}'
+            )
+            return None
+        certificate = resolve_certificate(connector)
+        if certificate is None and client_certificate_id(connector) is not None:
+            self.logger.warning(
+                f'OPC server {server_id}: client_certificate_id '
+                f'{client_certificate_id(connector)} has no active client certificate in '
+                f'opc_certificates; connecting without security'
+            )
+        return {
+            'server_id': server_id,
+            'url': connector['url'],
+            'subscription_period_ms': connector.get('subscription_period_ms'),
+            'security': {
+                'mode': connector['security']['mode'],
+                'policy': connector['security']['policy'],
+            },
+            'certificate': certificate,
+        }

ingestor/engines/write.py

@@ -18,6 +18,7 @@ from sientia_do.observability.sientia_monitoring import SientiaMonitoring
 import ingestor.metrics as metrics
 from ingestor.connection.clock import MonotonicClock
 from ingestor.connection.pool import OpcConnectionPool
+from ingestor.connection.servers import ServerConfigResolver
 from ingestor.converters import ConversionError, to_typed_value
 from ingestor.io.data_sink import DataSink
 
@@ -77,9 +78,13 @@ class WriteEngine(SientiaMonitoring):
         write_timestamp: bool = True,
         write_quality: bool = True,
         min_pause_s: float = 0.0,
+        servers: ServerConfigResolver | None = None,
     ):
         self.pool = pool
         self.sink = sink
+        # Resolves the server connection by server_id (SIENTIAPDE-1975 GW1). Without it,
+        # no server is known and every write fails as server_not_found.
+        self.servers = servers
         self.write_timestamp = write_timestamp
         self.write_quality = write_quality
         self.min_pause_s = min_pause_s
@@ -105,11 +110,24 @@ class WriteEngine(SientiaMonitoring):
         except ConversionError as e:
             return await self._emit_error(cmd, 'conversion', str(e), 'error')
 
-        dv = self._build_datavalue(cmd, typed)
+        try:
+            dv = self._build_datavalue(cmd, typed)
+        except (KeyError, TypeError, ValueError) as e:
+            # write_timestamp on with a missing or non-ISO timestamp.
+            return await self._emit_error(cmd, 'bad_timestamp', repr(e), 'error')
+
+        try:
+            server_cfg = await self._server_cfg(cmd)
+        except Exception as e:
+            return await self._emit_error(cmd, 'server_lookup', str(e), 'error')
+        if server_cfg is None:
+            return await self._emit_error(
+                cmd, 'server_not_found', f'OPC server {cmd["server_id"]} is not configured', 'error'
+            )
 
         start = self.clock.elapsed()
         try:
-            client = await self.pool.get(cmd['server_id'], self._server_cfg(cmd))
+            client = await self.pool.get(cmd['server_id'], server_cfg)
             node = client.get_node(cmd['addr'])
             await node.write_value(dv)
         except Exception as e:
@@ -127,15 +145,13 @@ class WriteEngine(SientiaMonitoring):
                 now = self.clock.elapsed()
         self._last_write_at = now
 
-    def _server_cfg(self, cmd: dict) -> dict:
-        # ponytail: minimal cfg built straight from the C2 message. Resolving
-        # the full connector config (url/security) from the connectors DB by
-        # server_id is F4 wiring, not this task — pool is mocked in tests.
-        return {
-            'url': cmd.get('url'),
-            'use_security': cmd.get('use_security', False),
-            'server_uri': cmd.get('server_uri'),
-        }
+    async def _server_cfg(self, cmd: dict) -> dict | None:
+        """Connection settings of the target server, resolved by server_id (live pooled
+        connection, then connectors → opc_certificates). The command carries only the
+        server_id (C2)."""
+        if self.servers is None:
+            return None
+        return await self.servers.resolve(str(cmd['server_id']))
 
     def _variant_type(self, data_type: str) -> ua.VariantType:
         # direct index: unknown dtypes already became ConversionError in to_typed_value
@@ -156,6 +172,12 @@ class WriteEngine(SientiaMonitoring):
         return isinstance(exc, UaStatusCodeError) and type(exc).__name__ in _SESSION_BAD_NAMES
 
     async def _emit_error(self, cmd: dict, error_type: str, message: str, status: str):
+        # The reason only lives here: the row gets write_status alone (SIENTIAPDE-1975).
+        self.logger.error(
+            f'OPC write failed ({status}, {error_type}): server={cmd.get("server_id")} '
+            f'addr={cmd.get("addr")} model_id={cmd.get("model_id")} '
+            f'timestamp={cmd.get("timestamp")}: {message}'
+        )
         tags = {'pod_id': self.pod_id, 'server_name': cmd.get('server_id', 'unknown')}
         await self.emit_metric(
             metric_object=metrics.OPC_ERROR_BY_TYPE, tags={**tags, 'error_type': error_type}
@@ -164,6 +186,17 @@ class WriteEngine(SientiaMonitoring):
             metric_object=metrics.OPC_WRITE_TOTAL,
             tags={**tags, 'model_id': cmd.get('model_id', 'unknown'), 'result': 'error'},
         )
+        # Same series as a success (status='ok'), so it always shows the last outcome.
+        await self.emit_metric(
+            metric_object=metrics.OPC_WRITE_STATUS_BY_MODEL,
+            method='set',
+            value=0,
+            tags={
+                'pod_id': self.pod_id,
+                'model_id': cmd.get('model_id', 'unknown'),
+                'status': 'ok',
+            },
+        )
         result = WriteResult(ok=False, status=status, opc_status_code=error_type, message=message)
         await self.sink.update_write_status(cmd, result)
         return result

ingestor/gateway_runtime.py

@@ -2,8 +2,8 @@
 object graph and wires it into the app's lifecycle.
 
 Reuses the existing ingestor's logger/notification_handler/metrics_controller and
-its already-connected DataManager. The OpcConnectionPool (and its CertProvider,
-MinIO + Mongo) belongs to the read path in IngestorManager (SIENTIAPDE-1974):
+its already-connected DataManager. The OpcConnectionPool (and its CertProvider, which
+downloads each server's client certificate from MinIO) belongs to the read path in IngestorManager (SIENTIAPDE-1974):
 reads start in prepare_ingestor, before this runtime exists, and must keep running
 if anything here fails. The runtime only borrows the pool for writes and gRPC.
 
@@ -22,6 +22,7 @@ from ingestor.adapters.grpc_server import GrpcServer
 from ingestor.adapters.nats_consumer import NatsWriteConsumer
 from ingestor.config import GatewayConfig
 from ingestor.connection.pool import OpcConnectionPool
+from ingestor.connection.servers import ServerConfigResolver
 from ingestor.engines.active_read import ActiveReadEngine
 from ingestor.engines.write import WriteEngine
 from ingestor.io.data_sink import DataSink
@@ -61,6 +62,13 @@ class GatewayRuntime(SientiaMonitoring):
         )
         self.sink = DataSink(data_manager, postgres)
 
+        # Write commands carry only the server_id (SIENTIAPDE-1975): the connection is
+        # resolved here, from the live pooled connection or the connectors →
+        # opc_certificates documents, with the rule the orchestrator uses for the read
+        # slots (SIENTIAPDE-1974 D1).
+        self.servers = ServerConfigResolver(
+            self.pool, data_manager.mongo_repository, logger, metadata=self.metadata
+        )
         self.write_engine = WriteEngine(
             self.pool,
             self.sink,
@@ -70,6 +78,7 @@ class GatewayRuntime(SientiaMonitoring):
             write_timestamp=config.write_timestamp,
             write_quality=config.write_quality,
             min_pause_s=config.write_min_pause_s,
+            servers=self.servers,
         )
         self.nats_consumer = NatsWriteConsumer(
             config.nats_url,
@@ -110,6 +119,11 @@ class GatewayRuntime(SientiaMonitoring):
 
     async def idle_sweep(self) -> None:
         await self.pool.release_idle(self.config.idle_conn_max_s)
+        if self._nats_started and self.nats_consumer.closed:
+            # Reconnection is unbounded once connected; a closed connection means it
+            # was closed for good (e.g. by the server), so subscribe again.
+            self.logger.warning('GatewayRuntime: NATS connection closed; starting again')
+            self._nats_started = False
         if not self._nats_started:
             await self._start_nats()
 

ingestor/io/data_sink.py

@@ -1,6 +1,7 @@
+import asyncio
 import re
 
-from sqlalchemy import text
+from sqlalchemy import Select, Update, case, column, select, table, update
 
 import ingestor.metrics as metrics
 
@@ -8,6 +9,34 @@ import ingestor.metrics as metrics
 # match before a trailing '\n', which is not what we want for a SQL identifier.
 _IDENTIFIER_RE = re.compile(r'\w+', re.ASCII)
 
+# Worst-wins precedence of predictions.write_status (SIENTIAPDE-1975), from best to worst;
+# NULL, 'pending' and unknown values rank 0. The gateway is the only writer that updates the
+# column: the Laborious only sets the value of the rows it inserts ('pending', PI outcome).
+STATUS_RANK = {'ok': 1, 'error': 2, 'session_bad': 3}
+
+
+def write_status_statements(
+    schema: str, table_name: str, status: str, timestamp, model_id
+) -> tuple[Update, Select]:
+    """The UPDATE that records `status` unless the row has a worse one, and the SELECT that
+    tells whether the row exists. Built with SQLAlchemy Core: identifiers and values are
+    never formatted into the SQL text."""
+    predictions = table(
+        table_name,
+        column('write_status'),
+        column('timestamp'),
+        column('model_id'),
+        schema=schema,
+    )
+    row = (predictions.c.timestamp == timestamp) & (predictions.c.model_id == model_id)
+    current_rank = case(STATUS_RANK, value=predictions.c.write_status, else_=0)
+    return (
+        update(predictions)
+        .where(row, current_rank <= STATUS_RANK.get(status, 0))
+        .values(write_status=status),
+        select(1).select_from(predictions).where(row),
+    )
+
 
 class DataSink:
     """Gateway egress wrapper: fans a reading out to every configured name/collection."""
@@ -23,14 +52,16 @@ class DataSink:
     async def publish_reading(self, tag: str, value, timestamp, names: list[dict]) -> None:
         """Publish one MongoDB document per name entry and record its tag value metric.
 
-        `model_id` is only written when the name carries one: names converted from a
-        slot v1 have none, and their documents must stay identical to the legacy ones."""
+        Every name of the slot v2 carries collection, tag_name and model_id (parse_slot
+        checks it)."""
         for name in names:
-            doc = {'tag': tag, 'name': name['tag_name']}
-            if 'model_id' in name:
-                doc['model_id'] = name['model_id']
-            doc['timestamp'] = timestamp
-            doc['value'] = value
+            doc = {
+                'tag': tag,
+                'name': name['tag_name'],
+                'model_id': name['model_id'],
+                'timestamp': timestamp,
+                'value': value,
+            }
             await self.data_manager.publish(name['collection'], doc)
             await self.emit_metric(
                 metric_object=metrics.OPC_TAG_VALUE,
@@ -39,37 +70,76 @@ class DataSink:
                 tags={
                     'pod_id': self.data_manager.pod_id,
                     'tag_name': name['tag_name'],
-                    'model_id': name.get('model_id', ''),
+                    'model_id': name['model_id'],
                 },
             )
 
-    async def update_write_status(self, cmd: dict, result) -> None:  # NOSONAR
-        """UPDATE the per-model prediction row's write_status (C3), by timestamp+model_id.
+    async def update_write_status(self, cmd: dict, result) -> str:
+        """Record a write outcome on the prediction row (C3), keeping the worst one.
 
-        schema/table_name are per-model SQL identifiers, not values — they can't be
-        bound params, so they're checked against an allowlist before interpolation.
-        status/timestamp/model_id ARE bound params, never f-stringed.
+        The row is found by timestamp + model_id. A prediction is written to several
+        tags, each answered by its own command, so a better outcome never replaces a
+        worse one: ``ok`` < ``error`` < ``session_bad``, and ``pending``/NULL are
+        replaced by any answer, so the order of the answers does not matter. Only the
+        gateway updates the column (SIENTIAPDE-1975): the Laborious publishes the commands
+        and sets ``pending`` (merged with its PI Web API outcome) on the row it inserts.
+
+        Never raises: a failure here must not reach the NATS callback nor hide the OPC
+        write itself, which already happened. The outcome is logged and counted instead.
+
+        Return:
+            str: ``updated``, ``kept`` (the row already has a worse outcome),
+            ``row_not_found`` (e.g. the Laborious INSERT failed), ``invalid`` (bad or
+            missing schema/table/timestamp/model_id) or ``error``.
         """
-        schema = self._validate_identifier(cmd['schema'])
-        table = self._validate_identifier(cmd['table_name'])
-        # schema/table are allowlist-validated identifiers above; status/timestamp/model_id
-        # are bound params below, never interpolated — not a real injection vector.
-        stmt = text(
-            f'UPDATE {schema}.{table} SET write_status = :status '  # noqa: S608 # nosec B608
-            'WHERE timestamp = :timestamp AND model_id = :model_id'
+        try:
+            schema = self._validate_identifier(cmd.get('schema'))
+            table = self._validate_identifier(cmd.get('table_name'))
+            if cmd.get('timestamp') is None or cmd.get('model_id') is None:
+                raise ValueError('missing timestamp or model_id')
+        except ValueError as e:
+            return await self._status_outcome('invalid', cmd, f'write_status not recorded: {e}')
+
+        statements = write_status_statements(
+            schema, table, result.status, cmd['timestamp'], cmd['model_id']
         )
-        params = {
-            'status': result.status,
-            'timestamp': cmd['timestamp'],
-            'model_id': cmd['model_id'],
-        }
-        # ponytail: sync engine call inside async def — same pattern sientia_do's own
-        # Postgres activities use (e.g. export_data_to_postgres); no async pg driver here.
+        try:
+            # The engine is synchronous (no async pg driver here): run it off the event
+            # loop, which also serves the subscriptions of the read path.
+            outcome = await asyncio.to_thread(self._update_sync, *statements)
+        except Exception as e:
+            return await self._status_outcome('error', cmd, f'write_status UPDATE failed: {e}')
+        detail = {
+            'updated': f'write_status={result.status} recorded',
+            'kept': f'write_status={result.status} not recorded: the row has a worse outcome',
+            'row_not_found': f'write_status={result.status} not recorded: no prediction row',
+        }[outcome]
+        return await self._status_outcome(outcome, cmd, detail)
+
+    def _update_sync(self, update_stmt: Update, exists_stmt: Select) -> str:
         with self.postgres.engine.begin() as connection:
-            connection.execute(stmt, params)
+            if connection.execute(update_stmt).rowcount:
+                return 'updated'
+            row = connection.execute(exists_stmt).first()
+        return 'kept' if row is not None else 'row_not_found'
+
+    async def _status_outcome(self, outcome: str, cmd: dict, detail: str) -> str:
+        message = (
+            f'{detail} (server={cmd.get("server_id")} addr={cmd.get("addr")} '
+            f'model_id={cmd.get("model_id")} timestamp={cmd.get("timestamp")})'
+        )
+        if outcome in ('updated', 'kept'):
+            self.data_manager.logger.debug(message)
+        else:
+            self.data_manager.logger.warning(message)
+        await self.emit_metric(
+            metric_object=metrics.OPC_WRITE_STATUS_UPDATE_TOTAL,
+            tags={'pod_id': self.data_manager.pod_id, 'result': outcome},
+        )
+        return outcome
 
     @staticmethod
-    def _validate_identifier(name: str) -> str:
+    def _validate_identifier(name: object) -> str:
         if not isinstance(name, str) or not _IDENTIFIER_RE.fullmatch(name):
             raise ValueError(f'invalid SQL identifier: {name!r}')
         return name

ingestor/io/slots.py

@@ -1,28 +1,27 @@
-"""Parse the slot written by the Orchestrator to Redis into the slot v2 structure (contract C1).
+"""Parse the slot v2 the Orchestrator writes to Redis (contract C1, SIENTIAPDE-1974).
 
-Both shapes are accepted so the gateway and the Orchestrator can be deployed in
-any order (SIENTIAPDE-1974):
-
-- v2: each tag carries `names` (one entry per collection / tag name / model);
-- v1 (legacy): each tag carries `tag_name` + `topics`, and the server carries the
-  file-based `cert_path`. v1 is converted to v2: one `names` entry per topic, and
-  `use_security` is true when the server had a certificate path (the certificate
-  itself now always comes from MinIO via CertProvider, never from that path).
+Each server carries its connection (`url`, `subscription_period_ms`), its OPC security
+(`security`: mode + policy) and its client certificate (`certificate`: certificate_id,
+application_uri and the MinIO objects of the cert and the private key; None = connect
+without security, D5). Each tag carries `names`: one entry per collection / tag name /
+model the reading is written to.
 """
 
-SERVER_FIELDS = ('server_id', 'url', 'server_uri', 'use_security', 'subscription_period_ms')
+SERVER_FIELDS = ('server_id', 'url', 'subscription_period_ms', 'security', 'certificate')
 # What a connection/subscription needs (the pool keys clients by server_id itself).
 CONNECTION_FIELDS = tuple(f for f in SERVER_FIELDS if f != 'server_id')
+NAME_FIELDS = ('collection', 'tag_name', 'model_id')
 
 
 def _parse_names(address: str, tag: dict) -> list[dict]:
     names = tag.get('names')
-    if names:
-        return names
-    topics = tag.get('topics')
-    if topics and tag.get('tag_name'):
-        return [{'collection': topic, 'tag_name': tag['tag_name']} for topic in topics]
-    raise ValueError(f'tag {address!r} has no names')
+    if not names:
+        raise ValueError(f'tag {address!r} has no names')
+    for name in names:
+        missing = [field for field in NAME_FIELDS if field not in name]
+        if missing:
+            raise ValueError(f'tag {address!r} has a name without {missing}')
+    return names
 
 
 def _parse_tag(address: str, tag: dict) -> dict:
@@ -34,14 +33,19 @@ def _parse_tag(address: str, tag: dict) -> dict:
 
 
 def _parse_server(name: str, server: dict) -> dict:
-    parsed = {field: server[field] for field in SERVER_FIELDS if field in server}
+    if not isinstance(server.get('security'), dict):
+        raise ValueError(f'server {name!r} has no security')
+    parsed = {field: server.get(field) for field in SERVER_FIELDS}
     parsed['name'] = server.get('name', name)
-    if 'use_security' not in parsed:
-        parsed['use_security'] = bool(server.get('cert_path'))
     parsed['tags'] = {addr: _parse_tag(addr, tag) for addr, tag in server['tags'].items()}
     return parsed
 
 
 def parse_slot(raw: dict) -> dict:
-    """Normalize a raw slot (v1 or v2) into `{server_name: {..., tags: {addr: tag}}}`."""
+    """Normalize a raw slot v2 into `{server_name: {..., tags: {addr: tag}}}`.
+
+    Raises:
+        KeyError | TypeError | ValueError: When the slot is not a valid slot v2 (e.g. a
+            slot v1 with `topics`).
+    """
     return {name: _parse_server(name, server) for name, server in raw.items()}

ingestor/managers/ingestor_manager.py

@@ -1,6 +1,5 @@
 import traceback
 from copy import deepcopy
-from os import getenv
 
 from sientia_do.notifications.handlers import CoreNotificationHandler as NotificationHandler
 from sientia_do.notifications.models import NotificationLevel
@@ -15,16 +14,9 @@ from ingestor.connection.pool import OpcConnectionPool
 from ingestor.io.data_sink import DataSink
 from ingestor.io.slots import parse_slot
 from ingestor.managers.data_manager import DataManager
-from ingestor.managers.opc_manager import OpcManager
 from ingestor.managers.resource_manager import ResourceManager
 from ingestor.managers.server_reader import ServerReader
 
-# READ_ENGINE (SIENTIAPDE-1974): 'v2' = ServerReader/SubscriptionEngine over the shared
-# pool, certificate from MinIO (default). 'legacy' = plan B: the untouched rc3
-# OpcManager, slot v1 only, certificate files from the Gitea clone (cert_path). Kept
-# for validation until the end of 1974; remove once unused and the tests pass without it.
-READ_ENGINES = ('v2', 'legacy')
-
 
 class IngestorManager(SientiaMonitoring):
     """
@@ -85,7 +77,6 @@ class IngestorManager(SientiaMonitoring):
         notification_handler: NotificationHandler,
         metrics_controller: MetricsController,
         gateway_config: GatewayConfig | None = None,
-        read_engine: str | None = None,
     ):
         redis_host: str = redis_data['host']
         redis_port: int = int(redis_data['port'])
@@ -108,12 +99,12 @@ class IngestorManager(SientiaMonitoring):
             metrics_controller=metrics_controller,
         )
         self.opc_managers: dict = {}
-        # Read path (SIENTIAPDE-1974): pooled connections with the certificate from
-        # MinIO. Nothing here touches the network at construction (MinIO is lazy), so
-        # a MinIO/Postgres/NATS outage can't stop the reads before they start.
+        # Read path (SIENTIAPDE-1974): pooled connections with each server's client
+        # certificate (named in the slot) from MinIO. Nothing here touches the network at
+        # construction (MinIO is lazy), so a MinIO/Postgres/NATS outage can't stop the
+        # reads before they start.
         config = gateway_config or GatewayConfig.certs_from_env()
         self.cert_provider = build_cert_provider(
-            mongo=self.data_manager.mongo_repository,
             config=config,
             logger=logger,
             notification_handler=notification_handler,
@@ -124,10 +115,6 @@ class IngestorManager(SientiaMonitoring):
             self.cert_provider, logger, notification_handler, metrics_controller
         )
         self.sink = DataSink(self.data_manager, None)
-        self._server_cert_warned: set[str] = set()
-        self.read_engine = self._resolve_read_engine(
-            read_engine or getenv('READ_ENGINE') or 'v2', logger
-        )
         self.resource_manager = ResourceManager(
             host=redis_host,
             port=redis_port,
@@ -147,32 +134,7 @@ class IngestorManager(SientiaMonitoring):
 
         self.metadata = metadata
 
-    @staticmethod
-    def _resolve_read_engine(value: str, logger: Logger) -> str:
-        engine = value.strip().lower()
-        if engine not in READ_ENGINES:
-            logger.warning(f'Invalid READ_ENGINE {value!r}, using v2 ({"|".join(READ_ENGINES)})')
-            engine = 'v2'
-        logger.info(f'OPC read engine: {engine}')
-        return engine
-
-    def _build_reader(self, server_config: dict) -> ServerReader | OpcManager:
-        if self.read_engine == 'legacy':
-            # Plan B: the rc3 OpcManager, unchanged (own connection, cert files).
-            return OpcManager(
-                name=server_config['name'],
-                url=server_config['url'],
-                subscription_period_ms=server_config['subscription_period_ms'],
-                data_manager=self.data_manager,
-                logger=self.logger,
-                server_uri=server_config['server_uri'],
-                notification_handler=self.notification_handler,
-                metadata=self.metadata,
-                cert_path=server_config.get('cert_path'),
-                private_key_path=server_config.get('private_key_path'),
-                server_cert_path=server_config.get('server_cert_path'),
-                metrics_controller=self.metrics_controller,
-            )
+    def _build_reader(self, server_config: dict) -> ServerReader:
         return ServerReader(
             name=server_config['name'],
             server_config=server_config,
@@ -184,24 +146,21 @@ class IngestorManager(SientiaMonitoring):
             metadata=self.metadata,
         )
 
-    async def initialize_opc_from_config(
-        self, server_config: dict
-    ) -> ServerReader | OpcManager | None:
+    async def initialize_opc_from_config(self, server_config: dict) -> ServerReader | None:
         """
         Initializes the reader of one OPC UA server from its slot configuration.
 
-        With READ_ENGINE=v2 (default) creates a ServerReader and opens its pooled
-        connection; with `use_security`, the certificate comes from MinIO. With
-        READ_ENGINE=legacy (plan B) creates the rc3 OpcManager, which connects on its
-        own with the certificate files of the slot v1 (`cert_path`).
+        Creates a ServerReader and opens its pooled connection. With a `certificate`, the
+        connection uses the slot's security mode and policy, with the certificate
+        downloaded from MinIO; without one, it connects without security (D5).
 
         Args:
-            server_config (dict): The server configuration of the slot, without the
-                'tags' key. v2: 'name', 'server_id', 'url', 'server_uri',
-                'use_security', 'subscription_period_ms'. legacy: the slot v1 fields.
+            server_config (dict): The server configuration of the slot v2, without the
+                'tags' key: 'name', 'server_id', 'url', 'subscription_period_ms',
+                'security' and 'certificate'.
 
         Returns:
-            ServerReader | OpcManager | None: The connected reader, or None if the
+            ServerReader | None: The connected reader, or None if the
             connection fails (logged and notified, so the other servers keep working).
         """
 
@@ -254,11 +213,7 @@ class IngestorManager(SientiaMonitoring):
                 the shared connection stays for writes/gRPC until the idle sweep.
         """
         if server in self.opc_managers:
-            if self.read_engine == 'legacy':
-                # OpcManager owns its connection: shutdown always disconnects it.
-                await self.opc_managers[server].shutdown()
-            else:
-                await self.opc_managers[server].shutdown(evict=evict)
+            await self.opc_managers[server].shutdown(evict=evict)
             del self.opc_managers[server]
         for slot, _config in self.managed_tags.items():
             self.managed_tags[slot].pop(server, None)
@@ -567,14 +522,11 @@ class IngestorManager(SientiaMonitoring):
             if slot in reader.subscriptions:
                 await self._unsubscribe_reader(server, reader, slot)
 
-    async def _unsubscribe_reader(
-        self, server: str, reader: ServerReader | OpcManager, slot: str
-    ) -> None:
+    async def _unsubscribe_reader(self, server: str, reader: ServerReader, slot: str) -> None:
         """
         Unsubscribe one reader from a slot without letting a failure stop the main loop.
 
-        A failure (e.g. the legacy OpcManager, whose delete() raises when the server
-        session is gone) increments OPC_SUBSCRIPTION_ERRORS, sends
+        A failure (e.g. delete() raising when the server session is gone) increments OPC_SUBSCRIPTION_ERRORS, sends
         OPC_UNSUBSCRIBE_ERROR_<slot>:<server> and is logged. The slot is dropped from the
         reader's books, as the v2 engine already does, so it can be subscribed again.
 
@@ -584,7 +536,7 @@ class IngestorManager(SientiaMonitoring):
 
         Args:
             server (str): The reader's key in opc_managers
-            reader (ServerReader | OpcManager): The OPC reader
+            reader (ServerReader): The OPC reader
             slot (str): The slot to unsubscribe
         """
         try:
@@ -664,22 +616,17 @@ class IngestorManager(SientiaMonitoring):
 
     def parse_slot_config(self, slot: str, raw: dict) -> dict | None:
         """
-        Normalizes the raw slot read from Redis to the slot v2 structure.
+        Normalizes the raw slot read from Redis to the slot v2 structure (contract C1).
 
-        Accepts slot v1 (`topics`) and v2 (`names`), so the gateway and the
-        Orchestrator can be deployed in any order (SIENTIAPDE-1974). With
-        READ_ENGINE=legacy the raw slot v1 is kept as is (what OpcManager reads), and
-        a slot v2 is rejected.
+        Only the slot v2 is accepted (SIENTIAPDE-1974): the gateway and the Orchestrator
+        are deployed together, and a slot v1 (`topics`, no `security`) is rejected.
 
         Returns:
             dict | None: The parsed slot, or None when the config is malformed
             (logged and notified).
         """
         try:
-            if self.read_engine == 'legacy':
-                self._validate_slot_v1(raw)
-                return raw
-            parsed = parse_slot(raw)
+            return parse_slot(raw)
         except (AttributeError, KeyError, TypeError, ValueError) as e:
             self.send_notification(
                 metadata=self.metadata,
@@ -690,32 +637,6 @@ class IngestorManager(SientiaMonitoring):
             )
             self.logger.error(f'Slot {slot} has an invalid configuration: {e!r}')
             return None
-        self._warn_server_cert_path(raw)
-        return parsed
-
-    @staticmethod
-    def _validate_slot_v1(raw: dict) -> None:
-        """READ_ENGINE=legacy: OpcManager only understands slot v1 (`tag_name` +
-        `topics` per tag); a v2 slot (`names`) would fail in its datachange callback."""
-        for name, server in raw.items():
-            for address, tag in server['tags'].items():
-                if 'topics' not in tag or 'tag_name' not in tag or 'frequency' not in tag:
-                    raise ValueError(
-                        f'READ_ENGINE=legacy needs a slot v1 (tag_name + topics); '
-                        f'server {name!r}, tag {address!r} is not'
-                    )
-
-    def _warn_server_cert_path(self, raw: dict) -> None:
-        """Logs once per server when a slot still pins a server certificate
-        (`server_cert_path`): connections now come from the pool with the platform
-        certificate from MinIO, and that file-based pin is no longer applied."""
-        for name, server in raw.items():
-            if server.get('server_cert_path') and name not in self._server_cert_warned:
-                self._server_cert_warned.add(name)
-                self.logger.warning(
-                    f'OPC server {name}: server_cert_path is set in the slot but is no '
-                    f'longer applied (server certificate validation is disabled)'
-                )
 
     async def drop_slot_leases(self, ids: list[str]) -> None:
         """

ingestor/managers/opc_manager.py

@@ -1,588 +0,0 @@
-import asyncio
-import json
-import traceback
-from pathlib import Path
-
-from asyncua import Client
-from asyncua.crypto.security_policies import SecurityPolicyBasic256
-from sientia_do.notifications.handlers import CoreNotificationHandler as NotificationHandler
-from sientia_do.notifications.models import NotificationLevel
-from sientia_do.observability.logger import Logger
-from sientia_do.observability.metrics_controller import MetricsController
-from sientia_do.observability.sientia_monitoring import SientiaMonitoring
-from sientia_do.temporal.constants import DATETIME_FORMAT_WITH_TZ, OPC_TIMEZONE
-
-import ingestor.metrics as metrics
-from ingestor.managers.data_manager import DataManager
-
-
-class OpcManager(SientiaMonitoring):
-    """
-    Manages OPC UA server connections and tag subscriptions.
-
-    The OpcManager is responsible for:
-    - Establishing and maintaining secure connections to OPC UA servers
-    - Managing tag subscriptions and data collection
-    - Handling server reconnection and error recovery
-    - Processing OPC data and forwarding it to the data manager
-    - Monitoring connection health and performance metrics
-
-    The manager supports both secure and unsecured connections, with optional
-    certificate-based authentication for enhanced security.
-
-    Args:
-        name (str): Unique identifier for the OPC server
-        url (str): OPC UA server endpoint URL
-        data_manager (DataManager): Manager for data persistence and export
-        logger (Logger): Logger instance for application logging
-        server_uri (str): OPC UA server application URI
-        notification_handler (NotificationHandler): Handler for sending notifications
-        metadata (dict): Application metadata for notifications and tracking
-        cert_path (str, optional): Path to client certificate file for secure connections
-        private_key_path (str, optional): Path to client private key file
-        server_cert_path (str, optional): Path to server certificate file for validation
-
-    Attributes:
-        url (str): OPC UA server endpoint URL
-        name (str): Unique identifier for the OPC server
-        server_uri (str): OPC UA server application URI
-        data_queue (dict): Queue for buffering OPC data before processing
-        non_receive_count (int): Counter for cycles without data reception
-        client (Client): OPC UA client instance
-        cert_path (str): Path to client certificate file
-        private_key_path (str): Path to client private key file
-        server_cert_path (str): Path to server certificate file
-        nodes (dict): Dictionary of OPC node references
-        subscriptions (dict): Active OPC subscriptions
-        data_manager (DataManager): Manager for data persistence and export
-        metadata (dict): Application metadata
-    """
-
-    def __init__(
-        self,
-        name: str,
-        url: str,
-        subscription_period_ms: int,
-        data_manager: DataManager,
-        logger: Logger,
-        server_uri: str,
-        notification_handler: NotificationHandler,
-        metrics_controller: MetricsController,
-        metadata: dict,
-        cert_path: str | None = None,
-        private_key_path: str | None = None,
-        server_cert_path: str | None = None,
-    ):
-        self.url = url
-        self.name = name
-        self.server_uri = server_uri
-        self.data_queue: dict = {}
-        self.non_receive_count = 0
-        self.client: Client | None = None
-        self.subscription_period_ms = subscription_period_ms
-        self.cert_path = cert_path
-        self.private_key_path = private_key_path
-        self.server_cert_path = server_cert_path
-        self.nodes: dict = {}
-        self.subscriptions: dict = {}
-        self.data_manager = data_manager
-        self.metadata = metadata
-
-        SientiaMonitoring.__init__(
-            self,
-            logger=logger,
-            notification_handler=notification_handler,
-            metrics_controller=metrics_controller,
-        )
-
-        metrics.OPC_CONNECTION_STATUS.labels(
-            pod_id=self.pod_id, server_name=self.name, server_url=self.url
-        ).set(0)
-        metrics.OPC_TAGS_SUBSCRIBED.labels(pod_id=self.pod_id, server_name=self.name).set(0)
-
-    def __str__(self):
-        """
-        String representation of the OPC Manager.
-
-        Returns:
-            str: Human-readable representation showing server details and current state.
-        """
-        return (
-            f'OpcManager(name={self.name}, url={self.url}, server_uri={self.server_uri})\n'
-            f'nodes={self.nodes}, subscriptions={self.subscriptions}'
-        )
-
-    async def shutdown(self):
-        """
-        Comprehensive cleanup method for graceful shutdown.
-
-        This method ensures proper cleanup of all OPC UA resources:
-        - Closes active subscriptions
-        - Disconnects from the OPC server
-        - Releases allocated resources
-
-        Should be called before the application terminates to prevent resource leaks
-        and ensure clean disconnection from OPC servers.
-        """
-        try:
-            await self.disconnect()
-        except Exception as e:
-            self.logger.error(f'Error during cleanup: {e}')
-
-    async def set_security(self):
-        """
-        Configures the security settings for the OPC UA client.
-
-        This method sets up the security policy, certificates, and timeouts
-        required for establishing a secure connection with the OPC UA server.
-        It implements Basic256 security policy with certificate-based authentication.
-
-        Raises:
-            ValueError: If either the certificate path or private key path is not provided.
-
-        Security Settings:
-            - Security Policy: Basic256
-            - Secure Channel Timeout: 10,000,000 ms
-            - Session Timeout: 10,000,000 ms
-
-        The method configures:
-            - Client application URI
-            - Certificate-based authentication
-            - Server certificate validation (if provided)
-            - Connection timeouts for stability
-        """
-
-        if not all([self.cert_path, self.private_key_path]):
-            raise ValueError(
-                'Certificate and private key paths must be provided for secure connection.'
-            )
-        cert = str(Path(self.cert_path)) if self.cert_path else None
-        private_key = str(Path(self.private_key_path)) if self.private_key_path else None
-        server_cert = str(Path(self.server_cert_path)) if self.server_cert_path else None
-        assert cert is not None and private_key is not None
-
-        if self.client:
-            self.client.application_uri = self.server_uri
-            self.logger.info('Setting security...')
-            await self.client.set_security(
-                SecurityPolicyBasic256,
-                certificate=cert,
-                private_key=private_key,
-                server_certificate=server_cert,
-            )
-            self.client.secure_channel_timeout = 10000000
-            self.client.session_timeout = 10000000
-
-    async def connect(self):
-        """
-        Establishes a connection to the OPC server.
-
-        This method initializes the OPC client using the provided URL and
-        sets up security if a certificate path is specified. It then
-        attempts to connect to the server and logs the connection status.
-
-        The connection process includes:
-        1. Client initialization with server URL
-        2. Security configuration (if certificates are provided)
-        3. Connection establishment
-        4. Metrics recording for monitoring
-
-        Raises:
-            Exception: If the connection to the OPC server fails.
-
-        Metrics:
-            - OPC_CONNECTIONS_TOTAL: Incremented on connection attempt
-            - OPC_CONNECTION_STATUS: Set to 1 on successful connection
-        """
-
-        await self.emit_metric(
-            metric_object=metrics.OPC_CONNECTIONS_TOTAL,
-            tags={
-                'pod_id': self.pod_id,
-                'server_name': self.name,
-            },
-        )
-        try:
-            self.client = Client(self.url, timeout=10, watchdog_intervall=3600000)
-            assert self.client is not None  # Informa ao mypy que client não é None
-
-            self.client.name = self.pod_id
-            self.client.description = self.pod_id
-            pod_uri = self.pod_id.replace('-', ':')
-            self.client.application_uri = pod_uri
-            self.client.product_uri = pod_uri
-
-            if self.cert_path:
-                await self.set_security()
-            self.logger.info(f'Starting connection to {self.name}...')
-            await self.client.connect()
-            await self.emit_metric(
-                metric_object=metrics.OPC_CONNECTION_STATUS,
-                method='set',
-                value=1,
-                tags={
-                    'pod_id': self.pod_id,
-                    'server_name': self.name,
-                    'server_url': self.url,
-                },
-            )
-            self.logger.info(f'Connection to {self.name} successful.')
-        except Exception:
-            await self.disconnect()
-
-            raise
-
-    async def create_subscription(self, name: str):
-        """
-        Creates a subscription with the specified monitoring period.
-
-        This method establishes a subscription to monitor data changes or events
-        from the OPC UA server. If the client is not connected, an exception is raised.
-
-        Args:
-            name (str): The name identifier for the subscription
-            period (int, optional): The monitoring period in milliseconds. Defaults to 500 ms.
-
-        Raises:
-            ValueError: If the client is not connected.
-
-        Side Effects:
-            - Sets the `self.period` attribute to the specified or default period.
-            - Creates a subscription and assigns it to `self.subscriptions[name]`.
-            - Logs the creation of the subscription.
-            - Increments subscription creation metrics.
-        """
-
-        if not self.client:
-            raise ValueError('Client not connected. Call connect first.')
-        try:
-            self.subscriptions[name] = await self.client.create_subscription(
-                self.subscription_period_ms, self
-            )
-            self.logger.info(f'Subscription {name} created on {self.name}.')
-            await self.emit_metric(
-                metric_object=metrics.OPC_SUBSCRIPTIONS_CREATED,
-                method='inc',
-                value=1,
-                tags={
-                    'pod_id': self.pod_id,
-                    'server_name': self.name,
-                    'slot_name': name,
-                },
-            )
-        except Exception as e:
-            self.logger.error(f'Failed to create subscription {name} on {self.name}: {e}')
-            raise
-
-    async def subscribe(self, subscription: str, nodes: dict, collect_period: int):
-        """
-        Subscribes to a set of OPC UA nodes for data change notifications.
-
-        This method adds the specified nodes to the subscription and configures
-        their data collection rules based on the provided collection period and
-        node-specific frequency.
-
-        Args:
-            subscription (str): The name of the subscription to use
-            nodes (dict): A dictionary where keys are node identifiers and values are
-                configurations for each node. Each configuration must include a 'frequency'
-                key indicating the frequency of data collection in Hz.
-            collect_period (int): The data collection period in seconds.
-
-        Raises:
-            ValueError: If the subscription has not been created by calling
-                `create_subscription` prior to this method.
-
-        Side Effects:
-            - Updates internal node tracking and cycle rules
-            - Establishes data change monitoring for specified nodes
-            - Updates metrics for subscribed tags count
-        """
-
-        if not self.subscriptions.get(subscription):
-            raise ValueError('Subscription not created. Call create_subscription first.')
-
-        self.logger.info(f'Subscribing to {subscription} on {self.name}...')
-        self.logger.info(f'Subscribing to nodes: {nodes}')
-        assert self.client is not None  # Informa ao mypy que client não é None
-        addr_nodes = [self.client.get_node(n) for n in nodes]
-        self.logger.debug(f'Addr nodes: {addr_nodes}')
-        self.nodes.update(nodes)
-        self.logger.debug(f'Nodes: {self.nodes}')
-
-        self.collect_period = collect_period
-
-        for node, config in self.nodes.items():
-            self.nodes[node]['cycle_rule'] = {
-                'cycle_increment': collect_period * 1000 / float(config['frequency']),
-                'cycle_count': 0,
-            }
-
-        await self.subscriptions[subscription].subscribe_data_change(addr_nodes)
-
-        await self.emit_metric(
-            metric_object=metrics.OPC_TAGS_SUBSCRIBED,
-            method='set',
-            value=len(self.nodes),
-            tags={
-                'pod_id': self.pod_id,
-                'server_name': self.name,
-            },
-        )
-
-    async def unsubscribe(self, subscription: str):
-        """
-        Unsubscribes from a given subscription.
-
-        This method removes the specified subscription and cleans up associated
-        resources. It handles cases where the subscription doesn't exist gracefully.
-
-        Args:
-            subscription (str): The name of the subscription to unsubscribe from.
-
-        Logs:
-            - A warning if the specified subscription does not exist.
-            - An info message upon successful unsubscription.
-
-        Behavior:
-            - If the subscription exists, it is deleted and removed from the
-              subscriptions dictionary.
-            - If the subscription does not exist, no action is taken.
-        """
-
-        if not self.subscriptions.get(subscription):
-            self.logger.warning(f"Subscription '{subscription}' not found. Cannot unsubscribe.")
-            return
-        await self.subscriptions[subscription].delete()
-        del self.subscriptions[subscription]
-        self.logger.info(f'Unsubscribed from {subscription}.')
-
-    async def disconnection_fallback(self) -> list:
-        """
-        Tries 5 times to disconnect from the OPC UA server, with a delay of 100ms x try.
-        """
-
-        assert self.client is not None
-        error_stack = []
-        for i in range(5):
-            try:
-                self.logger.info(f'Disconnecting from OPC UA server, attempt {i + 1} of 5')
-                await self.client.disconnect()
-                return []
-            except Exception as e:
-                self.logger.error(
-                    f'Failed to disconnect from OPC UA serve  in attempt {i + 1} of 5: {e}'
-                )
-                error_stack.append(
-                    {
-                        'attempt': i + 1,
-                        'error': str(e),
-                        'traceback': traceback.format_exc(),
-                    }
-                )
-                await asyncio.sleep(0.1 * i)
-        return error_stack
-
-    async def disconnect(self):
-        """
-        Disconnects from the OPC UA server.
-
-        This method handles the disconnection process by deleting all subscriptions
-        and disconnecting the client from the OPC UA server. It logs the disconnection
-        process and handles any exceptions that may occur during cleanup.
-
-        Raises:
-            Exception: If an error occurs while deleting the subscription or disconnecting
-                       from the OPC UA server, it logs the error details.
-
-        Side Effects:
-            - Deletes all active subscriptions
-            - Disconnects the OPC client
-            - Updates connection status metrics
-            - Clears internal client reference
-        """
-
-        self.logger.warning('Disconnecting from OPC server')
-        if self.client is None:
-            self.logger.warning('Client already disconnected.')
-            return
-        try:
-            for sub in self.subscriptions:
-                await self.subscriptions[sub].delete()
-            self.logger.warning('Deleted all subscriptions.')
-        except Exception as sub_error:
-            self.logger.error(f'Failed to clean up subscription: {sub_error}')
-
-        errors = await self.disconnection_fallback()
-
-        if errors:
-            await self.send_notification_async(
-                metadata=self.metadata,
-                notification_id=f'OPC_DISCONNECTION_ERROR_{self.name}',
-                message=f'Failed to disconnect from OPC UA server {self.name} after 5 attempts',
-                block='opc_manager',
-                level=NotificationLevel.ERROR,
-                attachment_content=json.dumps(errors, indent=4),
-            )
-        else:
-            self.logger.warning('Disconnected from OPC UA server.')
-
-        del self.client
-        self.client = None
-        await self.emit_metric(
-            metric_object=metrics.OPC_CONNECTION_STATUS,
-            method='set',
-            value=0,
-            tags={
-                'pod_id': self.pod_id,
-                'server_name': self.name,
-                'server_url': self.url,
-            },
-        )
-        await self.emit_metric(
-            metric_object=metrics.OPC_TAGS_SUBSCRIBED,
-            method='set',
-            value=0,
-            tags={
-                'pod_id': self.pod_id,
-                'server_name': self.name,
-            },
-        )
-
-    async def datachange_notification(self, node, _val, data):
-        """
-        Handles data change notifications for monitored OPC UA nodes.
-
-        This method is triggered when a monitored node's value changes. It processes
-        the notification, updates internal state, and publishes the data to the
-        appropriate topics.
-
-        Args:
-            node (NodeId): The OPC UA node that triggered the data change notification.
-            _val (Any): The new value of the node (unused in this implementation).
-            data (DataChangeNotification): The data change notification object containing
-                details about the change.
-
-        Behavior:
-            - Extracts the value and source timestamp from the monitored item.
-            - Resets the cycle count for the node's cycle rule.
-            - Resets the non-receive count.
-            - Constructs a data dictionary containing the tag, tag name, timestamp, and value.
-            - Publishes the data to all topics associated with the node.
-        """
-
-        # get data value
-        monitored_item = data.monitored_item
-        value = monitored_item.Value.Value.Value
-        # source_timestamp
-        source_timestamp = monitored_item.Value.SourceTimestamp.replace(tzinfo=OPC_TIMEZONE)
-        tag = str(node)
-
-        self.logger.debug(
-            f'Data change notification received for tag:'
-            f'{tag} after {self.nodes[tag]["cycle_rule"]["cycle_count"]} cycles'
-        )
-
-        data = {
-            'tag': tag,
-            'name': self.nodes[str(node)]['tag_name'],
-            'timestamp': source_timestamp.strftime(DATETIME_FORMAT_WITH_TZ),
-            'value': value,
-        }
-
-        for topic in self.nodes[tag]['topics']:
-            await self.data_manager.publish(topic, data)
-
-        self.nodes[tag]['cycle_rule']['cycle_count'] = 0
-        self.non_receive_count = 0
-        await self.emit_metric(
-            metric_object=metrics.OPC_CYCLES_WITHOUT_DATA,
-            method='set',
-            value=0,
-            tags={
-                'pod_id': self.pod_id,
-                'server_name': self.name,
-            },
-        )
-
-    async def check_cycles(self):
-        """
-        Checks the cycle counts for all monitored nodes and sends
-        notifications if thresholds are exceeded.
-
-        This method iterates through all monitored nodes and updates their cycle counts based on
-        configured increments. If a node's cycle count exceeds a threshold (5 cycles), it triggers
-        a warning notification.
-
-        Side Effects:
-            - Updates cycle counts for all monitored nodes
-            - Sends warning notifications for nodes exceeding cycle thresholds
-        """
-        for node, config in self.nodes.items():
-            self.nodes[node]['cycle_rule']['cycle_count'] += config['cycle_rule']['cycle_increment']
-            if self.nodes[node]['cycle_rule']['cycle_count'] >= 5:
-                name = config['tag_name']
-                cycles = self.nodes[node]['cycle_rule']['cycle_count']
-                await self.send_notification_async(
-                    metadata=self.metadata,
-                    notification_id=f'TAG_{node}:{name}_LISTENNING_STOPPED',
-                    message=f'{cycles} cycles without receive from {node}:{name}',
-                    block='opc_manager',
-                    level=NotificationLevel.WARNING,
-                )
-
-    async def check_opc_listenning(self) -> bool:
-        """
-        Checks the OPC connection and triggers notifications if the connection is lost.
-
-        This method monitors the data reception health by tracking cycles without
-        data. It sends notifications at different thresholds and can trigger
-        reconnection attempts.
-
-        Returns:
-            bool: True if the connection is lost and reconnection should be attempted,
-                  False otherwise.
-
-        Side Effects:
-            - Increments non-receive count
-            - Updates metrics for cycles without data
-            - Sends warning notifications at 5 cycles
-            - Sends error notifications and triggers reconnection at 15 cycles
-        """
-
-        self.non_receive_count += 1
-        await self.emit_metric(
-            metric_object=metrics.OPC_CYCLES_WITHOUT_DATA,
-            method='set',
-            value=self.non_receive_count,
-            tags={
-                'pod_id': self.pod_id,
-                'server_name': self.name,
-            },
-        )
-        if self.non_receive_count >= 5:
-            await self.send_notification_async(
-                metadata=self.metadata,
-                notification_id=f'OPC_LISTENNING_STOPPED__{self.name}',
-                message=f'{self.non_receive_count} cycles without '
-                f'receive from OPC {self.name}. Tags: {json.dumps(self.nodes)}',
-                block='opc_manager',
-                level=NotificationLevel.ERROR,
-            )
-        if self.non_receive_count >= 15:
-            await self.emit_metric(
-                metric_object=metrics.OPC_RECONNECTIONS_TOTAL,
-                tags={
-                    'pod_id': self.pod_id,
-                    'server_name': self.name,
-                },
-            )
-            await self.send_notification_async(
-                metadata=self.metadata,
-                notification_id=f'OPC_CONNECTION_RETRY__{self.name}',
-                message=f'Retrying to connect to server {self.name}',
-                block='opc_manager',
-                level=NotificationLevel.ERROR,
-            )
-            return True
-        return False

ingestor/managers/server_reader.py

@@ -1,13 +1,11 @@
 """Per-server read path driven by the IngestorManager (SIENTIAPDE-1974).
 
-Replaces the legacy OpcManager behind the same interface the IngestorManager
-already uses (connect / create_subscription / subscribe / unsubscribe /
-check_cycles / check_opc_listenning / shutdown), so the slot lease loop is
-unchanged. The difference is underneath:
+Exposes the interface the IngestorManager drives (connect / create_subscription /
+subscribe / unsubscribe / check_cycles / check_opc_listenning / shutdown):
 
-- the connection comes from the shared OpcConnectionPool, so the certificate is
-  fetched from MinIO by CertProvider (Basic256Sha256) — never from a file shipped
-  in the Gitea repository;
+- the connection comes from the shared OpcConnectionPool, with the slot's security
+  mode and policy and the client certificate the slot names, which CertProvider
+  downloads from MinIO (no certificate: no security);
 - readings go through SubscriptionEngine, which fans each value out to every
   `names` entry of the slot v2 (one document per collection / tag name / model).
 """
@@ -58,7 +56,7 @@ class ServerReader:
 
     async def connect(self):
         """Open (or reuse) the pooled connection; raises if the server is unreachable
-        or the certificate can't be fetched, like OpcManager.connect did."""
+        or the certificate can't be fetched."""
         await self.pool.get(self.server_id, self.server_cfg)
 
     async def create_subscription(self, slot: str):

ingestor/metrics.py

@@ -156,6 +156,12 @@ OPC_WRITE_STATUS_BY_MODEL = Gauge(
     'Last write status per model (1=ok,0=error)',
     ['pod_id', 'model_id', 'status'],
 )
+OPC_WRITE_STATUS_UPDATE_TOTAL = Counter(
+    'opc_write_status_update_total',
+    'write_status UPDATEs on the prediction rows',
+    # result: updated|kept|row_not_found|invalid|error
+    ['pod_id', 'result'],
+)
 # --- Gateway: Active read metrics ---
 OPC_ACTIVE_READ_TOTAL = Counter(
     'opc_active_read_total',

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ingestor"
-version = "2.2.0-rc1"
+version = "2.0.0-rc4"
 description = "Sientia DataOps Ingestor - OPC Tag Ingestor"
 readme = "README.md"
 requires-python = ">=3.11"

scripts/generate-and-push-opc-cert.sh

@@ -1,95 +0,0 @@
-#!/usr/bin/env bash
-# Gera o certificado X.509 de cliente OPC UA do ingestor e sobe pro Gitea
-# in-cluster (repo que o pod clona no boot — nada muda no values/deployment,
-# só rollout restart depois do push).
-#
-# Uso:
-#   GITEA_PASSWORD='<senha>' ./scripts/generate-and-push-opc-cert.sh
-#
-# Vars (defaults pro cluster dev/gcp):
-#   GITEA_URL      default https://git.sientia.ai
-#   GITEA_USER     default gitea_admin
-#   GITEA_PASSWORD obrigatória
-#   GITEA_REPO     default gitea_admin/sientia-dataops-opc-ingestor
-#   BRANCH         default main
-#   APP_URI        default urn:sientia:opc-ingestor  (SAN URI do cert — TEM que
-#                  ser igual ao campo `uri` do documento na collection
-#                  OPC_servers: o ingestor usa esse valor como application_uri
-#                  e o KEPServer rejeita com BadCertificateUriInvalid se divergir)
-#   DAYS           default 3650
-#   FORCE=1        sobrescreve cert já existente no repo
-set -euo pipefail
-
-GITEA_URL=${GITEA_URL:-https://git.sientia.ai}
-GITEA_USER=${GITEA_USER:-gitea_admin}
-GITEA_PASSWORD=${GITEA_PASSWORD:?export GITEA_PASSWORD com a senha do Gitea}
-GITEA_REPO=${GITEA_REPO:-gitea_admin/sientia-dataops-opc-ingestor}
-BRANCH=${BRANCH:-main}
-APP_URI=${APP_URI:-urn:sientia:opc-ingestor}
-CN=${CN:-sientia-opc-ingestor}
-DAYS=${DAYS:-3650}
-
-CERT_DIR_IN_REPO="certs"
-CERT_NAME="opc-ingestor-cert.pem"
-KEY_NAME="opc-ingestor-key.pem"
-DER_NAME="opc-ingestor-cert.der"
-
-workdir=$(mktemp -d)
-trap 'rm -rf "$workdir"' EXIT
-
-echo ">> Gerando certificado (CN=${CN}, URI=${APP_URI}, ${DAYS} dias)..."
-# Extensões exigidas por OPC UA (KEPServer valida keyUsage/EKU e o SAN URI).
-openssl req -x509 -newkey rsa:2048 -sha256 -nodes -days "$DAYS" \
-  -keyout "$workdir/$KEY_NAME" -out "$workdir/$CERT_NAME" \
-  -subj "/CN=${CN}/O=Aignosi/OU=Sientia" \
-  -addext "subjectAltName=URI:${APP_URI},DNS:${CN}" \
-  -addext "keyUsage=critical,digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment" \
-  -addext "extendedKeyUsage=critical,clientAuth,serverAuth" \
-  -addext "basicConstraints=critical,CA:FALSE"
-
-# Cópia DER — só referência p/ import manual no KEPServer (o trust normal é
-# aceitar o cert rejeitado na 1ª conexão).
-openssl x509 -in "$workdir/$CERT_NAME" -outform der -out "$workdir/$DER_NAME"
-
-echo ">> Clonando ${GITEA_REPO}@${BRANCH} do Gitea..."
-encoded_pass=$(python3 - <<EOF
-import urllib.parse; print(urllib.parse.quote('''${GITEA_PASSWORD}''', safe=''))
-EOF
-)
-clone_url="${GITEA_URL%%/}"
-clone_url="${clone_url/:\/\//:\/\/${GITEA_USER}:${encoded_pass}@}/${GITEA_REPO}.git"
-git clone --depth 1 --branch "$BRANCH" "$clone_url" "$workdir/repo" 2>&1 | sed "s#${encoded_pass}#***#g"
-
-if [[ -f "$workdir/repo/$CERT_DIR_IN_REPO/$CERT_NAME" && "${FORCE:-0}" != "1" ]]; then
-  echo "ERRO: $CERT_DIR_IN_REPO/$CERT_NAME já existe no repo. Re-rodar com FORCE=1 sobrescreve" >&2
-  echo "      (o cert antigo deixa de valer — retrust no KEPServer necessário)." >&2
-  exit 1
-fi
-
-mkdir -p "$workdir/repo/$CERT_DIR_IN_REPO"
-cp "$workdir/$CERT_NAME" "$workdir/$KEY_NAME" "$workdir/$DER_NAME" "$workdir/repo/$CERT_DIR_IN_REPO/"
-
-cd "$workdir/repo"
-git add "$CERT_DIR_IN_REPO"
-git -c user.name="opc-cert-script" -c user.email="platform@aignosi.com.br" \
-  commit -m "Add OPC UA client certificate for KEPServer connection (URI ${APP_URI})"
-git push origin "$BRANCH" 2>&1 | sed "s#${encoded_pass}#***#g"
-
-fingerprint=$(openssl x509 -in "$CERT_DIR_IN_REPO/$CERT_NAME" -noout -fingerprint -sha1)
-
-cat <<EOF
-
-== PRONTO ==
-Cert no Gitea: ${GITEA_REPO}@${BRANCH} -> ${CERT_DIR_IN_REPO}/{${CERT_NAME},${KEY_NAME},${DER_NAME}}
-${fingerprint}
-
-Paths dentro do pod (o entrypoint clona o repo em /app):
-  cert_path:        /app/${CERT_DIR_IN_REPO}/${CERT_NAME}
-  private_key_path: /app/${CERT_DIR_IN_REPO}/${KEY_NAME}
-
-Próximos passos:
-  1. Documento na collection OPC_servers com "uri": "${APP_URI}" (ver README/PR).
-  2. kubectl -n sientia rollout restart deployment/opc-ingestor   # re-clona o repo
-  3. Pod tenta conectar -> no KEPServer: OPC UA Configuration Manager ->
-     Trusted Clients -> cert '${CN}' rejeitado -> Trust -> restart runtime.
-EOF

scripts/seed_e2e.py

@@ -3,7 +3,7 @@
 Populates:
   1. Mongo `connectors` + `connector_tags` (each tag with `metadata: {}` — required
      for documentation, per the feature owner).
-  2. Redis slot v1 (`slot:opc_tags:1`) so the legacy subscription path collects them.
+  2. Redis slot v2 (`slot:opc_tags:1`, contract C1) so the subscription path collects them.
   3. Postgres test predictions table (with the `write_status` column) + one row the
      NATS write test can UPDATE (correlated by timestamp + model_id).
 
@@ -37,41 +37,53 @@ except ImportError:
 
 OPC_URL = os.getenv('OPC_SERVER_URL', 'opc.tcp://localhost:49320')
 
+# No client certificate: the gateway connects without security (SIENTIAPDE-1974 D5).
+# To test security, point client_certificate_id at an active client certificate of
+# opc_certificates and set mode/policy (e.g. SignAndEncrypt / Basic256Sha256).
+SECURITY = {'mode': 'None', 'policy': 'None', 'client_certificate_id': None}
+
 CONNECTOR = {
-    'connector_id': 9001,
+    'connector_id': 'e2e-kepserver',
     'type': 'opc_ua',
     'name': 'e2e-kepserver',
     'url': OPC_URL,
-    'server_uri': 'urn:e2e:gateway:test',
     'subscription_period_ms': 1000,
-    'use_security': False,  # round 1; flip after certs are in sientia-dataops
+    'security': SECURITY,
     'status': 'Active',
     'description': 'OPC Gateway e2e validation connector (SIENTIAPDE-1972)',
 }
 
 # metadata: {} on every tag — feature owner asked test tags to carry the field.
 TAGS = [
-    {'tag_id': 9101, 'connector_id': 9001, 'address': 'ns=2;s=Simulation Examples.Functions.Ramp1',
+    {'tag_id': 9101, 'connector_id': 'e2e-kepserver', 'address': 'ns=2;s=Simulation Examples.Functions.Ramp1',
      'tag_name': 'e2e_ramp', 'category': 'Read', 'data_type': 'float', 'unit': '-',
      'status': 'Active', 'description': 'e2e read tag', 'metadata': {}},
-    {'tag_id': 9102, 'connector_id': 9001, 'address': 'ns=2;s=Simulation Examples.Functions.User1',
+    {'tag_id': 9102, 'connector_id': 'e2e-kepserver', 'address': 'ns=2;s=Simulation Examples.Functions.User1',
      'tag_name': 'e2e_write_target', 'category': 'Write', 'data_type': 'float', 'unit': '-',
      'status': 'Active', 'description': 'e2e write tag', 'metadata': {}},
 ]
 
-# Slot v1 (legacy path the gateway still runs until SIENTIAPDE-1974 delivers slot v2).
-SLOT_V1 = {
+# Slot v2 (contract C1), as the Orchestrator writes it for the connector above.
+SLOT = {
     'e2e-kepserver': {
         'name': 'e2e-kepserver',
-        'server_id': '9001',
+        'server_id': 'e2e-kepserver',
         'url': OPC_URL,
-        'server_uri': 'urn:e2e:gateway:test',
         'subscription_period_ms': 1000,
+        'security': {'mode': SECURITY['mode'], 'policy': SECURITY['policy']},
+        'certificate': None,
         'tags': {
             TAGS[0]['address']: {
-                'tag_name': TAGS[0]['tag_name'],
-                'frequency': '5000',
-                'topics': ['raw_e2e-gateway-test'],
+                'frequency': 5000,
+                'active_read': {'enabled': False},
+                'names': [
+                    {
+                        'collection': 'raw_e2e-gateway-test',
+                        'tag_name': TAGS[0]['tag_name'],
+                        'model_id': '9001',
+                        'model_name': 'e2e-model',
+                    }
+                ],
             }
         },
     }
@@ -94,12 +106,11 @@ ON CONFLICT DO NOTHING;
 
 # The matching NATS write command for the row above (paste into `nats pub opc.write '...'`).
 NATS_CMD = {
-    'msg_id': 'e2e-0001', 'server_id': '9001', 'addr': TAGS[1]['address'],
+    'msg_id': 'e2e-0001', 'server_id': 'e2e-kepserver', 'addr': TAGS[1]['address'],
     'value': 42.5, 'data_type': 'float', 'tag_type': 'prediction',
     'model_id': '9001', 'model_name': 'e2e-model',
     'schema': 'sientia_data', 'table_name': 'e2e_predictions',
     'timestamp': '2026-07-15T12:00:00Z',
-    'url': OPC_URL, 'use_security': False, 'server_uri': 'urn:e2e:gateway:test',
 }
 
 
@@ -119,8 +130,8 @@ def seed_redis():
     import redis
 
     r = redis.Redis(host=os.getenv('REDIS_HOST', 'localhost'), port=int(os.getenv('REDIS_PORT', '6379')))
-    r.set('slot:opc_tags:1', json.dumps(SLOT_V1))
-    print('redis: wrote slot:opc_tags:1 (v1)')
+    r.set('slot:opc_tags:1', json.dumps(SLOT))
+    print('redis: wrote slot:opc_tags:1 (v2)')
 
 
 def seed_postgres():
@@ -144,7 +155,7 @@ def main() -> int:
     if args.print_only:
         print('connector:', json.dumps(CONNECTOR, indent=2))
         print('tags:', json.dumps(TAGS, indent=2))
-        print('slot v1:', json.dumps(SLOT_V1, indent=2))
+        print('slot v2:', json.dumps(SLOT, indent=2))
         print('sql:', PREDICTIONS_SQL)
         print('nats write cmd:', json.dumps(NATS_CMD, indent=2))
         return 0

tests/unit/adapters/test_nats_consumer.py

@@ -1,3 +1,5 @@
+import asyncio
+import inspect
 import json
 from unittest.mock import AsyncMock, MagicMock, patch
 
@@ -37,20 +39,99 @@ async def test_handle_invalid_json_drops_message_and_logs_error():
     consumer.logger.error.assert_called_once()
 
 
+CLIENT = 'ingestor.adapters.nats_consumer.Client'
+
+
+def _fake_nc(connect=None):
+    nc = MagicMock()
+    nc.connect = connect or AsyncMock()
+    nc.subscribe = AsyncMock()
+    nc.close = AsyncMock()
+    nc.drain = AsyncMock()
+    nc.is_closed = False
+    return nc
+
+
+async def _hang(*_args, **_kwargs):
+    await asyncio.sleep(10)
+
+
 @pytest.mark.asyncio
 async def test_start_subscribes_with_subject_and_queue():
     consumer = make_consumer(subject='opc.write', queue='gateway-writers')
-    mock_nc = AsyncMock()
-    mock_nc.subscribe = AsyncMock()
+    nc = _fake_nc()
 
-    with patch('ingestor.adapters.nats_consumer.nats') as mock_nats:
-        mock_nats.connect = AsyncMock(return_value=mock_nc)
+    with patch(CLIENT, return_value=nc):
         await consumer.start()
 
-    mock_nats.connect.assert_awaited_once_with(servers='nats://localhost:4222')
-    mock_nc.subscribe.assert_awaited_once_with(
-        'opc.write', queue='gateway-writers', cb=consumer._handle
-    )
+    kwargs = nc.connect.await_args.kwargs
+    assert kwargs['servers'] == 'nats://localhost:4222'
+    # Reconnection is unbounded once connected: an outage never closes it for good.
+    assert kwargs['max_reconnect_attempts'] == -1
+    nc.subscribe.assert_awaited_once_with('opc.write', queue='gateway-writers', cb=consumer._handle)
+    assert consumer.closed is False
+
+
+@pytest.mark.asyncio
+async def test_start_gives_up_after_the_connect_timeout():
+    """start() runs on the main loop: an unreachable NATS must not block it."""
+    consumer = make_consumer(connect_timeout_s=0.1)
+    nc = _fake_nc(connect=AsyncMock(side_effect=_hang))
+    # Outer guard: fails the test instead of hanging it if start() ignores its timeout.
+    start = asyncio.wait_for(consumer.start(), timeout=5)
+
+    with patch(CLIENT, return_value=nc):
+        with pytest.raises(TimeoutError):
+            await start
+
+    nc.close.assert_awaited_once()
+    assert consumer.closed is True
+
+
+@pytest.mark.asyncio
+async def test_start_closes_the_connection_when_subscribe_fails():
+    consumer = make_consumer()
+    nc = _fake_nc()
+    nc.subscribe.side_effect = RuntimeError('permissions violation')
+
+    with patch(CLIENT, return_value=nc), pytest.raises(RuntimeError):
+        await consumer.start()
+
+    nc.close.assert_awaited_once()
+    assert consumer.closed is True
+
+
+@pytest.mark.asyncio
+async def test_handle_unexpected_engine_error_is_logged_not_raised():
+    consumer = make_consumer()
+    consumer.write_engine.write.side_effect = RuntimeError('bug')
+    msg = MagicMock(data=json.dumps({'server_id': '1'}).encode())
+
+    await consumer._handle(msg)
+
+    assert 'bug' in consumer.logger.error.call_args.args[0]
+
+
+@pytest.mark.asyncio
+async def test_connection_events_are_logged_in_one_line():
+    consumer = make_consumer()
+    nc = _fake_nc()
+    with patch(CLIENT, return_value=nc):
+        await consumer.start()
+    callbacks = nc.connect.await_args.kwargs
+    # nats-py raises InvalidCallbackTypeError on connect for a non-coroutine callback.
+    for name in ('error_cb', 'disconnected_cb', 'reconnected_cb'):
+        assert inspect.iscoroutinefunction(callbacks[name]), name
+
+    await callbacks['error_cb'](OSError('connection refused'))
+    await callbacks['disconnected_cb']()
+    await callbacks['reconnected_cb']()
+
+    warnings = [c.args[0] for c in consumer.logger.warning.call_args_list]
+    assert warnings == [
+        'NATS error: OSError connection refused',
+        'Disconnected from NATS; reconnecting',
+    ]
 
 
 @pytest.mark.asyncio
@@ -63,14 +144,28 @@ async def test_stop_before_start_does_not_raise():
 @pytest.mark.asyncio
 async def test_stop_after_start_drains_and_unsubscribes():
     consumer = make_consumer()
-    mock_nc = AsyncMock()
-    mock_sub = AsyncMock()
-    mock_nc.subscribe = AsyncMock(return_value=mock_sub)
+    nc = _fake_nc()
+    sub = AsyncMock()
+    nc.subscribe = AsyncMock(return_value=sub)
 
-    with patch('ingestor.adapters.nats_consumer.nats') as mock_nats:
-        mock_nats.connect = AsyncMock(return_value=mock_nc)
+    with patch(CLIENT, return_value=nc):
         await consumer.start()
         await consumer.stop()
 
-    mock_sub.unsubscribe.assert_awaited_once()
-    mock_nc.drain.assert_awaited_once()
+    sub.unsubscribe.assert_awaited_once()
+    nc.drain.assert_awaited_once()
+
+
+@pytest.mark.asyncio
+async def test_stop_on_a_connection_closed_for_good_does_not_raise():
+    consumer = make_consumer()
+    nc = _fake_nc()
+    with patch(CLIENT, return_value=nc):
+        await consumer.start()
+    nc.is_closed = True
+    consumer.sub.unsubscribe = AsyncMock(side_effect=RuntimeError('connection closed'))
+
+    await consumer.stop()  # no raise
+
+    nc.drain.assert_not_awaited()
+    assert consumer.closed is True

tests/unit/conftest.py

@@ -1,13 +1,13 @@
 from unittest.mock import AsyncMock, MagicMock
 
-from pytest import fixture
+import pytest
 
 
-@fixture
+@pytest.fixture
 def reader_double():
     """Factory of OPC reader doubles holding subscriptions for the given slots.
 
-    Like the real readers (ServerReader, OpcManager), `subscriptions` is keyed by slot
+    Like the real ServerReader, `subscriptions` is keyed by slot
     and `unsubscribe` drops the slot from it.
     """
 

tests/unit/connection/test_certificates.py

@@ -0,0 +1,67 @@
+import pytest
+
+from ingestor.connection.certificates import (
+    client_certificate_id,
+    has_security,
+    resolve_certificate,
+)
+
+LOOKED_UP = {
+    'certificate_id': 1,
+    'application_uri': 'urn:sientia:opc-ingestor',
+    'cert': {'bucket': 'opc-certificates', 'object_name': 'client/p/cert.pem'},
+    'private_key': {'bucket': 'opc-certificates', 'object_name': 'client/p/key.pem'},
+}
+
+
+def _connector(certificates, certificate_id=1):
+    return {
+        'connector_id': 'kepserver',
+        'url': 'opc.tcp://kep:49320',
+        'security': {
+            'mode': 'SignAndEncrypt',
+            'policy': 'Basic256Sha256',
+            'client_certificate_id': certificate_id,
+        },
+        'certificates': certificates,
+    }
+
+
+def test_resolve_certificate_keeps_only_the_slot_fields():
+    looked_up = {
+        **LOOKED_UP,
+        'thumbprint_sha1': 'ab',
+        'cert': {**LOOKED_UP['cert'], 'sha256': None},
+    }
+
+    assert resolve_certificate(_connector([looked_up])) == LOOKED_UP
+
+
+@pytest.mark.parametrize(
+    'certificates',
+    [
+        pytest.param(None, id='no-lookup'),
+        pytest.param([], id='not-found'),
+        pytest.param([{**LOOKED_UP, 'application_uri': ''}], id='no-application-uri'),
+        pytest.param([{**LOOKED_UP, 'cert': None}], id='no-cert'),
+        pytest.param([{**LOOKED_UP, 'private_key': {'object_name': 'k'}}], id='key-no-bucket'),
+        pytest.param(['not a document'], id='not-a-document'),
+    ],
+)
+def test_resolve_certificate_without_a_usable_certificate(certificates):
+    assert resolve_certificate(_connector(certificates)) is None
+
+
+def test_client_certificate_id():
+    assert client_certificate_id(_connector([])) == 1
+    assert client_certificate_id(_connector([], certificate_id=None)) is None
+    assert client_certificate_id({'connector_id': 'x'}) is None
+    assert client_certificate_id({'security': None}) is None
+
+
+def test_has_security():
+    assert has_security(_connector([]))
+    assert has_security({'security': {'mode': 'None', 'policy': 'None'}})
+    assert not has_security({'security': {'mode': 'Sign'}})
+    assert not has_security({'security': None})
+    assert not has_security({})

tests/unit/connection/test_certs.py

@@ -1,136 +1,109 @@
-from unittest.mock import AsyncMock
+import asyncio
+from unittest.mock import AsyncMock, MagicMock
 
 import pytest
 
-from ingestor.connection.certs import CertProvider
+from ingestor.connection.certs import CertificateNotConfiguredError, CertProvider, LazyMinio
 
 
-@pytest.mark.asyncio
-async def test_fetch_returns_local_paths(tmp_path):
-    mongo = AsyncMock()
-    mongo.find_one.return_value = {
-        'client_cert': {'minio_object_name': 'platform/certificates/client_cert_c.pem'},
-        'private_key': {'minio_object_name': 'platform/certificates/private_key_k.pem'},
+def certificate(certificate_id=1, name='sientia-platform', ext='pem', bucket='opc-certificates'):
+    """A client certificate as the slot v2 carries it."""
+    return {
+        'certificate_id': certificate_id,
+        'application_uri': 'urn:sientia:opc-ingestor',
+        'cert': {'bucket': bucket, 'object_name': f'client/{name}/cert.{ext}'},
+        'private_key': {'bucket': bucket, 'object_name': f'client/{name}/key.{ext}'},
     }
-    minio = AsyncMock()
-    minio.download_file.side_effect = [b'cert-bytes', b'key-bytes']
-
-    provider = CertProvider(mongo, minio, cache_dir=str(tmp_path), bucket='sientia-dataops')
-    cert, key = await provider.fetch()
-
-    assert cert.endswith('c.pem') and key.endswith('k.pem')
-    assert minio.download_file.await_count == 2
-    mongo.find_one.assert_awaited_once_with('platform_certificates', {}, {})
-    minio.download_file.assert_any_await(
-        'platform/certificates/client_cert_c.pem', 'sientia-dataops', {}
-    )
-    assert (tmp_path / 'client_cert_c.pem').read_bytes() == b'cert-bytes'
-    assert (tmp_path / 'private_key_k.pem').read_bytes() == b'key-bytes'
 
 
-@pytest.mark.asyncio
-async def test_lazy_minio_builds_client_once_on_first_download():
-    from unittest.mock import MagicMock
-
-    from ingestor.connection.certs import LazyMinio
-
-    client = AsyncMock()
-    client.download_file.return_value = b'bytes'
-    factory = MagicMock(return_value=client)
-    minio = LazyMinio(factory)
-
-    factory.assert_not_called()  # no network at construction
-    assert await minio.download_file('a.pem', 'bucket', {}) == b'bytes'
-    await minio.download_file('b.pem', 'bucket', {})
-
-    factory.assert_called_once()
-    assert client.download_file.await_count == 2
-
-
-def test_build_cert_provider_is_lazy_and_uses_config(monkeypatch):
-    from unittest.mock import MagicMock
-
-    from ingestor.config import GatewayConfig
-    from ingestor.connection import certs
-
-    minio_cls = MagicMock()
-    monkeypatch.setattr(certs, 'MinioRepository', minio_cls)
-    mongo = MagicMock()
-
-    provider = certs.build_cert_provider(
-        mongo=mongo,
-        config=GatewayConfig(cert_bucket='streamlit-connectors', cert_cache_dir='certs-cache'),
-        logger=MagicMock(),
-        notification_handler=MagicMock(),
-        metrics_controller=MagicMock(),
-    )
-
-    assert provider.mongo is mongo
-    assert provider.bucket == 'streamlit-connectors'
-    assert provider.cache_dir == 'certs-cache'
-    assert isinstance(provider.minio, certs.LazyMinio)
-    minio_cls.assert_not_called()  # MinIO is only reached when a cert is needed
-
-
-def _provider(tmp_path, doc=None, data=b'pem'):
-    from unittest.mock import MagicMock
-
-    mongo = MagicMock()
-    mongo.find_one = AsyncMock(
-        return_value=doc
-        if doc is not None
-        else {
-            'client_cert': {'minio_object_name': 'platform_certificates/c.pem'},
-            'private_key': {'minio_object_name': 'platform_certificates/k.pem'},
-        }
-    )
+def _provider(tmp_path, data=b'pem'):
     minio = MagicMock()
     minio.download_file = AsyncMock(return_value=data)
-    return CertProvider(mongo=mongo, minio=minio, cache_dir=str(tmp_path), bucket='b'), minio
+    return CertProvider(minio=minio, cache_dir=str(tmp_path)), minio
+
+
+@pytest.mark.asyncio
+async def test_fetch_downloads_the_objects_the_certificate_names(tmp_path):
+    minio = AsyncMock()
+    minio.download_file.side_effect = [b'cert-bytes', b'key-bytes']
+    provider = CertProvider(minio, cache_dir=str(tmp_path))
+
+    cert, key = await provider.fetch(certificate(7))
+
+    assert (cert, key) == (str(tmp_path / '7' / 'cert.pem'), str(tmp_path / '7' / 'key.pem'))
+    assert [c.args for c in minio.download_file.await_args_list] == [
+        ('client/sientia-platform/cert.pem', 'opc-certificates', {}),
+        ('client/sientia-platform/key.pem', 'opc-certificates', {}),
+    ]
+    assert (tmp_path / '7' / 'cert.pem').read_bytes() == b'cert-bytes'
+    assert (tmp_path / '7' / 'key.pem').read_bytes() == b'key-bytes'
+
+
+@pytest.mark.asyncio
+async def test_local_files_keep_the_object_extension(tmp_path):
+    """asyncua picks the format (.pem / .der) by the extension"""
+    provider, _minio = _provider(tmp_path)
+
+    cert, key = await provider.fetch(certificate(ext='der'))
+
+    assert cert.endswith('cert.der')
+    assert key.endswith('key.der')
 
 
 @pytest.mark.asyncio
 @pytest.mark.parametrize(
-    'doc',
+    'broken',
     [
-        None,
-        {},
-        {'client_cert': {'minio_object_name': 'c.pem'}},
-        {'client_cert': None, 'private_key': {}},
+        pytest.param({'cert': None}, id='no-cert'),
+        pytest.param({'private_key': {'bucket': 'b'}}, id='key-without-object'),
+        pytest.param({'certificate_id': None}, id='no-id'),
+        pytest.param({'certificate_id': '../etc'}, id='id-not-an-integer'),
     ],
 )
-async def test_fetch_without_platform_certificate_raises_clear_error(tmp_path, doc):
-    from ingestor.connection.certs import CertificateNotConfiguredError
-
+async def test_fetch_of_an_incomplete_certificate_raises_clear_error(tmp_path, broken):
     provider, minio = _provider(tmp_path)
-    provider.mongo.find_one.return_value = doc
+    incomplete = {**certificate(), **broken}
 
-    with pytest.raises(CertificateNotConfiguredError, match='platform_certificates'):
-        await provider.fetch()
+    with pytest.raises(CertificateNotConfiguredError, match='opc_certificates|certificate_id'):
+        await provider.fetch(incomplete)
     minio.download_file.assert_not_awaited()
 
 
 @pytest.mark.asyncio
-async def test_fetch_is_cached_until_invalidated(tmp_path):
+async def test_fetch_is_cached_per_certificate_until_invalidated(tmp_path):
     provider, minio = _provider(tmp_path)
 
-    first = await provider.fetch()
-    second = await provider.fetch()
-    assert first == second
+    first = await provider.fetch(certificate(1))
+    assert await provider.fetch(certificate(1)) == first
     assert minio.download_file.await_count == 2  # cert + key, once
 
-    provider.invalidate()
-    await provider.fetch()
+    other = await provider.fetch(certificate(2, name='other'))
+    assert other != first
     assert minio.download_file.await_count == 4
 
+    provider.invalidate(1)
+    await provider.fetch(certificate(1))
+    await provider.fetch(certificate(2, name='other'))  # still cached
+    assert minio.download_file.await_count == 6
+
+
+@pytest.mark.asyncio
+async def test_a_replaced_certificate_is_downloaded_again(tmp_path):
+    """Same certificate_id, other objects in the slot: the cache does not serve the old files"""
+    provider, minio = _provider(tmp_path)
+
+    await provider.fetch(certificate(1, name='old'))
+    await provider.fetch(certificate(1, name='new'))
+
+    assert minio.download_file.await_count == 4
+    assert minio.download_file.await_args_list[-1].args[0] == 'client/new/key.pem'
+
 
 @pytest.mark.asyncio
 async def test_concurrent_fetches_download_once(tmp_path):
-    import asyncio
-
     provider, minio = _provider(tmp_path)
 
-    results = await asyncio.gather(*(provider.fetch() for _ in range(5)))
+    results = await asyncio.gather(*(provider.fetch(certificate()) for _ in range(5)))
 
     assert len(set(results)) == 1
     assert minio.download_file.await_count == 2
@@ -138,11 +111,51 @@ async def test_concurrent_fetches_download_once(tmp_path):
 
 @pytest.mark.asyncio
 async def test_download_replaces_file_atomically(tmp_path):
-    provider, minio = _provider(tmp_path, data=b'new')
-    (tmp_path / 'c.pem').write_bytes(b'old')
+    provider, _minio = _provider(tmp_path, data=b'new')
+    (tmp_path / '1').mkdir()
+    (tmp_path / '1' / 'cert.pem').write_bytes(b'old')
 
-    cert_path, _ = await provider.fetch()
+    cert_path, _ = await provider.fetch(certificate(1))
 
-    assert (tmp_path / 'c.pem').read_bytes() == b'new'
-    assert cert_path == str(tmp_path / 'c.pem')
-    assert not list(tmp_path.glob('*.tmp'))
+    assert (tmp_path / '1' / 'cert.pem').read_bytes() == b'new'
+    assert cert_path == str(tmp_path / '1' / 'cert.pem')
+    assert not list(tmp_path.glob('**/*.tmp'))
+
+
+@pytest.mark.asyncio
+async def test_lazy_minio_builds_one_client_per_bucket_on_first_download():
+    clients = {'a': AsyncMock(), 'b': AsyncMock()}
+    for client in clients.values():
+        client.download_file.return_value = b'bytes'
+    factory = MagicMock(side_effect=lambda bucket: clients[bucket])
+    minio = LazyMinio(factory)
+
+    factory.assert_not_called()  # no network at construction
+    assert await minio.download_file('x.pem', 'a', {}) == b'bytes'
+    await minio.download_file('y.pem', 'a', {})
+    await minio.download_file('z.pem', 'b', {})
+
+    assert [c.args for c in factory.call_args_list] == [('a',), ('b',)]
+    assert clients['a'].download_file.await_count == 2
+    clients['b'].download_file.assert_awaited_once_with('z.pem', 'b', {})
+
+
+def test_build_cert_provider_is_lazy_and_uses_config(monkeypatch):
+    from ingestor.config import GatewayConfig
+    from ingestor.connection import certs
+
+    minio_cls = MagicMock()
+    monkeypatch.setattr(certs, 'MinioRepository', minio_cls)
+
+    provider = certs.build_cert_provider(
+        config=GatewayConfig(cert_cache_dir='certs-cache'),
+        logger=MagicMock(),
+        notification_handler=MagicMock(),
+        metrics_controller=MagicMock(),
+    )
+
+    assert provider.cache_dir == 'certs-cache'
+    assert isinstance(provider.minio, certs.LazyMinio)
+    minio_cls.assert_not_called()  # MinIO is only reached when a cert is needed
+    provider.minio._factory('opc-certificates')
+    assert minio_cls.call_args.kwargs['bucket'] == 'opc-certificates'

tests/unit/connection/test_clock.py

@@ -5,4 +5,6 @@ def test_elapsed_is_non_negative_and_monotonic():
     clk = MonotonicClock()
     a = clk.elapsed()
     b = clk.elapsed()
-    assert a >= 0 and b >= a  # never negative even if wall clock jumps back
+    # never negative even if wall clock jumps back
+    assert a >= 0
+    assert b >= a

tests/unit/connection/test_pool.py

@@ -6,6 +6,15 @@ import pytest
 import ingestor.metrics as metrics
 from ingestor.connection.pool import OpcConnectionPool
 
+SECURITY = {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'}
+CERTIFICATE = {
+    'certificate_id': 7,
+    'application_uri': 'urn:sientia:opc-ingestor',
+    'cert': {'bucket': 'opc-certificates', 'object_name': 'client/p/cert.pem'},
+    'private_key': {'bucket': 'opc-certificates', 'object_name': 'client/p/key.pem'},
+}
+SECURED = {'url': 'opc.tcp://h:4840', 'security': SECURITY, 'certificate': CERTIFICATE}
+
 
 def make_pool(cert_provider=None):
     pool = OpcConnectionPool(
@@ -23,8 +32,8 @@ async def test_get_reuses_connection_per_server():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        c1 = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
-        c2 = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        c1 = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
+        c2 = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     assert c1 is c2
     assert fake_client.connect.await_count == 1
@@ -37,7 +46,7 @@ async def test_get_failure_records_metric_and_raises():
     fake_client.connect.side_effect = RuntimeError('boom')
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
         with pytest.raises(RuntimeError):
-            await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+            await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     pool.emit_metric.assert_any_await(
         metric_object=metrics.OPC_CONNECTIONS_FAILED,
@@ -48,13 +57,13 @@ async def test_get_failure_records_metric_and_raises():
     # a later successful get must reconnect (failed attempt was not cached)
     fake_client2 = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client2):
-        client = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        client = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
     assert client is fake_client2
     assert fake_client2.connect.await_count == 1
 
 
 @pytest.mark.asyncio
-async def test_get_applies_security_when_requested():
+async def test_get_applies_the_certificate_and_security_of_the_server():
     cert_provider = AsyncMock()
     cert_provider.fetch.return_value = ('cert.pem', 'key.pem')
     pool = make_pool(cert_provider=cert_provider)
@@ -63,13 +72,32 @@ async def test_get_applies_security_when_requested():
         patch('ingestor.connection.pool.Client', return_value=fake_client),
         patch('ingestor.connection.pool.apply_security', new=AsyncMock()) as mock_apply,
     ):
-        await pool.get(
-            'srv',
-            {'url': 'opc.tcp://h:4840', 'server_uri': 'urn:test', 'use_security': True},
-        )
+        await pool.get('srv', SECURED)
 
-    cert_provider.fetch.assert_awaited_once()
-    mock_apply.assert_awaited_once_with(fake_client, 'urn:test', 'cert.pem', 'key.pem', None)
+    cert_provider.fetch.assert_awaited_once_with(CERTIFICATE)
+    mock_apply.assert_awaited_once_with(fake_client, SECURITY, CERTIFICATE, 'cert.pem', 'key.pem')
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    'cfg',
+    [
+        pytest.param({**SECURED, 'certificate': None}, id='no-certificate'),
+        pytest.param({**SECURED, 'security': {'mode': 'None', 'policy': 'None'}}, id='mode-none'),
+    ],
+)
+async def test_get_without_certificate_connects_without_security(cfg):
+    """No client certificate (or mode None): no security, no MinIO download (D5)"""
+    cert_provider = AsyncMock()
+    pool = make_pool(cert_provider=cert_provider)
+    with (
+        patch('ingestor.connection.pool.Client', return_value=AsyncMock()),
+        patch('ingestor.connection.pool.apply_security', new=AsyncMock()) as mock_apply,
+    ):
+        await pool.get('srv', cfg)
+
+    cert_provider.fetch.assert_not_awaited()
+    mock_apply.assert_not_awaited()
 
 
 @pytest.mark.asyncio
@@ -77,7 +105,7 @@ async def test_release_idle_evicts_and_disconnects():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     client, _ = pool._connections['srv']
     pool._connections['srv'] = (client, -1000.0)  # force idle
@@ -97,7 +125,7 @@ async def test_release_idle_keeps_recently_used():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     await pool.release_idle(max_idle_s=3600)
 
@@ -110,8 +138,8 @@ async def test_close_all_disconnects_everything():
     pool = make_pool()
     fake_a, fake_b = AsyncMock(), AsyncMock()
     with patch('ingestor.connection.pool.Client', side_effect=[fake_a, fake_b]):
-        await pool.get('a', {'url': 'opc.tcp://h:4840', 'use_security': False})
-        await pool.get('b', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('a', {'url': 'opc.tcp://h:4840', 'certificate': None})
+        await pool.get('b', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     await pool.close_all()
 
@@ -126,7 +154,7 @@ async def test_release_idle_disconnect_retries_then_gives_up():
     fake_client = AsyncMock()
     fake_client.disconnect.side_effect = RuntimeError('nope')
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     client, _ = pool._connections['srv']
     pool._connections['srv'] = (client, -1000.0)
@@ -142,7 +170,7 @@ async def test_success_path_emits_connection_metrics():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     pool.emit_metric.assert_any_await(
         metric_object=metrics.OPC_CONNECTIONS_TOTAL,
@@ -170,7 +198,7 @@ async def test_concurrent_get_same_server_connects_once():
 
     clients[0].connect.side_effect = slow_connect
 
-    cfg = {'url': 'opc.tcp://h:4840', 'use_security': False}
+    cfg = {'url': 'opc.tcp://h:4840', 'certificate': None}
     with patch('ingestor.connection.pool.Client', side_effect=clients) as ctor:
         t1 = asyncio.create_task(pool.get('srv', cfg))
         await started.wait()  # t1 is mid-connect, holding the per-server lock
@@ -191,7 +219,7 @@ async def test_release_idle_keeps_pinned_connection():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     pool.pin('srv')
     await pool.release_idle(max_idle_s=0)
@@ -205,7 +233,7 @@ async def test_release_idle_evicts_after_last_unpin():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     pool.pin('srv')
     pool.pin('srv')  # two slots subscribed on the same server
@@ -233,9 +261,9 @@ async def test_evict_disconnects_now_and_next_get_reconnects():
     pool = make_pool()
     first, second = AsyncMock(), AsyncMock()
     with patch('ingestor.connection.pool.Client', side_effect=[first, second]):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
         await pool.evict('srv')
-        client = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        client = await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     first.disconnect.assert_awaited_once()
     assert client is second
@@ -255,8 +283,8 @@ async def test_get_normalizes_server_id_to_str():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        c1 = await pool.get(1, {'url': 'opc.tcp://h:4840', 'use_security': False})
-        c2 = await pool.get('1', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        c1 = await pool.get(1, {'url': 'opc.tcp://h:4840', 'certificate': None})
+        c2 = await pool.get('1', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     assert c1 is c2
     assert fake_client.connect.await_count == 1
@@ -267,8 +295,8 @@ async def test_get_with_different_cfg_reuses_client_and_warns():
     pool = make_pool()
     fake_client = AsyncMock()
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
-        client = await pool.get('srv', {'url': 'opc.tcp://10.0.0.5:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
+        client = await pool.get('srv', {'url': 'opc.tcp://10.0.0.5:4840', 'certificate': None})
 
     assert client is fake_client
     assert fake_client.connect.await_count == 1
@@ -276,6 +304,21 @@ async def test_get_with_different_cfg_reuses_client_and_warns():
     assert 'url' in pool.logger.warning.call_args.args[0]
 
 
+@pytest.mark.asyncio
+async def test_get_with_another_certificate_warns():
+    cert_provider = AsyncMock()
+    cert_provider.fetch.return_value = ('cert.pem', 'key.pem')
+    pool = make_pool(cert_provider=cert_provider)
+    with (
+        patch('ingestor.connection.pool.Client', return_value=AsyncMock()),
+        patch('ingestor.connection.pool.apply_security', new=AsyncMock()),
+    ):
+        await pool.get('srv', SECURED)
+        await pool.get('srv', {**SECURED, 'certificate': {**CERTIFICATE, 'certificate_id': 8}})
+
+    assert 'certificate' in pool.logger.warning.call_args.args[0]
+
+
 @pytest.mark.asyncio
 async def test_secured_connect_failure_invalidates_cert_cache():
     from unittest.mock import MagicMock
@@ -290,9 +333,9 @@ async def test_secured_connect_failure_invalidates_cert_cache():
         patch('ingestor.connection.pool.apply_security', AsyncMock()),
     ):
         with pytest.raises(RuntimeError):
-            await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': True})
+            await pool.get('srv', SECURED)
 
-    cert_provider.invalidate.assert_called_once()
+    cert_provider.invalidate.assert_called_once_with(7)
 
 
 @pytest.mark.asyncio
@@ -305,7 +348,7 @@ async def test_insecure_connect_failure_keeps_cert_cache():
     fake_client.connect.side_effect = RuntimeError('unreachable')
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
         with pytest.raises(RuntimeError):
-            await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+            await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     cert_provider.invalidate.assert_not_called()
 
@@ -317,7 +360,7 @@ async def test_connect_identifies_the_pod_on_the_server():
     pool = make_pool()
     fake_client = MagicMock(connect=AsyncMock())
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'use_security': False})
+        await pool.get('srv', {'url': 'opc.tcp://h:4840', 'certificate': None})
 
     assert fake_client.name == pool.pod_id
     assert fake_client.description == pool.pod_id
@@ -326,19 +369,35 @@ async def test_connect_identifies_the_pod_on_the_server():
 
 
 @pytest.mark.asyncio
-async def test_connect_server_uri_still_wins_for_application_uri():
-    pool = make_pool()
-    fake_client = MagicMock(connect=AsyncMock())
+async def test_connect_with_certificate_uses_its_application_uri():
+    """apply_security sets the certificate's URI: it must match its SAN URI
+    (BadCertificateUriInvalid otherwise)"""
+    cert_provider = AsyncMock()
+    cert_provider.fetch.return_value = ('cert.pem', 'key.pem')
+    pool = make_pool(cert_provider=cert_provider)
+    fake_client = MagicMock(connect=AsyncMock(), set_security=AsyncMock())
     with patch('ingestor.connection.pool.Client', return_value=fake_client):
-        await pool.get(
-            'srv',
-            {
-                'url': 'opc.tcp://h:4840',
-                'use_security': False,
-                'server_uri': 'urn:sientia:opc-ingestor',
-            },
-        )
+        await pool.get('srv', SECURED)
 
-    # must match the certificate SAN URI (BadCertificateUriInvalid otherwise)
     assert fake_client.application_uri == 'urn:sientia:opc-ingestor'
     assert fake_client.name == pool.pod_id
+
+
+@pytest.mark.asyncio
+async def test_config_for_returns_a_copy_of_the_live_connection_settings():
+    pool = make_pool()
+    cfg = {'url': 'opc.tcp://h:4840', 'security': SECURITY, 'certificate': None}
+    with patch('ingestor.connection.pool.Client', return_value=AsyncMock()):
+        await pool.get(1, cfg)
+
+    found = pool.config_for('1')
+    assert found == cfg
+    found['url'] = 'mutated'
+    assert pool.config_for(1)['url'] == 'opc.tcp://h:4840'
+
+    await pool.evict('1')
+    assert pool.config_for('1') is None
+
+
+def test_config_for_unknown_server_is_none():
+    assert make_pool().config_for('missing') is None

tests/unit/connection/test_security.py

@@ -1,18 +1,92 @@
 from unittest.mock import AsyncMock, MagicMock
 
 import pytest
+from asyncua import ua
+from asyncua.crypto.security_policies import (
+    SecurityPolicyAes128Sha256RsaOaep,
+    SecurityPolicyBasic256Sha256,
+)
 
-from ingestor.connection.security import SESSION_TIMEOUT_MS, apply_security
+from ingestor.connection.security import (
+    SESSION_TIMEOUT_MS,
+    UnsupportedSecurityError,
+    apply_security,
+    is_secured,
+)
+
+CERTIFICATE = {'certificate_id': 1, 'application_uri': 'urn:sientia:opc-ingestor'}
+
+
+def _client():
+    client = MagicMock()
+    client.set_security = AsyncMock()
+    return client
 
 
 @pytest.mark.asyncio
-async def test_apply_security_sets_sha256_and_channel_lifetime():
-    client = MagicMock()
-    client.set_security = AsyncMock()
-    await apply_security(client, 'urn:x', 'c.pem', 'k.pem', None)
+async def test_apply_security_uses_mode_and_policy_and_channel_lifetime():
+    client = _client()
+
+    await apply_security(
+        client, {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'}, CERTIFICATE, 'c', 'k'
+    )
+
+    assert client.application_uri == 'urn:sientia:opc-ingestor'
     assert client.session_timeout == SESSION_TIMEOUT_MS
     assert client.secure_channel_timeout == int(SESSION_TIMEOUT_MS * 1.25)
     args, kwargs = client.set_security.call_args
-    from asyncua.crypto.security_policies import SecurityPolicyBasic256Sha256
-
     assert args[0] is SecurityPolicyBasic256Sha256
+    assert kwargs == {
+        'certificate': 'c',
+        'private_key': 'k',
+        'server_certificate': None,
+        'mode': ua.MessageSecurityMode.SignAndEncrypt,
+    }
+
+
+@pytest.mark.asyncio
+async def test_apply_security_sign_with_another_policy():
+    client = _client()
+
+    await apply_security(
+        client, {'mode': 'Sign', 'policy': 'Aes128Sha256RsaOaep'}, CERTIFICATE, 'c', 'k'
+    )
+
+    args, kwargs = client.set_security.call_args
+    assert args[0] is SecurityPolicyAes128Sha256RsaOaep
+    assert kwargs['mode'] == ua.MessageSecurityMode.Sign
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    'security',
+    [
+        {'mode': 'Encrypt', 'policy': 'Basic256Sha256'},
+        {'mode': 'Sign', 'policy': 'Basic999'},
+        {'mode': 'Sign', 'policy': 'None'},
+        {},
+    ],
+)
+async def test_apply_security_rejects_unknown_mode_or_policy(security):
+    client = _client()
+
+    with pytest.raises(UnsupportedSecurityError):
+        await apply_security(client, security, CERTIFICATE, 'c', 'k')
+    client.set_security.assert_not_awaited()
+
+
+@pytest.mark.parametrize(
+    ('server_cfg', 'secured'),
+    [
+        ({'security': {'mode': 'SignAndEncrypt'}, 'certificate': CERTIFICATE}, True),
+        ({'security': {'mode': 'Sign'}, 'certificate': CERTIFICATE}, True),
+        # No certificate: no security (D5), whatever the mode
+        ({'security': {'mode': 'SignAndEncrypt'}, 'certificate': None}, False),
+        ({'security': {'mode': 'SignAndEncrypt'}}, False),
+        # mode None: no security even with a certificate
+        ({'security': {'mode': 'None'}, 'certificate': CERTIFICATE}, False),
+        ({}, False),
+    ],
+)
+def test_is_secured(server_cfg, secured):
+    assert is_secured(server_cfg) is secured

tests/unit/connection/test_servers.py

@@ -0,0 +1,149 @@
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from ingestor.connection.servers import CONNECTORS, ServerConfigResolver
+
+LOOKED_UP = {
+    'certificate_id': 1,
+    'application_uri': 'urn:sientia:opc-ingestor',
+    'cert': {'bucket': 'opc-certificates', 'object_name': 'client/p/cert.pem'},
+    'private_key': {'bucket': 'opc-certificates', 'object_name': 'client/p/key.pem'},
+}
+# A connector as the aggregation returns it ($lookup on opc_certificates)
+CONNECTOR = {
+    'connector_id': 'kepserver',
+    'url': 'opc.tcp://kepserver:49320',
+    'subscription_period_ms': 500,
+    'security': {
+        'mode': 'SignAndEncrypt',
+        'policy': 'Basic256Sha256',
+        'client_certificate_id': 1,
+    },
+    'certificates': [LOOKED_UP],
+}
+# The same server in the slot v2 format
+CONFIG = {
+    'server_id': 'kepserver',
+    'url': 'opc.tcp://kepserver:49320',
+    'subscription_period_ms': 500,
+    'security': {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'},
+    'certificate': LOOKED_UP,
+}
+
+
+def make_resolver(connector=None, pooled=None, ttl_s=60.0):
+    pool = MagicMock()
+    pool.config_for = MagicMock(return_value=pooled)
+    mongo = MagicMock()
+    mongo.aggregate = AsyncMock(return_value=[connector] if connector is not None else [])
+    return ServerConfigResolver(pool, mongo, MagicMock(), ttl_s=ttl_s)
+
+
+@pytest.mark.asyncio
+async def test_live_pooled_connection_wins_over_mongo():
+    """Writes reuse exactly the settings the read path opened the connection with."""
+    pooled = {'url': 'opc.tcp://read:1', 'security': CONFIG['security'], 'certificate': None}
+    resolver = make_resolver(connector=CONNECTOR, pooled=pooled)
+
+    assert await resolver.resolve('kepserver') == pooled
+    resolver.pool.config_for.assert_called_once_with('kepserver')
+    resolver.mongo.aggregate.assert_not_awaited()
+
+
+@pytest.mark.asyncio
+async def test_connector_with_its_certificate_in_the_slot_format():
+    resolver = make_resolver(connector=CONNECTOR)
+
+    assert await resolver.resolve('kepserver') == CONFIG
+    collection, pipeline, _ = resolver.mongo.aggregate.await_args.args
+    assert collection == CONNECTORS
+    assert pipeline[0] == {
+        '$match': {'connector_id': 'kepserver', 'type': 'opc_ua', 'status': 'Active'}
+    }
+    # The same rule as the orchestrator: only an active client certificate (D5)
+    [lookup] = [stage['$lookup'] for stage in pipeline if '$lookup' in stage]
+    assert lookup['from'] == 'opc_certificates'
+    assert lookup['pipeline'][0]['$match']['status'] == 'active'
+    assert lookup['pipeline'][0]['$match']['kind'] == 'client'
+    resolver.logger.warning.assert_not_called()
+
+
+@pytest.mark.asyncio
+async def test_connector_without_certificate_id_connects_without_security():
+    connector = {
+        **CONNECTOR,
+        'security': {'mode': 'None', 'policy': 'None', 'client_certificate_id': None},
+        'certificates': [],
+    }
+    resolver = make_resolver(connector=connector)
+
+    cfg = await resolver.resolve('kepserver')
+
+    assert cfg['certificate'] is None
+    assert cfg['security'] == {'mode': 'None', 'policy': 'None'}
+    resolver.logger.warning.assert_not_called()
+
+
+@pytest.mark.asyncio
+async def test_certificate_id_without_an_active_client_certificate_warns():
+    resolver = make_resolver(connector={**CONNECTOR, 'certificates': []})
+
+    cfg = await resolver.resolve('kepserver')
+
+    assert cfg == {**CONFIG, 'certificate': None}
+    assert 'client_certificate_id 1' in resolver.logger.warning.call_args.args[0]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    'connector',
+    [
+        pytest.param({**CONNECTOR, 'url': ''}, id='no-url'),
+        pytest.param({**CONNECTOR, 'security': {'mode': 'Sign'}}, id='no-policy'),
+        pytest.param({k: v for k, v in CONNECTOR.items() if k != 'security'}, id='no-security'),
+    ],
+)
+async def test_incomplete_connectors_are_not_found(connector):
+    resolver = make_resolver(connector=connector)
+
+    assert await resolver.resolve('kepserver') is None
+    resolver.logger.warning.assert_called_once()
+
+
+@pytest.mark.asyncio
+async def test_unknown_server_is_cached_briefly():
+    resolver = make_resolver(ttl_s=60.0)
+    resolver.missing_ttl_s = 10.0
+    resolver.clock = MagicMock()
+    resolver.clock.elapsed.side_effect = [0.0, 5.0, 11.0]
+
+    assert await resolver.resolve('9') is None
+    assert await resolver.resolve('9') is None  # within missing_ttl_s: no lookup
+    assert await resolver.resolve('9') is None  # expired: looked up again
+    assert resolver.mongo.aggregate.await_count == 2
+    assert resolver.logger.warning.call_count == 2
+
+
+@pytest.mark.asyncio
+async def test_lookups_are_cached_for_the_ttl():
+    resolver = make_resolver(connector=CONNECTOR, ttl_s=60.0)
+    resolver.clock = MagicMock()
+    resolver.clock.elapsed.side_effect = [0.0, 30.0, 61.0]
+
+    first = await resolver.resolve('kepserver')
+    first['url'] = 'mutated by the caller'
+    second = await resolver.resolve('kepserver')  # within the TTL: cached, and a copy
+    await resolver.resolve('kepserver')  # expired: looked up again
+
+    assert second['url'] == 'opc.tcp://kepserver:49320'
+    assert resolver.mongo.aggregate.await_count == 2
+
+
+@pytest.mark.asyncio
+async def test_mongo_errors_propagate_to_the_caller():
+    resolver = make_resolver()
+    resolver.mongo.aggregate = AsyncMock(side_effect=RuntimeError('mongo down'))
+
+    with pytest.raises(RuntimeError, match='mongo down'):
+        await resolver.resolve('kepserver')

tests/unit/engines/test_write.py

@@ -7,8 +7,22 @@ from asyncua import ua
 import ingestor.metrics as metrics
 from ingestor.engines.write import WriteEngine
 
+SERVER_CFG = {
+    'server_id': 'srv1',
+    'url': 'opc.tcp://kepserver:49320',
+    'security': {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'},
+    'certificate': None,
+}
+
+
+def make_resolver(cfg=SERVER_CFG):
+    resolver = MagicMock()
+    resolver.resolve = AsyncMock(return_value=cfg)
+    return resolver
+
 
 def make_engine(**kwargs):
+    kwargs.setdefault('servers', make_resolver())
     eng = WriteEngine(
         pool=AsyncMock(),
         sink=AsyncMock(),
@@ -257,3 +271,106 @@ async def test_no_rate_limit_when_min_pause_is_zero():
         await eng.write(base_cmd())
         await eng.write(base_cmd())
         sleep_mock.assert_not_awaited()
+
+
+# --- SIENTIAPDE-1975: server resolution (GW1) and error reporting (GW4) ---
+
+
+@pytest.mark.asyncio
+async def test_write_uses_the_resolved_server_connection():
+    """The C2 command of the Laborious carries only the server_id."""
+    eng = make_engine()
+    eng.pool.get.return_value = make_client()
+
+    await eng.write(base_cmd())
+
+    eng.servers.resolve.assert_awaited_once_with('srv1')
+    eng.pool.get.assert_awaited_once_with('srv1', SERVER_CFG)
+
+
+@pytest.mark.asyncio
+async def test_write_unknown_server_records_error_without_connecting():
+    eng = make_engine(servers=make_resolver(cfg=None))
+
+    result = await eng.write(base_cmd())
+
+    assert result.status == 'error'
+    assert result.opc_status_code == 'server_not_found'
+    eng.pool.get.assert_not_awaited()
+    eng.sink.update_write_status.assert_awaited_once_with(base_cmd(), result)
+
+
+@pytest.mark.asyncio
+async def test_write_server_lookup_failure_records_error():
+    resolver = make_resolver()
+    resolver.resolve.side_effect = RuntimeError('mongo down')
+    eng = make_engine(servers=resolver)
+
+    result = await eng.write(base_cmd())
+
+    assert result.status == 'error'
+    assert result.opc_status_code == 'server_lookup'
+    assert 'mongo down' in result.message
+    eng.pool.get.assert_not_awaited()
+
+
+@pytest.mark.asyncio
+async def test_write_ignores_a_connection_carried_by_the_command():
+    """The server is always resolved by server_id (SIENTIAPDE-1974): connection fields
+    in the command are not an override"""
+    eng = make_engine()
+    eng.pool.get.return_value = make_client()
+
+    await eng.write(base_cmd(url='opc.tcp://h:4840'))
+
+    eng.pool.get.assert_awaited_once_with('srv1', SERVER_CFG)
+
+
+@pytest.mark.asyncio
+async def test_write_without_resolver_is_server_not_found():
+    eng = make_engine(servers=None)
+
+    result = await eng.write(base_cmd(url='opc.tcp://h:4840'))
+
+    assert result.opc_status_code == 'server_not_found'
+    eng.pool.get.assert_not_awaited()
+
+
+@pytest.mark.asyncio
+async def test_write_error_logs_the_reason_and_sets_the_model_status():
+    eng = make_engine()
+    node = make_node()
+    node.write_value.side_effect = RuntimeError('BadWriteNotSupported')
+    eng.pool.get.return_value = make_client(node)
+
+    await eng.write(base_cmd())
+
+    logged = eng.logger.error.call_args.args[0]
+    assert 'RuntimeError' in logged
+    assert 'BadWriteNotSupported' in logged
+    assert 'addr=ns=2;i=5' in logged
+    assert 'model_id=m1' in logged
+    status_call = next(
+        c
+        for c in eng.emit_metric.await_args_list
+        if c.kwargs['metric_object'] is metrics.OPC_WRITE_STATUS_BY_MODEL
+    )
+    # Same series as a success, so it shows the last outcome.
+    assert status_call.kwargs['value'] == 0
+    assert status_call.kwargs['tags'] == {'pod_id': eng.pod_id, 'model_id': 'm1', 'status': 'ok'}
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize('timestamp', [None, 'not-a-timestamp'])
+async def test_write_bad_timestamp_records_error_without_raising(timestamp):
+    eng = make_engine(write_timestamp=True)
+    cmd = base_cmd(timestamp=timestamp)
+    if timestamp is None:
+        del cmd['timestamp']
+
+    result = await eng.write(cmd)
+
+    assert result.status == 'error'
+    assert result.opc_status_code == 'bad_timestamp'
+    eng.pool.get.assert_not_awaited()
+    eng.sink.update_write_status.assert_awaited_once()

tests/unit/io/test_data_sink_publish.py

@@ -67,26 +67,3 @@ async def test_publish_reading_emits_tag_value_metric_per_name():
         value=42,
         tags={'pod_id': dm.pod_id, 'tag_name': 'Counter', 'model_id': '1'},
     )
-
-
-@pytest.mark.asyncio
-async def test_publish_reading_without_model_id_keeps_legacy_doc():
-    """Names converted from a slot v1 carry no model_id: the document must be the
-    same as the legacy OpcManager one (SIENTIAPDE-1974 NF2)."""
-    sink, dm = make_sink()
-    names = [{'collection': 'raw_x', 'tag_name': 'Counter'}]
-
-    await sink.publish_reading(
-        tag='ns=2;i=2', value=42, timestamp='2026-07-13 00:00:00+0000', names=names
-    )
-
-    collection, doc = dm.publish.await_args_list[0].args
-    assert collection == 'raw_x'
-    assert doc == {
-        'tag': 'ns=2;i=2',
-        'name': 'Counter',
-        'timestamp': '2026-07-13 00:00:00+0000',
-        'value': 42,
-    }
-    _, kwargs = sink.emit_metric.await_args
-    assert kwargs['tags']['model_id'] == ''

tests/unit/io/test_data_sink_status.py

@@ -1,89 +1,150 @@
-from unittest.mock import MagicMock
+from unittest.mock import AsyncMock, MagicMock
 
 import pytest
+from sqlalchemy import create_engine, text
+from sqlalchemy.dialects import postgresql
 
+import ingestor.metrics as metrics
 from ingestor.engines.write import WriteResult
-from ingestor.io.data_sink import DataSink
+from ingestor.io.data_sink import DataSink, write_status_statements
+
+CMD = {
+    'server_id': '1',
+    'addr': 'ns=2;s=Pred',
+    'schema': 'sientia_data',
+    'table_name': 'pred_m1',
+    'model_id': '1',
+    'timestamp': '2026-07-13T00:00:00Z',
+}
 
 
-def make_sink():
+def _result(status):
+    return WriteResult(ok=status == 'ok', status=status, opc_status_code='x', message='')
+
+
+def make_sink(rowcount=1, existing_row=True):
     postgres = MagicMock()
     connection = MagicMock()
+    connection.execute.return_value.rowcount = rowcount
+    connection.execute.return_value.first.return_value = (1,) if existing_row else None
     postgres.engine.begin.return_value.__enter__.return_value = connection
-    sink = DataSink(data_manager=MagicMock(), postgres=postgres)
+    data_manager = MagicMock()
+    data_manager.emit_metric = AsyncMock()
+    sink = DataSink(data_manager=data_manager, postgres=postgres)
     return sink, connection
 
 
+def _outcome_metric(sink):
+    calls = [
+        c
+        for c in sink.data_manager.emit_metric.await_args_list
+        if c.kwargs['metric_object'] is metrics.OPC_WRITE_STATUS_UPDATE_TOTAL
+    ]
+    assert len(calls) == 1
+    return calls[0].kwargs['tags']['result']
+
+
 @pytest.mark.asyncio
-async def test_update_write_status_targets_model_row():
+async def test_update_write_status_targets_model_row_with_precedence():
     sink, connection = make_sink()
-    cmd = {
-        'schema': 'sientia_data',
-        'table_name': 'pred_m1',
-        'model_id': '1',
-        'timestamp': '2026-07-13T00:00:00Z',
-    }
-    result = WriteResult(ok=True, status='ok', opc_status_code='Good', message='')
 
-    await sink.update_write_status(cmd, result)
+    outcome = await sink.update_write_status(CMD, _result('ok'))
 
+    assert outcome == 'updated'
     assert connection.execute.call_count == 1
-    stmt, params = connection.execute.call_args[0]
-    sql = str(stmt)
-    assert 'update' in sql.lower() and 'write_status' in sql.lower()
-    assert 'sientia_data.pred_m1' in sql
+    (stmt,) = connection.execute.call_args[0]
+    compiled = stmt.compile(dialect=postgresql.dialect())
+    sql = str(compiled)
+    assert sql.startswith('UPDATE sientia_data.pred_m1 SET write_status=')
+    assert 'CASE sientia_data.pred_m1.write_status WHEN' in sql
     # values must be bound params, never interpolated into the SQL text itself
-    assert params == {'status': 'ok', 'timestamp': '2026-07-13T00:00:00Z', 'model_id': '1'}
-    assert 'ok' not in sql
     assert '2026-07-13T00:00:00Z' not in sql
+    assert "'ok'" not in sql
+    assert compiled.params['write_status'] == 'ok'
+    assert '2026-07-13T00:00:00Z' in compiled.params.values()
+    assert _outcome_metric(sink) == 'updated'
 
 
 @pytest.mark.asyncio
-async def test_update_write_status_rejects_bad_schema():
-    sink, connection = make_sink()
-    cmd = {
-        'schema': 'a; DROP TABLE x',
-        'table_name': 'pred_m1',
-        'model_id': '1',
-        'timestamp': '2026-07-13T00:00:00Z',
-    }
-    result = WriteResult(ok=False, status='error', opc_status_code='Bad', message='x')
+async def test_update_write_status_kept_when_the_row_has_a_worse_outcome():
+    sink, _ = make_sink(rowcount=0, existing_row=True)
 
-    with pytest.raises(ValueError):
-        await sink.update_write_status(cmd, result)
-
-    connection.execute.assert_not_called()
+    assert await sink.update_write_status(CMD, _result('ok')) == 'kept'
+    assert _outcome_metric(sink) == 'kept'
 
 
 @pytest.mark.asyncio
-async def test_update_write_status_rejects_trailing_newline_identifier():
-    sink, connection = make_sink()
-    cmd = {
-        'schema': 'sientia_data\n',
-        'table_name': 'pred_m1',
-        'model_id': '1',
-        'timestamp': '2026-07-13T00:00:00Z',
-    }
-    result = WriteResult(ok=True, status='ok', opc_status_code='Good', message='')
+async def test_update_write_status_row_not_found_is_logged():
+    """E.g. the Laborious INSERT failed and it wrote to OPC anyway (SIENTIAPDE-1975 D10)."""
+    sink, _ = make_sink(rowcount=0, existing_row=False)
 
-    with pytest.raises(ValueError):
-        await sink.update_write_status(cmd, result)
-
-    connection.execute.assert_not_called()
+    assert await sink.update_write_status(CMD, _result('error')) == 'row_not_found'
+    assert 'no prediction row' in sink.data_manager.logger.warning.call_args.args[0]
+    assert _outcome_metric(sink) == 'row_not_found'
 
 
 @pytest.mark.asyncio
-async def test_update_write_status_rejects_bad_table_name():
+async def test_update_write_status_database_error_is_not_raised():
     sink, connection = make_sink()
-    cmd = {
-        'schema': 'sientia_data',
-        'table_name': 'pred_m1; DROP TABLE x',
-        'model_id': '1',
-        'timestamp': '2026-07-13T00:00:00Z',
-    }
-    result = WriteResult(ok=False, status='error', opc_status_code='Bad', message='x')
+    connection.execute.side_effect = RuntimeError('column "write_status" does not exist')
 
-    with pytest.raises(ValueError):
-        await sink.update_write_status(cmd, result)
+    assert await sink.update_write_status(CMD, _result('ok')) == 'error'
+    assert 'write_status' in sink.data_manager.logger.warning.call_args.args[0]
+    assert _outcome_metric(sink) == 'error'
 
+
+@pytest.mark.parametrize(
+    ('field', 'value'),
+    [
+        ('schema', 'a; DROP TABLE x'),
+        ('schema', 'sientia_data\n'),
+        ('table_name', 'pred_m1; DROP TABLE x'),
+        ('schema', None),
+        ('timestamp', None),
+        ('model_id', None),
+    ],
+)
+@pytest.mark.asyncio
+async def test_update_write_status_rejects_invalid_commands(field, value):
+    sink, connection = make_sink()
+
+    outcome = await sink.update_write_status({**CMD, field: value}, _result('error'))
+
+    assert outcome == 'invalid'
     connection.execute.assert_not_called()
+    assert _outcome_metric(sink) == 'invalid'
+
+
+def _apply(initial, answers):
+    """Run the real statements on SQLite for each answer; return the final status."""
+    engine = create_engine('sqlite://')
+    with engine.begin() as connection:
+        connection.execute(
+            text('CREATE TABLE pred (model_id INTEGER, timestamp TEXT, write_status TEXT)')
+        )
+        connection.execute(
+            text('INSERT INTO pred VALUES (1, :ts, :status)'),
+            {'ts': '2026-07-13T00:00:00Z', 'status': initial},
+        )
+        for answer in answers:
+            update_stmt, exists_stmt = write_status_statements(
+                None, 'pred', answer, '2026-07-13T00:00:00Z', 1
+            )
+            connection.execute(update_stmt)
+            assert connection.execute(exists_stmt).first() is not None
+        return connection.execute(text('SELECT write_status FROM pred')).scalar()
+
+
+@pytest.mark.parametrize(
+    ('initial', 'answers', 'expected'),
+    [
+        ('pending', ['ok'], 'ok'),
+        ('pending', ['error', 'ok'], 'error'),
+        ('pending', ['ok', 'error'], 'error'),
+        ('pending', ['session_bad', 'error', 'ok'], 'session_bad'),
+        ('error', ['ok'], 'error'),
+        (None, ['ok'], 'ok'),
+    ],
+)
+def test_update_statement_keeps_the_worst_answer(initial, answers, expected):
+    assert _apply(initial, answers) == expected

tests/unit/io/test_slots.py

@@ -1,11 +1,24 @@
-from ingestor.io.slots import parse_slot
+import pytest
+
+from ingestor.io.slots import CONNECTION_FIELDS, parse_slot
+
+SECURITY = {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'}
+CERTIFICATE = {
+    'certificate_id': 1,
+    'application_uri': 'urn:sientia:opc-ingestor',
+    'cert': {'bucket': 'opc-certificates', 'object_name': 'client/p/cert.pem'},
+    'private_key': {'bucket': 'opc-certificates', 'object_name': 'client/p/key.pem'},
+}
+NAME = {'collection': 'c', 'tag_name': 't', 'model_name': 'm', 'model_id': '1'}
 
 RAW = {
     'server1': {
         'server_id': '1',
+        'name': 'server1',
         'url': 'opc.tcp://h:4840',
-        'use_security': True,
         'subscription_period_ms': 1000,
+        'security': SECURITY,
+        'certificate': CERTIFICATE,
         'tags': {
             'ns=2;i=2': {
                 'frequency': '15000',
@@ -30,6 +43,17 @@ RAW = {
 }
 
 
+def _server(**fields):
+    server = {
+        'server_id': '1',
+        'url': 'u',
+        'security': {'mode': 'None', 'policy': 'None'},
+        'certificate': None,
+        'tags': {'ns=2;i=1': {'frequency': '1000', 'names': [NAME]}},
+    }
+    return {'s': {**server, **fields}}
+
+
 def test_parse_slot_multi_name():
     slot = parse_slot(RAW)
     tag = slot['server1']['tags']['ns=2;i=2']
@@ -37,127 +61,71 @@ def test_parse_slot_multi_name():
     assert tag['active_read']['cycle_multiple'] == 3
 
 
-def test_parse_slot_defaults_active_read_disabled():
-    raw = {
-        's': {
-            'server_id': '1',
-            'url': 'u',
-            'tags': {
-                'ns=2;i=1': {
-                    'frequency': '1000',
-                    'names': [
-                        {'collection': 'c', 'tag_name': 't', 'model_name': 'm', 'model_id': '1'}
-                    ],
-                }
-            },
-        }
+def test_parse_slot_keeps_security_and_certificate():
+    server = parse_slot(RAW)['server1']
+
+    assert server['security'] == SECURITY
+    assert server['certificate'] == CERTIFICATE
+    assert {field: server[field] for field in CONNECTION_FIELDS} == {
+        'url': 'opc.tcp://h:4840',
+        'subscription_period_ms': 1000,
+        'security': SECURITY,
+        'certificate': CERTIFICATE,
     }
-    tag = parse_slot(raw)['s']['tags']['ns=2;i=1']
-    assert tag['active_read'] == {'enabled': False}
 
 
-def test_parse_slot_rejects_tag_without_names():
-    raw = {'s': {'server_id': '1', 'url': 'u', 'tags': {'ns=2;i=1': {'frequency': '1000'}}}}
-    import pytest
-
-    with pytest.raises((KeyError, ValueError)):
-        parse_slot(raw)
-
-
-RAW_V1 = {
-    'kepserver': {
-        'name': 'kepserver',
-        'server_id': '1',
-        'url': 'opc.tcp://kepserver:49320',
-        'server_uri': 'urn:sientia:opc-ingestor',
-        'cert_path': '/app/certs/opc-ingestor-cert.pem',
-        'private_key_path': '/app/certs/opc-ingestor-key.pem',
-        'server_cert_path': None,
-        'subscription_period_ms': 15000.0,
-        'tags': {
-            'ns=2;s=A.TargetSin': {
-                'aggr_func': 'lts',
-                'frequency': 30000,
-                'server_id': '1',
-                'tag_address': 'ns=2;s=A.TargetSin',
-                'tag_name': 'TargetSin',
-                'topics': ['raw_scouter-sin-approx', 'raw_other-model'],
-            }
-        },
-    }
-}
-
-
-def test_parse_slot_converts_v1_topics_to_names():
-    """Slot v1 (current Orchestrator) is read as v2: one name per topic, no model_id,
-    so the documents stay identical to the legacy ones (SIENTIAPDE-1974 G1)."""
-    server = parse_slot(RAW_V1)['kepserver']
-    tag = server['tags']['ns=2;s=A.TargetSin']
-
-    assert tag['names'] == [
-        {'collection': 'raw_scouter-sin-approx', 'tag_name': 'TargetSin'},
-        {'collection': 'raw_other-model', 'tag_name': 'TargetSin'},
-    ]
-    assert tag['frequency'] == 30000
-    assert tag['active_read'] == {'enabled': False}
-
-
-def test_parse_slot_v1_cert_path_means_security_but_path_is_dropped():
-    server = parse_slot(RAW_V1)['kepserver']
-
-    assert server['use_security'] is True
-    assert server['name'] == 'kepserver'
-    assert server['server_id'] == '1'
-    assert server['subscription_period_ms'] == 15000.0
-    # the certificate always comes from MinIO now: no file path survives the parse
-    assert 'cert_path' not in server
-    assert 'private_key_path' not in server
-
-
-def test_parse_slot_v1_without_cert_path_is_insecure():
-    raw = {
-        's': {
-            'server_id': '2',
-            'url': 'u',
-            'cert_path': None,
-            'tags': {'ns=2;i=1': {'tag_name': 't', 'frequency': '1000', 'topics': ['raw_c']}},
-        }
-    }
+def test_parse_slot_without_certificate_key_means_no_certificate():
+    raw = _server()
+    del raw['s']['certificate']
 
     server = parse_slot(raw)['s']
 
-    assert server['use_security'] is False
+    assert server['certificate'] is None
     assert server['name'] == 's'  # falls back to the slot key
 
 
-def test_parse_slot_v2_explicit_use_security_wins():
-    raw = {
-        's': {
-            'server_id': '1',
-            'url': 'u',
-            'use_security': False,
-            'cert_path': '/legacy/path',
-            'tags': {
-                'ns=2;i=1': {
-                    'frequency': '1000',
-                    'names': [{'collection': 'c', 'tag_name': 't', 'model_id': '1'}],
+def test_parse_slot_defaults_active_read_disabled():
+    tag = parse_slot(_server())['s']['tags']['ns=2;i=1']
+    assert tag['active_read'] == {'enabled': False}
+
+
+@pytest.mark.parametrize(
+    'raw',
+    [
+        pytest.param(_server(tags={'ns=2;i=1': {'frequency': '1000'}}), id='tag-without-names'),
+        pytest.param(
+            _server(tags={'ns=2;i=1': {'frequency': '1', 'names': [{'collection': 'c'}]}}),
+            id='name-without-tag-name',
+        ),
+        pytest.param(
+            _server(
+                tags={
+                    'ns=2;i=1': {
+                        'frequency': '1',
+                        'names': [{'collection': 'c', 'tag_name': 't'}],
+                    }
+                }
+            ),
+            id='name-without-model-id',
+        ),
+        pytest.param(_server(security=None), id='no-security'),
+        # Slot v1 (an Orchestrator before SIENTIAPDE-1974): no longer accepted
+        pytest.param(
+            {
+                'kepserver': {
+                    'server_id': '1',
+                    'url': 'opc.tcp://kepserver:49320',
+                    'server_uri': 'urn:sientia:opc-ingestor',
+                    'cert_path': '/app/certs/opc-ingestor-cert.pem',
+                    'tags': {
+                        'ns=2;s=A': {'tag_name': 'A', 'frequency': 30000, 'topics': ['raw_x']}
+                    },
                 }
             },
-        }
-    }
-
-    assert parse_slot(raw)['s']['use_security'] is False
-
-
-def test_parse_slot_rejects_v1_tag_without_tag_name():
-    raw = {
-        's': {
-            'server_id': '1',
-            'url': 'u',
-            'tags': {'ns=2;i=1': {'frequency': '1', 'topics': ['c']}},
-        }
-    }
-    import pytest
-
-    with pytest.raises(ValueError):
+            id='slot-v1',
+        ),
+    ],
+)
+def test_parse_slot_rejects_invalid_slots(raw):
+    with pytest.raises((KeyError, ValueError)):
         parse_slot(raw)

tests/unit/managers/test_ingestor_manager.py

@@ -9,7 +9,6 @@ from ingestor.config import GatewayConfig
 from ingestor.connection.pool import OpcConnectionPool
 from ingestor.io.slots import parse_slot
 from ingestor.managers.ingestor_manager import IngestorManager
-from ingestor.managers.opc_manager import OpcManager
 from ingestor.managers.server_reader import ServerReader
 
 metadata = {
@@ -22,17 +21,27 @@ metadata = {
 }
 
 
-def raw_slot_v1(tag_name='Counter', address='ns=2;i=2'):
-    """A slot as the current (v1) Orchestrator writes it to Redis."""
+SECURITY = {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'}
+CERTIFICATE = {
+    'certificate_id': 1,
+    'application_uri': 'urn:sientia:opc-ingestor',
+    'cert': {'bucket': 'opc-certificates', 'object_name': 'client/p/cert.pem'},
+    'private_key': {'bucket': 'opc-certificates', 'object_name': 'client/p/key.pem'},
+}
+
+
+def raw_slot(tag_name='Counter', address='ns=2;i=2'):
+    """A slot v2 as the Orchestrator writes it to Redis (contract C1)."""
+    name = {'collection': 'raw_a', 'tag_name': tag_name, 'model_id': '1', 'model_name': 'm'}
     return {
         'server1': {
             'name': 'server1',
             'server_id': '1',
             'url': 'opc.tcp://localhost:4840',
-            'server_uri': 'urn:sientia:opc-ingestor',
-            'cert_path': '/app/certs/opc-ingestor-cert.pem',
             'subscription_period_ms': 500.0,
-            'tags': {address: {'tag_name': tag_name, 'frequency': '1000', 'topics': ['raw_a']}},
+            'security': SECURITY,
+            'certificate': CERTIFICATE,
+            'tags': {address: {'frequency': '1000', 'names': [name]}},
         }
     }
 
@@ -124,7 +133,6 @@ def test___init__(
     # Read path (SIENTIAPDE-1974): pool + read sink built here, nothing connected yet.
     assert isinstance(ingestor.pool, OpcConnectionPool)
     assert ingestor.pool.cert_provider is ingestor.cert_provider
-    assert ingestor.cert_provider.mongo is data_manager_mock.return_value.mongo_repository
     assert ingestor.sink.data_manager is data_manager_mock.return_value
     assert ingestor.sink.postgres is None
 
@@ -143,10 +151,9 @@ def test___init___cert_provider_uses_gateway_config(resource_manager_mock, data_
         notification_handler=MagicMock(),
         metadata=metadata['metadata'],
         metrics_controller=MagicMock(),
-        gateway_config=GatewayConfig(cert_bucket='my-bucket', cert_cache_dir='certs-cache'),
+        gateway_config=GatewayConfig(cert_cache_dir='certs-cache'),
     )
 
-    assert ingestor.cert_provider.bucket == 'my-bucket'
     assert ingestor.cert_provider.cache_dir == 'certs-cache'
 
 
@@ -157,9 +164,9 @@ async def test_initialize_opc_from_config(server_reader, ingestor_manager):
         'name': 'server1',
         'server_id': '1',
         'url': 'opc.tcp://localhost:4840',
-        'server_uri': 'urn:sientia:opc-ingestor',
-        'use_security': True,
         'subscription_period_ms': 1000,
+        'security': SECURITY,
+        'certificate': CERTIFICATE,
     }
 
     server_reader.return_value = MagicMock(connect=AsyncMock())
@@ -191,9 +198,9 @@ async def test_initialize_opc_from_config_exception(
         'name': 'server1',
         'server_id': '1',
         'url': 'opc.tcp://localhost:4840',
-        'server_uri': 'urn:sientia:opc-ingestor',
-        'use_security': True,
         'subscription_period_ms': 1000,
+        'security': SECURITY,
+        'certificate': CERTIFICATE,
     }
 
     ingestor_manager.logger.error = MagicMock()
@@ -483,13 +490,13 @@ async def test_get_number_of_slots_empty(metrics, ingestor_manager):
 @mark.asyncio
 async def test_get_slot_leases_1_success(ingestor_manager):
     ingestor_manager.resource_manager.lease_tag = AsyncMock(return_value=True)
-    ingestor_manager.resource_manager.get_tag_slot = AsyncMock(return_value=raw_slot_v1())
+    ingestor_manager.resource_manager.get_tag_slot = AsyncMock(return_value=raw_slot())
 
     ingestor_manager.number_of_slots = 1
     result = await ingestor_manager.get_slot_leases()
 
     # Stored already normalized to slot v2 (v1 converted by parse_slot).
-    assert result == {'1': parse_slot(raw_slot_v1())}
+    assert result == {'1': parse_slot(raw_slot())}
     assert ingestor_manager.managed_tags == result
 
 
@@ -497,22 +504,22 @@ async def test_get_slot_leases_1_success(ingestor_manager):
 async def test_get_slot_leases_2_success(ingestor_manager):
     ingestor_manager.resource_manager.lease_tag = AsyncMock(side_effect=[True, True])
     ingestor_manager.resource_manager.get_tag_slot = AsyncMock(
-        side_effect=[raw_slot_v1('Counter'), raw_slot_v1('Ramp', 'ns=2;i=3')]
+        side_effect=[raw_slot('Counter'), raw_slot('Ramp', 'ns=2;i=3')]
     )
 
     ingestor_manager.number_of_slots = 2
     result = await ingestor_manager.get_slot_leases(max_slots=2)
 
     assert result == {
-        '1': parse_slot(raw_slot_v1('Counter')),
-        '2': parse_slot(raw_slot_v1('Ramp', 'ns=2;i=3')),
+        '1': parse_slot(raw_slot('Counter')),
+        '2': parse_slot(raw_slot('Ramp', 'ns=2;i=3')),
     }
 
 
 @mark.asyncio
 async def test_get_slot_leases_skips_malformed_slot(ingestor_manager):
     ingestor_manager.resource_manager.lease_tag = AsyncMock(return_value=True)
-    # tag without names/topics/frequency
+    # tag without names/frequency
     ingestor_manager.resource_manager.get_tag_slot = AsyncMock(
         return_value={'server1': {'server_id': '1', 'url': 'u', 'tags': {'ns=2;i=2': {}}}}
     )
@@ -624,8 +631,8 @@ async def test_unsubscribe_slot_reader_failure_does_not_raise(ingestor_manager,
 
 @mark.asyncio
 async def test_unsubscribe_slot_with_real_readers(ingestor_manager):
-    """Contract: both real readers keep `subscriptions` keyed by slot, and
-    unsubscribe_slot closes the slot's subscription on each of them"""
+    """Contract: the real reader keeps `subscriptions` keyed by slot, and
+    unsubscribe_slot closes the slot's subscription"""
     v2_subscription = AsyncMock()
     client = MagicMock(create_subscription=AsyncMock(return_value=v2_subscription))
     pool = MagicMock(get=AsyncMock(return_value=client), evict=AsyncMock())
@@ -646,35 +653,13 @@ async def test_unsubscribe_slot_with_real_readers(ingestor_manager):
     server_reader.engine.emit_metric = AsyncMock()
     await server_reader.create_subscription('slot1')
 
-    with patch('ingestor.managers.opc_manager.metrics'):
-        legacy_reader = OpcManager(
-            name='legacy',
-            url='opc.tcp://legacy:4840',
-            data_manager=AsyncMock(),
-            subscription_period_ms=1000,
-            logger=MagicMock(),
-            server_uri='opc.tcp://legacy:4840',
-            notification_handler=MagicMock(),
-            metadata={},
-            metrics_controller=AsyncMock(),
-        )
-    legacy_reader.emit_metric = AsyncMock()
-    legacy_subscription = AsyncMock()
-    legacy_reader.client = MagicMock(
-        create_subscription=AsyncMock(return_value=legacy_subscription)
-    )
-    await legacy_reader.create_subscription('slot1')
-
     assert 'slot1' in server_reader.subscriptions
-    assert 'slot1' in legacy_reader.subscriptions
-    ingestor_manager.opc_managers = {'kepserver': server_reader, 'legacy': legacy_reader}
+    ingestor_manager.opc_managers = {'kepserver': server_reader}
 
     await ingestor_manager.unsubscribe_slot('slot1')
 
     v2_subscription.delete.assert_awaited_once()
-    legacy_subscription.delete.assert_awaited_once()
     assert 'slot1' not in server_reader.subscriptions
-    assert 'slot1' not in legacy_reader.subscriptions
     pool.pin.assert_called_once_with('1')
     pool.unpin.assert_called_once_with('1')
 
@@ -683,13 +668,13 @@ async def test_unsubscribe_slot_with_real_readers(ingestor_manager):
 async def test_update_slot_config_unsubscribes_removed_slot(ingestor_manager, reader_double):
     """A slot deleted from Redis is unsubscribed right away, not left to the main loop"""
     ingestor_manager.managed_tags = {
-        'slot1': parse_slot(raw_slot_v1('Same')),
-        'slot2': parse_slot(raw_slot_v1('Same')),
+        'slot1': parse_slot(raw_slot('Same')),
+        'slot2': parse_slot(raw_slot('Same')),
     }
     reader = reader_double('slot1', 'slot2')
     ingestor_manager.opc_managers = {'test_server': reader}
     ingestor_manager.resource_manager.get_tag_slot = AsyncMock(
-        side_effect=lambda slot: raw_slot_v1('Same') if slot == 'slot1' else None
+        side_effect=lambda slot: raw_slot('Same') if slot == 'slot1' else None
     )
     ingestor_manager.resource_manager.renew_tag_lease = AsyncMock()
 
@@ -703,20 +688,20 @@ async def test_update_slot_config_unsubscribes_removed_slot(ingestor_manager, re
 @mark.asyncio
 async def test_update_slot_config(ingestor_manager):
     ingestor_manager.managed_tags = {
-        'slot1': parse_slot(raw_slot_v1('Old')),
-        'slot2': parse_slot(raw_slot_v1('Same')),
-        'slot3': parse_slot(raw_slot_v1('Old')),
+        'slot1': parse_slot(raw_slot('Old')),
+        'slot2': parse_slot(raw_slot('Same')),
+        'slot3': parse_slot(raw_slot('Old')),
     }
 
     ingestor_manager.resource_manager.get_tag_slot = AsyncMock(
-        side_effect=[raw_slot_v1('Updated'), raw_slot_v1('Same'), None]
+        side_effect=[raw_slot('Updated'), raw_slot('Same'), None]
     )
     ingestor_manager.resource_manager.renew_tag_lease = AsyncMock()
 
     await ingestor_manager.update_slot_config()
 
-    assert ingestor_manager.managed_tags['slot1'] == parse_slot(raw_slot_v1('Updated'))
-    assert ingestor_manager.managed_tags['slot2'] == parse_slot(raw_slot_v1('Same'))
+    assert ingestor_manager.managed_tags['slot1'] == parse_slot(raw_slot('Updated'))
+    assert ingestor_manager.managed_tags['slot2'] == parse_slot(raw_slot('Same'))
     assert 'slot3' not in ingestor_manager.managed_tags
 
     ingestor_manager.resource_manager.renew_tag_lease.assert_any_call('slot1')
@@ -783,8 +768,8 @@ async def test_manage_server_create_subscription_failure(ingestor_manager):
 
 @mark.asyncio
 async def test_manage_server_subscribe_and_unsubscribe_failure(ingestor_manager):
-    """subscribe() fails and so does the cleanup unsubscribe (legacy reader, session
-    gone): manage_server reports the failure and nothing escapes to the main loop"""
+    """subscribe() fails and so does the cleanup unsubscribe (session gone):
+    manage_server reports the failure and nothing escapes to the main loop"""
     reader = AsyncMock()
     reader.subscriptions = {}
     reader.subscribe.side_effect = Exception('Subscription error')
@@ -886,7 +871,7 @@ async def test_subscribe_to_tags(ingestor_manager):
 
 @mark.asyncio
 async def test_update_slot_config_keeps_last_good_config_when_malformed(ingestor_manager):
-    good = parse_slot(raw_slot_v1('Counter'))
+    good = parse_slot(raw_slot('Counter'))
     ingestor_manager.managed_tags = {'slot1': good}
     ingestor_manager.resource_manager.get_tag_slot = AsyncMock(
         return_value={'server1': {'server_id': '1', 'url': 'u', 'tags': {'ns=2;i=2': {}}}}
@@ -922,7 +907,7 @@ def test___init___ignores_malformed_write_env(
     """A malformed write/gRPC-only env var must not stop the read path (review #1)."""
     monkeypatch.setenv('GRPC_PORT', '')
     monkeypatch.setenv('WRITE_MIN_PAUSE_S', '0,5')
-    monkeypatch.setenv('CERT_BUCKET', 'streamlit-connectors')
+    monkeypatch.setenv('CERT_CACHE_DIR', 'certs-cache')
 
     ingestor = IngestorManager(
         redis_data={'host': 'localhost', 'port': 6379},
@@ -937,147 +922,4 @@ def test___init___ignores_malformed_write_env(
         metrics_controller=MagicMock(),
     )
 
-    assert ingestor.cert_provider.bucket == 'streamlit-connectors'
-
-
-def test_parse_slot_config_warns_once_about_server_cert_path(ingestor_manager):
-    raw = raw_slot_v1()
-    raw['server1']['server_cert_path'] = '/app/certs/kepserver.der'
-
-    ingestor_manager.parse_slot_config('1', raw)
-    ingestor_manager.parse_slot_config('1', raw)  # every loop refreshes the slot
-
-    warnings = [
-        c for c in ingestor_manager.logger.warning.call_args_list if 'server_cert_path' in str(c)
-    ]
-    assert len(warnings) == 1
-
-
-# --- READ_ENGINE=legacy: plan B with the untouched rc3 OpcManager -------------------
-
-
-def make_manager(read_engine=None):
-    with (
-        patch('ingestor.managers.ingestor_manager.DataManager'),
-        patch('ingestor.managers.ingestor_manager.ResourceManager'),
-    ):
-        manager = IngestorManager(
-            redis_data={'host': 'localhost', 'port': 6379},
-            lease_ttl=60,
-            heartbeat_ttl=60,
-            poll_interval=5,
-            mongo_connection_string='mongodb://localhost:27017',
-            mongo_database='sientia',
-            logger=MagicMock(),
-            notification_handler=MagicMock(),
-            metadata=metadata['metadata'],
-            metrics_controller=MagicMock(),
-            read_engine=read_engine,
-        )
-    manager.send_notification = MagicMock()
-    manager.emit_metric = AsyncMock()
-    return manager
-
-
-def test_read_engine_defaults_to_v2(monkeypatch):
-    monkeypatch.delenv('READ_ENGINE', raising=False)
-
-    assert make_manager().read_engine == 'v2'
-
-
-def test_read_engine_from_env(monkeypatch):
-    monkeypatch.setenv('READ_ENGINE', 'Legacy ')
-
-    assert make_manager().read_engine == 'legacy'
-
-
-def test_invalid_read_engine_falls_back_to_v2(monkeypatch):
-    monkeypatch.setenv('READ_ENGINE', 'v3')
-
-    manager = make_manager()
-
-    assert manager.read_engine == 'v2'
-    assert any('Invalid READ_ENGINE' in str(c) for c in manager.logger.warning.call_args_list)
-
-
-@mark.asyncio
-@patch('ingestor.managers.ingestor_manager.ServerReader')
-@patch('ingestor.managers.ingestor_manager.OpcManager')
-async def test_legacy_initialize_builds_rc3_opc_manager(opc_manager, server_reader):
-    manager = make_manager('legacy')
-    server_config = {
-        'name': 'server1',
-        'server_id': '1',
-        'url': 'opc.tcp://localhost:4840',
-        'subscription_period_ms': 1000,
-        'server_uri': 'urn:sientia:opc-ingestor',
-        'cert_path': '/app/certs/opc-ingestor-cert.pem',
-        'private_key_path': '/app/certs/opc-ingestor-key.pem',
-        'server_cert_path': None,
-    }
-    opc_manager.return_value = MagicMock(connect=AsyncMock())
-
-    result = await manager.initialize_opc_from_config(server_config)
-
-    server_reader.assert_not_called()
-    opc_manager.assert_called_once_with(
-        name='server1',
-        url='opc.tcp://localhost:4840',
-        subscription_period_ms=1000,
-        data_manager=manager.data_manager,
-        logger=manager.logger,
-        server_uri='urn:sientia:opc-ingestor',
-        notification_handler=manager.notification_handler,
-        metadata=metadata['metadata'],
-        cert_path='/app/certs/opc-ingestor-cert.pem',
-        private_key_path='/app/certs/opc-ingestor-key.pem',
-        server_cert_path=None,
-        metrics_controller=manager.metrics_controller,
-    )
-    assert result is opc_manager.return_value
-    result.connect.assert_awaited_once()
-
-
-def test_legacy_keeps_raw_slot_v1():
-    manager = make_manager('legacy')
-    raw = raw_slot_v1()
-
-    # OpcManager reads tag_name/topics and the file cert paths straight from the slot
-    assert manager.parse_slot_config('1', raw) is raw
-
-
-def test_legacy_rejects_slot_v2():
-    manager = make_manager('legacy')
-    raw_v2 = {
-        'server1': {
-            'name': 'server1',
-            'server_id': '1',
-            'url': 'opc.tcp://localhost:4840',
-            'use_security': True,
-            'subscription_period_ms': 500,
-            'tags': {
-                'ns=2;i=2': {
-                    'frequency': '1000',
-                    'names': [{'collection': 'raw_a', 'tag_name': 'Counter', 'model_id': '1'}],
-                }
-            },
-        }
-    }
-
-    assert manager.parse_slot_config('1', raw_v2) is None
-    _, kwargs = manager.send_notification.call_args
-    assert kwargs['notification_id'] == 'OPC_SLOT_CONFIG_ERROR_1'
-    assert 'READ_ENGINE=legacy' in kwargs['message']
-
-
-@mark.asyncio
-async def test_legacy_remove_server_calls_plain_shutdown():
-    manager = make_manager('legacy')
-    opc_manager = MagicMock(shutdown=AsyncMock())
-    manager.opc_managers = {'server1': opc_manager}
-    manager.managed_tags = {'slot1': {'server1': {}}}
-
-    await manager.remove_server('server1', evict=False)
-
-    opc_manager.shutdown.assert_awaited_once_with()
-    assert manager.opc_managers == {}
+    assert ingestor.cert_provider.cache_dir == 'certs-cache'

tests/unit/managers/test_opc_manager.py

@@ -1,627 +0,0 @@
-import json
-from datetime import datetime
-from unittest.mock import ANY, AsyncMock, MagicMock, call, patch
-
-import pytest
-from asyncua.crypto.security_policies import SecurityPolicyBasic256
-from pytest import fixture, mark
-from sientia_do.notifications.models import NotificationLevel
-
-from ingestor.managers.opc_manager import OpcManager
-
-tags = {
-    'ns=3;i=1001': {
-        'aggregation_function': 'LTS',
-        'frequency': 1000,
-        'max_value': 100,
-        'min_value': 0,
-        'tag_name': 'Counter',
-    },
-    'ns=3;i=1003': {
-        'aggregation_function': 'AVG',
-        'frequency': 1000,
-        'max_value': 100,
-        'min_value': 0,
-        'tag_name': 'Random',
-    },
-    'ns=3;i=1004': {
-        'aggregation_function': 'MDN',
-        'frequency': 1000,
-        'max_value': 100,
-        'min_value': 0,
-        'tag_name': 'Sawtooth',
-    },
-}
-
-metadata = {
-    'metadata': {
-        'model_id': 'test_model',
-        'model_name': 'test_model',
-        'workflow_name': 'test_workflow',
-        'schema_name': 'test_schedule',
-    },
-}
-
-
-@fixture
-@patch('ingestor.managers.opc_manager.metrics')
-def raw_opc_manager(mock_metrics):
-    opc_manager = OpcManager(
-        name='TestConnector',
-        url='opc.tcp://localhost:4840',
-        data_manager=AsyncMock(),
-        subscription_period_ms=1000,
-        logger=MagicMock(),
-        server_uri='opc.tcp://localhost:4840',
-        notification_handler=MagicMock(),
-        metadata=metadata['metadata'],
-        metrics_controller=AsyncMock(),
-    )
-
-    opc_manager.emit_metric = AsyncMock()
-    opc_manager.send_notification_async = AsyncMock()
-    opc_manager.send_notification = MagicMock()
-
-    return opc_manager
-
-
-@fixture
-def opc_manager(raw_opc_manager):
-    raw_opc_manager.client = AsyncMock()
-    raw_opc_manager.cert_path = 'cert.pem'
-    raw_opc_manager.private_key_path = 'private_key.pem'
-    raw_opc_manager.server_cert_path = 'server_cert.pem'
-
-    return raw_opc_manager
-
-
-@fixture
-def opc_manager_subscribed(opc_manager):
-    opc_manager.subscriptions['sub1'] = AsyncMock()
-
-    return opc_manager
-
-
-def test___str__(opc_manager):
-    assert (
-        str(opc_manager)
-        == 'OpcManager(name=TestConnector, url=opc.tcp://localhost:4840, server_uri=opc.tcp://localhost:4840)\nnodes={}, subscriptions={}'
-    )
-
-
-@mark.asyncio
-async def test_shutdown_success(opc_manager):
-    opc_manager.disconnect = AsyncMock()
-
-    await opc_manager.shutdown()
-
-    opc_manager.disconnect.assert_called_once()
-
-
-@mark.asyncio
-async def test_shutdown_error(opc_manager):
-    opc_manager.disconnect = AsyncMock(side_effect=Exception('Test error'))
-
-    await opc_manager.shutdown()
-
-    opc_manager.logger.error.assert_called_once_with('Error during cleanup: Test error')
-
-
-@mark.asyncio
-async def test_set_security_success(opc_manager):
-    await opc_manager.set_security()
-
-    assert opc_manager.client.application_uri == opc_manager.server_uri
-
-    opc_manager.client.set_security.assert_called_once_with(
-        SecurityPolicyBasic256,
-        certificate=opc_manager.cert_path,
-        private_key=opc_manager.private_key_path,
-        server_certificate=opc_manager.server_cert_path,
-    )
-
-    assert opc_manager.client.secure_channel_timeout == 10000000
-    assert opc_manager.client.session_timeout == 10000000
-
-
-@mark.asyncio
-async def test_set_security_no_cert(opc_manager):
-    opc_manager.cert_path = None
-    opc_manager.private_key_path = None
-
-    try:
-        await opc_manager.set_security()
-    except ValueError as e:
-        assert str(e) == 'Certificate and private key paths must be provided for secure connection.'
-    else:
-        raise AssertionError('ValueError not raised')
-
-    assert opc_manager.client.set_security.call_count == 0
-
-
-@mark.asyncio
-@patch('ingestor.managers.opc_manager.metrics')
-@patch('ingestor.managers.opc_manager.Client')
-async def test_connect_no_security(client, mock_metrics, raw_opc_manager):
-    raw_opc_manager.set_security = AsyncMock()
-    client.return_value = AsyncMock()
-
-    await raw_opc_manager.connect()
-
-    client.assert_called_once_with(raw_opc_manager.url, timeout=10, watchdog_intervall=3600000)
-    raw_opc_manager.client.connect.assert_called_once()
-    raw_opc_manager.set_security.assert_not_called()
-    raw_opc_manager.emit_metric.assert_has_calls(
-        [
-            call(
-                metric_object=mock_metrics.OPC_CONNECTIONS_TOTAL,
-                tags={
-                    'pod_id': raw_opc_manager.pod_id,
-                    'server_name': raw_opc_manager.name,
-                },
-            ),
-            call(
-                metric_object=mock_metrics.OPC_CONNECTION_STATUS,
-                method='set',
-                value=1,
-                tags={
-                    'pod_id': raw_opc_manager.pod_id,
-                    'server_name': raw_opc_manager.name,
-                    'server_url': raw_opc_manager.url,
-                },
-            ),
-        ],
-        any_order=True,
-    )
-
-
-@mark.asyncio
-@patch('ingestor.managers.opc_manager.Client')
-async def test_connect_with_security(client, raw_opc_manager):
-    raw_opc_manager.cert_path = 'cert.pem'
-    raw_opc_manager.private_key_path = 'private_key.pem'
-    raw_opc_manager.server_cert_path = 'server_cert.pem'
-    raw_opc_manager.set_security = AsyncMock()
-    client.return_value = AsyncMock()
-
-    await raw_opc_manager.connect()
-
-    client.assert_called_once_with(raw_opc_manager.url, timeout=10, watchdog_intervall=3600000)
-    raw_opc_manager.client.connect.assert_called_once()
-    raw_opc_manager.set_security.assert_called_once()
-
-
-@mark.asyncio
-@patch('ingestor.managers.opc_manager.Client')
-@patch('ingestor.managers.opc_manager.metrics')
-async def test_connect_exception_handling_and_metrics(
-    mock_metrics_module, mock_opc_client_class, raw_opc_manager
-):
-    mock_client_instance = mock_opc_client_class.return_value
-    simulated_error_message = 'Erro de conexão simulado'
-    mock_client_instance.connect.side_effect = Exception(simulated_error_message)
-
-    raw_opc_manager.disconnect = AsyncMock()
-
-    opc_manager_instance = raw_opc_manager
-    opc_manager_instance.cert_path = None
-
-    with pytest.raises(Exception, match=simulated_error_message):
-        await opc_manager_instance.connect()
-
-    opc_manager_instance.disconnect.assert_called_once()
-
-
-@mark.asyncio
-async def test_create_subscription_no_client(raw_opc_manager):
-    try:
-        await raw_opc_manager.create_subscription('sub1')
-    except ValueError as e:
-        assert str(e) == 'Client not connected. Call connect first.'
-    else:
-        raise AssertionError('ValueError not raised')
-
-
-@mark.asyncio
-async def test_create_subscription_success_has_period(opc_manager):
-    await opc_manager.create_subscription('sub1')
-
-    opc_manager.client.create_subscription.assert_called_once_with(
-        opc_manager.subscription_period_ms, opc_manager
-    )
-    assert opc_manager.subscriptions['sub1'] is not None
-
-
-@mark.asyncio
-async def test_create_subscription_success_no_period(opc_manager):
-    await opc_manager.create_subscription('sub1')
-
-    opc_manager.client.create_subscription.assert_called_once_with(
-        opc_manager.subscription_period_ms, opc_manager
-    )
-    assert opc_manager.subscriptions['sub1'] is not None
-
-
-@patch('ingestor.managers.opc_manager.metrics')
-@mark.asyncio
-async def test_create_subscription_with_metrics(metrics, opc_manager):
-    await opc_manager.create_subscription('sub1')
-
-    opc_manager.emit_metric.assert_called_once_with(
-        metric_object=metrics.OPC_SUBSCRIPTIONS_CREATED,
-        method='inc',
-        value=1,
-        tags={
-            'pod_id': opc_manager.pod_id,
-            'server_name': opc_manager.name,
-            'slot_name': 'sub1',
-        },
-    )
-
-
-@patch('ingestor.managers.opc_manager.metrics')
-@mark.asyncio
-async def test_create_subscription_exception_during_client_call(
-    mock_metrics_module, raw_opc_manager
-):
-    opc_manager_instance = raw_opc_manager
-    opc_manager_instance.client = AsyncMock()
-
-    subscription_name = 'test_sub_client_error'
-    simulated_error_message = 'Falha ao criar subscrição no cliente OPC'
-
-    opc_manager_instance.client.create_subscription.side_effect = Exception(simulated_error_message)
-
-    with pytest.raises(Exception, match=simulated_error_message):
-        await opc_manager_instance.create_subscription(subscription_name)
-
-    opc_manager_instance.client.create_subscription.assert_called_once_with(
-        opc_manager_instance.subscription_period_ms, opc_manager_instance
-    )
-
-    opc_manager_instance.logger.error.assert_called_once_with(
-        f'Failed to create subscription {subscription_name} on {opc_manager_instance.name}: {simulated_error_message}'
-    )
-
-    mock_metrics_module.OPC_SUBSCRIPTIONS_CREATED.labels.assert_not_called()
-
-
-@patch('ingestor.managers.opc_manager.metrics')
-@mark.asyncio
-async def test_subscribe_no_subscription(metrics, opc_manager):
-    try:
-        await opc_manager.subscribe('sub1', tags, 1000)
-    except ValueError as e:
-        assert str(e) == 'Subscription not created. Call create_subscription first.'
-    else:
-        raise AssertionError('ValueError not raised')
-    metrics.OPC_TAGS_SUBSCRIBED.labels.assert_not_called()
-
-
-@mark.asyncio
-async def test_subscribe_success(opc_manager_subscribed):
-    opc_manager_subscribed.client.get_node = MagicMock()
-    opc_manager_subscribed.nodes = {'ns=3;i=1001': 'data'}
-
-    await opc_manager_subscribed.subscribe('sub1', tags, 1000)
-
-    assert opc_manager_subscribed.nodes == tags
-    opc_manager_subscribed.subscriptions['sub1'].subscribe_data_change.assert_called_once_with(
-        [opc_manager_subscribed.client.get_node(n) for n in tags]
-    )
-
-
-@mark.asyncio
-async def test_unsubscribe_no_subscription(opc_manager):
-    await opc_manager.unsubscribe('sub1')
-
-    opc_manager.logger.warning.assert_called_once_with(
-        "Subscription 'sub1' not found. Cannot unsubscribe."
-    )
-    assert opc_manager.subscriptions.get('sub1') is None
-
-
-@mark.asyncio
-async def test_unsubscribe_success(opc_manager_subscribed):
-    await opc_manager_subscribed.unsubscribe('sub1')
-
-    assert opc_manager_subscribed.subscriptions.get('sub1') is None
-
-
-@mark.asyncio
-async def test_disconnection_fallback_success(opc_manager):
-    opc_manager.client = AsyncMock()
-    opc_manager.client.disconnect.return_value = True
-    result = await opc_manager.disconnection_fallback()
-    assert result == []
-
-
-@mark.asyncio
-async def test_disconnection_fallback_fail(opc_manager):
-    opc_manager.client = AsyncMock()
-    opc_manager.client.disconnect.side_effect = Exception('Test error')
-    result = await opc_manager.disconnection_fallback()
-    assert result == [
-        {'attempt': 1, 'error': 'Test error', 'traceback': ANY},
-        {'attempt': 2, 'error': 'Test error', 'traceback': ANY},
-        {'attempt': 3, 'error': 'Test error', 'traceback': ANY},
-        {'attempt': 4, 'error': 'Test error', 'traceback': ANY},
-        {'attempt': 5, 'error': 'Test error', 'traceback': ANY},
-    ]
-    assert opc_manager.client.disconnect.call_count == 5
-
-
-@mark.asyncio
-async def test_disconnect_success(opc_manager_subscribed):
-    opc_manager_subscribed.client = MagicMock()
-
-    await opc_manager_subscribed.disconnect()
-
-    opc_manager_subscribed.subscriptions['sub1'].delete.assert_called_once()
-    assert opc_manager_subscribed.client is None
-
-
-@mark.asyncio
-async def test_disconnect_no_client(opc_manager_subscribed):
-    opc_manager_subscribed.client = None
-    assert await opc_manager_subscribed.disconnect() is None
-
-    opc_manager_subscribed.logger.warning.assert_has_calls(
-        [
-            call('Client already disconnected.'),
-        ]
-    )
-
-
-@mark.asyncio
-async def test_disconnect_error_unsubscribe(opc_manager_subscribed):
-    opc_manager_subscribed.client = MagicMock(disconnect=AsyncMock())
-    opc_manager_subscribed.subscriptions['sub1'] = MagicMock(
-        delete=AsyncMock(side_effect=Exception('Test error'))
-    )
-
-    await opc_manager_subscribed.disconnect()
-
-    opc_manager_subscribed.subscriptions['sub1'].delete.assert_called_once()
-    opc_manager_subscribed.client = None
-    opc_manager_subscribed.logger.error.assert_called_once_with(
-        'Failed to clean up subscription: Test error'
-    )
-
-
-@mark.asyncio
-async def test_disconnect_error(opc_manager_subscribed):
-    opc_manager_subscribed.client = MagicMock()
-    opc_manager_subscribed.client.disconnect = MagicMock(side_effect=Exception('Test error'))
-
-    await opc_manager_subscribed.disconnect()
-
-    opc_manager_subscribed.subscriptions['sub1'].delete.assert_called_once()
-    opc_manager_subscribed.client = None
-
-
-@patch('ingestor.managers.opc_manager.metrics')
-@mark.asyncio
-async def test_disconnect_metrics_on_successful_path(mock_metrics_module, raw_opc_manager):
-    mock_metrics_module.OPC_CONNECTION_STATUS.reset_mock()
-    mock_metrics_module.OPC_TAGS_SUBSCRIBED.reset_mock()
-
-    raw_opc_manager.client = MagicMock()
-    mock_sub1 = MagicMock()
-    mock_sub2 = MagicMock()
-    raw_opc_manager.subscriptions = {'sub1': mock_sub1, 'sub2': mock_sub2}
-
-    await raw_opc_manager.disconnect()
-
-    raw_opc_manager.emit_metric.assert_has_calls(
-        [
-            call(
-                metric_object=mock_metrics_module.OPC_CONNECTION_STATUS,
-                method='set',
-                value=0,
-                tags={
-                    'pod_id': raw_opc_manager.pod_id,
-                    'server_name': raw_opc_manager.name,
-                    'server_url': raw_opc_manager.url,
-                },
-            ),
-            call(
-                metric_object=mock_metrics_module.OPC_TAGS_SUBSCRIBED,
-                method='set',
-                value=0,
-                tags={
-                    'pod_id': raw_opc_manager.pod_id,
-                    'server_name': raw_opc_manager.name,
-                },
-            ),
-        ],
-        any_order=True,
-    )
-
-
-@patch('ingestor.managers.opc_manager.metrics')
-@mark.asyncio
-async def test_datachange_notification(metrics, opc_manager_subscribed):
-    data = MagicMock(
-        monitored_item=MagicMock(
-            Value=MagicMock(
-                Value=MagicMock(Value=42),
-                SourceTimestamp=datetime.strptime('2021-01-01T00:00:00', '%Y-%m-%dT%H:%M:%S'),
-            )
-        )
-    )
-    opc_manager_subscribed.nodes = {
-        'ns=3;i=1001': {
-            'tag_name': 'Counter',
-            'cycle_rule': {'cycle_increment': 1.0, 'cycle_count': 2},
-            'topics': ['topic1', 'topic2'],
-        }
-    }
-
-    await opc_manager_subscribed.datachange_notification('ns=3;i=1001', None, data)
-
-    opc_manager_subscribed.data_manager.publish.assert_any_call(
-        'topic1',
-        {
-            'tag': 'ns=3;i=1001',
-            'name': 'Counter',
-            'timestamp': '2021-01-01 00:00:00-0300',
-            'value': 42,
-        },
-    )
-    opc_manager_subscribed.data_manager.publish.assert_any_call(
-        'topic2',
-        {
-            'tag': 'ns=3;i=1001',
-            'name': 'Counter',
-            'timestamp': '2021-01-01 00:00:00-0300',
-            'value': 42,
-        },
-    )
-    assert opc_manager_subscribed.nodes['ns=3;i=1001']['cycle_rule']['cycle_count'] == 0
-
-    opc_manager_subscribed.emit_metric.assert_called_once_with(
-        metric_object=metrics.OPC_CYCLES_WITHOUT_DATA,
-        method='set',
-        value=0,
-        tags={
-            'pod_id': opc_manager_subscribed.pod_id,
-            'server_name': opc_manager_subscribed.name,
-        },
-    )
-
-
-@mark.asyncio
-async def test_check_cycles_no_notification(opc_manager):
-    # Setup: node with cycle_count just below threshold
-    opc_manager.nodes = {
-        'ns=3;i=1001': {
-            'tag_name': 'Counter',
-            'cycle_rule': {'cycle_increment': 1.0, 'cycle_count': 3.0},
-        }
-    }
-
-    await opc_manager.check_cycles()
-
-    # After one increment, cycle_count = 4.0, still below threshold
-    assert opc_manager.nodes['ns=3;i=1001']['cycle_rule']['cycle_count'] == pytest.approx(4.0)
-    opc_manager.send_notification_async.assert_not_called()
-
-
-@mark.asyncio
-async def test_check_cycles_triggers_notification(opc_manager):
-    # Setup: node with cycle_count just below threshold, increment will cross threshold
-    opc_manager.nodes = {
-        'ns=3;i=1001': {
-            'tag_name': 'Counter',
-            'cycle_rule': {'cycle_increment': 2.5, 'cycle_count': 3.0},
-        }
-    }
-    opc_manager.notification_handler.build_and_send_notification = MagicMock()
-
-    await opc_manager.check_cycles()
-
-    # After increment, cycle_count = 5.5, should trigger notification
-    assert opc_manager.nodes['ns=3;i=1001']['cycle_rule']['cycle_count'] == pytest.approx(5.5)
-    opc_manager.send_notification_async.assert_called_once_with(
-        notification_id='TAG_ns=3;i=1001:Counter_LISTENNING_STOPPED',
-        message='5.5 cycles without receive from ns=3;i=1001:Counter',
-        block='opc_manager',
-        level=NotificationLevel.WARNING,
-        metadata=metadata['metadata'],
-    )
-
-
-@patch('ingestor.managers.opc_manager.metrics')
-@mark.asyncio
-async def test_check_opc_listenning_no_notification(metrics, opc_manager):
-    opc_manager.non_receive_count = 3
-
-    result = await opc_manager.check_opc_listenning()
-
-    assert opc_manager.non_receive_count == 4
-    opc_manager.send_notification_async.assert_not_called()
-    assert result is False
-
-    opc_manager.emit_metric.assert_called_once_with(
-        metric_object=metrics.OPC_CYCLES_WITHOUT_DATA,
-        method='set',
-        value=opc_manager.non_receive_count,
-        tags={
-            'pod_id': opc_manager.pod_id,
-            'server_name': opc_manager.name,
-        },
-    )
-
-
-@mark.asyncio
-async def test_check_opc_listenning_warning_notification(opc_manager):
-    opc_manager.non_receive_count = 4
-
-    result = await opc_manager.check_opc_listenning()
-
-    assert opc_manager.non_receive_count == 5
-    opc_manager.send_notification_async.assert_called_once_with(
-        notification_id=f'OPC_LISTENNING_STOPPED__{opc_manager.name}',
-        message=f'5 cycles without receive from OPC {opc_manager.name}. Tags: {json.dumps(opc_manager.nodes)}',
-        block='opc_manager',
-        level=NotificationLevel.ERROR,
-        metadata=metadata['metadata'],
-    )
-    assert result is False
-
-
-@patch('ingestor.managers.opc_manager.metrics')
-@mark.asyncio
-async def test_check_opc_listenning_error_notification_and_retry(metrics, opc_manager):
-    opc_manager.non_receive_count = 14
-
-    result = await opc_manager.check_opc_listenning()
-
-    assert opc_manager.non_receive_count == 15
-    # Should be called twice: once for 5, once for 15
-    assert opc_manager.send_notification_async.call_count == 2
-    calls = opc_manager.send_notification_async.call_args_list
-    # First call: 5 cycles warning
-    assert calls[0].kwargs == {
-        'notification_id': f'OPC_LISTENNING_STOPPED__{opc_manager.name}',
-        'message': f'15 cycles without receive from OPC {opc_manager.name}. Tags: {json.dumps(opc_manager.nodes)}',
-        'block': 'opc_manager',
-        'level': NotificationLevel.ERROR,
-        'metadata': metadata['metadata'],
-    }
-    # Second call: 15 cycles retry
-    assert calls[1].kwargs == {
-        'notification_id': f'OPC_CONNECTION_RETRY__{opc_manager.name}',
-        'message': f'Retrying to connect to server {opc_manager.name}',
-        'block': 'opc_manager',
-        'level': NotificationLevel.ERROR,
-        'metadata': metadata['metadata'],
-    }
-    assert result is True
-
-    opc_manager.emit_metric.assert_has_calls(
-        [
-            call(
-                metric_object=metrics.OPC_CYCLES_WITHOUT_DATA,
-                method='set',
-                value=opc_manager.non_receive_count,
-                tags={
-                    'pod_id': opc_manager.pod_id,
-                    'server_name': opc_manager.name,
-                },
-            ),
-        ]
-    )
-
-    opc_manager.emit_metric.assert_has_calls(
-        [
-            call(
-                metric_object=metrics.OPC_RECONNECTIONS_TOTAL,
-                tags={
-                    'pod_id': opc_manager.pod_id,
-                    'server_name': opc_manager.name,
-                },
-            ),
-        ]
-    )

tests/unit/managers/test_server_reader.py

@@ -9,9 +9,14 @@ SERVER_CONFIG = {
     'name': 'kepserver',
     'server_id': 1,  # the slot may carry it as int; the pool keys by str
     'url': 'opc.tcp://kepserver:49320',
-    'server_uri': 'urn:sientia:opc-ingestor',
-    'use_security': True,
     'subscription_period_ms': 15000.0,
+    'security': {'mode': 'SignAndEncrypt', 'policy': 'Basic256Sha256'},
+    'certificate': {
+        'certificate_id': 1,
+        'application_uri': 'urn:sientia:opc-ingestor',
+        'cert': {'bucket': 'opc-certificates', 'object_name': 'client/p/cert.pem'},
+        'private_key': {'bucket': 'opc-certificates', 'object_name': 'client/p/key.pem'},
+    },
 }
 
 
@@ -45,10 +50,8 @@ def test_server_cfg_keeps_only_connection_fields():
 
     assert reader.server_id == '1'
     assert reader.server_cfg == {
-        'url': 'opc.tcp://kepserver:49320',
-        'server_uri': 'urn:sientia:opc-ingestor',
-        'use_security': True,
-        'subscription_period_ms': 15000.0,
+        key: SERVER_CONFIG[key]
+        for key in ('url', 'subscription_period_ms', 'security', 'certificate')
     }
 
 
@@ -58,7 +61,7 @@ async def test_connect_opens_pooled_connection():
 
     await reader.connect()
 
-    # use_security=True: the pool fetches the certificate from MinIO (CertProvider)
+    # With a certificate, the pool fetches it from MinIO (CertProvider)
     pool.get.assert_awaited_once_with('1', reader.server_cfg)
 
 

tests/unit/test_config.py

@@ -8,7 +8,6 @@ _ENV_VARS = (
     'NATS_QUEUE',
     'GRPC_ENABLED',
     'GRPC_PORT',
-    'CERT_BUCKET',
     'CERT_CACHE_DIR',
     'WRITE_TIMESTAMP',
     'WRITE_QUALITY',
@@ -31,7 +30,6 @@ def test_from_env_defaults():
     assert cfg.nats_queue == 'gateway-writers'
     assert cfg.grpc_enabled is False
     assert cfg.grpc_port == 50051
-    assert cfg.cert_bucket == 'streamlit-connectors'
     assert cfg.cert_cache_dir == '/tmp/opc-certs'  # noqa: S108
     assert cfg.write_timestamp is True
     assert cfg.write_quality is True
@@ -76,12 +74,10 @@ def test_certs_from_env_ignores_malformed_numeric_vars(monkeypatch):
 
     monkeypatch.setenv('GRPC_PORT', '')
     monkeypatch.setenv('IDLE_CONN_MAX_S', 'abc')
-    monkeypatch.setenv('CERT_BUCKET', 'my-bucket')
     monkeypatch.setenv('CERT_CACHE_DIR', 'certs-cache')
 
     cfg = GatewayConfig.certs_from_env()
 
-    assert cfg.cert_bucket == 'my-bucket'
     assert cfg.cert_cache_dir == 'certs-cache'
     with pytest.raises(ValueError):
         GatewayConfig.from_env()  # the full config still fails, only the runtime uses it

tests/unit/test_gateway_runtime.py

@@ -39,6 +39,7 @@ def mocks(monkeypatch):
     patched['pool'].release_idle = AsyncMock()
     patched['NatsWriteConsumer'].return_value.start = AsyncMock()
     patched['NatsWriteConsumer'].return_value.stop = AsyncMock()
+    patched['NatsWriteConsumer'].return_value.closed = False
     patched['GrpcServer'].return_value.start = AsyncMock()
     patched['GrpcServer'].return_value.stop = AsyncMock()
     return patched
@@ -173,3 +174,47 @@ async def test_idle_sweep_releases_idle_connections(mocks):
     await runtime.idle_sweep()
 
     mocks['pool'].release_idle.assert_awaited_once_with(42.0)
+
+
+def test_write_engine_resolves_servers_from_the_pool_and_mongo(mocks):
+    """SIENTIAPDE-1975 GW1: write commands carry only the server_id."""
+    data_manager = MagicMock()
+    runtime = gr.GatewayRuntime(
+        config=GatewayConfig(),
+        data_manager=data_manager,
+        pool=mocks['pool'],
+        logger=MagicMock(),
+        notification_handler=MagicMock(),
+        metrics_controller=MagicMock(),
+    )
+
+    _, kwargs = mocks['WriteEngine'].call_args
+    assert kwargs['servers'] is runtime.servers
+    assert runtime.servers.pool is mocks['pool']
+    assert runtime.servers.mongo is data_manager.mongo_repository
+
+
+@pytest.mark.asyncio
+async def test_idle_sweep_restarts_a_closed_nats_connection(mocks):
+    runtime = make_runtime(pool=mocks['pool'])
+    nats_mock = mocks['NatsWriteConsumer'].return_value
+    await runtime.start()
+    assert nats_mock.start.await_count == 1
+
+    nats_mock.closed = True  # closed for good (e.g. by the server)
+    await runtime.idle_sweep()
+
+    assert nats_mock.start.await_count == 2
+    assert runtime._nats_started is True
+    runtime.logger.warning.assert_called_once()
+
+
+@pytest.mark.asyncio
+async def test_idle_sweep_leaves_a_live_nats_connection_alone(mocks):
+    runtime = make_runtime(pool=mocks['pool'])
+    nats_mock = mocks['NatsWriteConsumer'].return_value
+    await runtime.start()
+
+    await runtime.idle_sweep()
+
+    assert nats_mock.start.await_count == 1