2 Commits

Author SHA1 Message Date
Eduardo Rios
ea55a4db62 TEST ONLY: sqlalchemy<2.1 (No module named psycopg with sientia_do 1.12.2) - dev test, do not merge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 15:35:37 -03:00
Eduardo Rios
8ab9af170d SIENTIAPDE-1975: the gateway is the only writer that updates write_status
The Laborious no longer records OPC outcomes (it inserts the row as
'pending' and only publishes the commands), so the comments and docs that
asked to keep a Laborious copy of the UPDATE in sync are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:00:27 -03:00
4 changed files with 12 additions and 7 deletions

View File

@@ -60,7 +60,7 @@ Any change to slot/lease counts should be traced through `manage_leases`/`manage
**ServerReader** (SIENTIAPDE-1974, default `READ_ENGINE=v2`; the legacy `OpcManager` stays as plan B with `READ_ENGINE=legacy` — slot v1 only, certificate files from the Gitea clone — until it is unused and the tests pass the same without it) reads one OPC server through a `SubscriptionEngine`: the connection comes from the shared `OpcConnectionPool` (also used by NATS writes and gRPC), which fetches the platform certificate from MinIO via `CertProvider` and uses `SecurityPolicyBasic256Sha256` — no certificate files in the repo/image. Slots are parsed by `io/slots.py` (`parse_slot`), which accepts slot v2 (`names[]`) and converts the legacy v1 (`topics`). `datachange_notification` fans each value out to every `names` entry via `DataSink`. Subscribed connections are pinned in the pool so the idle sweep never drops them. Tracks `non_receive_count` to detect silently-dead subscriptions (`check_cycles` / `check_opc_listenning`); a lost server is shut down (pool connection evicted) and re-initialized on the next loop.
**Writes** (SIENTIAPDE-1975): `NatsWriteConsumer` → `WriteEngine` → pool, then `DataSink.update_write_status`. Commands (C2) carry only `server_id`; `ServerConfigResolver` (`connection/servers.py`) resolves the connection (command override → live pooled connection → `connectors` → `opc-servers`). `write_status` keeps the worst outcome (`ok` < `error` < `session_bad`; same rule as the Laborious' `laborious/utils/write_status.py` — keep both in sync). The NATS first connect is bounded (it runs on the main loop); reconnection after it is unbounded.
**Writes** (SIENTIAPDE-1975): `NatsWriteConsumer` → `WriteEngine` → pool, then `DataSink.update_write_status`. Commands (C2) carry only `server_id`; `ServerConfigResolver` (`connection/servers.py`) resolves the connection (command override → live pooled connection → `connectors` → `opc-servers`). `write_status` keeps the worst outcome (`ok` < `error` < `session_bad`); the gateway is the only one that updates it (the Laborious inserts the row as `pending` and never records OPC outcomes). The NATS first connect is bounded (it runs on the main loop); reconnection after it is unbounded.
**Metrics** (`ingestor/metrics.py`): all Prometheus metrics defined here, always labeled by `pod_id`. Emitted via `SientiaMonitoring.emit_metric(...)` (from `sientia_do`), not the prometheus client directly — new metrics should follow that pattern for consistency with logging/notifications.

View File

@@ -91,8 +91,9 @@ The Laborious publishes one command per tag (SIENTIAPDE-1975):
Lookups are cached for 60 s. An unknown server is a failed write (`server_not_found`).
- **Outcome**: `UPDATE ... SET write_status` on the prediction row (by `timestamp` +
`model_id`), keeping the worst outcome: `ok` < `error` < `session_bad`, and
`pending`/NULL are replaced by any answer. The Laborious applies the same rule to
the outcomes it records itself, so the order of the answers does not matter.
`pending`/NULL are replaced by any answer, so the order of the answers does not
matter. The gateway is the only one that updates the column: the Laborious inserts the
row as `pending` (merged with its PI Web API outcome) and never records OPC outcomes.
The UPDATE never raises; its outcome (`updated`, `kept`, `row_not_found`,
`invalid`, `error`) is logged and counted in `opc_write_status_update_total`.
The reason of a failed write is logged by `WriteEngine`.

View File

@@ -10,8 +10,8 @@ import ingestor.metrics as metrics
_IDENTIFIER_RE = re.compile(r'\w+', re.ASCII)
# Worst-wins precedence of predictions.write_status (SIENTIAPDE-1975), from best to worst;
# NULL, 'pending' and unknown values rank 0. Same rule as the Laborious
# (laborious/utils/write_status.py): keep both in sync.
# NULL, 'pending' and unknown values rank 0. The gateway is the only writer that updates the
# column: the Laborious only sets the value of the rows it inserts ('pending', PI outcome).
STATUS_RANK = {'ok': 1, 'error': 2, 'session_bad': 3}
@@ -78,8 +78,9 @@ class DataSink:
The row is found by timestamp + model_id. A prediction is written to several
tags, each answered by its own command, so a better outcome never replaces a
worse one: ``ok`` < ``error`` < ``session_bad``, and ``pending``/NULL are
replaced by any answer. The Laborious runs the same statement for the outcomes it
records itself (SIENTIAPDE-1975), so the order of the answers does not matter.
replaced by any answer, so the order of the answers does not matter. Only the
gateway updates the column (SIENTIAPDE-1975): the Laborious publishes the commands
and sets ``pending`` (merged with its PI Web API outcome) on the row it inserts.
Never raises: a failure here must not reach the NATS callback nor hide the OPC
write itself, which already happened. The outcome is logged and counted instead.

View File

@@ -4,6 +4,9 @@ redis
# (generic Mongo find_one/distinct/aggregate methods) — not in 1.12.2 yet.
# Local dev uses an editable install of the local checkout (feature/SIENTIAPDE-1590).
sientia_do==1.12.2
# TEST ONLY (SIENTIAPDE-1975 dev test): SQLAlchemy 2.1 makes psycopg (v3) the default driver for
# postgresql:// URLs; sientia_do 1.12.2 builds that URL and ships only psycopg2-binary.
sqlalchemy<2.1
prometheus_client
pymongo
nats-py>=2.6