OPC Gateway — E2E Validation Guide (SIENTIAPDE-1972)

Session script for validating the gateway against a real KepServer + cluster (with the feature owner). Each step names the command and the expected result. Run order matters.

In scope: cert management, subscription (legacy slot v1), NATS write → OPC → write_status, gRPC ReadTag, 1879 write flags, resilience, metrics. Out of scope (integration-gated): multi-name subscription / scheduled active reads / gRPC persist (need slot v2 → SIENTIAPDE-1974); the real Laborious publish (SIENTIAPDE-1975) — here we hand-publish the C2 message, which is the contract.

0. Prerequisites

What	How
KepServer reachable	`OPC_SERVER_URL` (default `opc.tcp://localhost:49320`); Simulation Examples tags enabled
NATS (test instance)	`docker run -d --name nats -p 4222:4222 nats:2` — no JetStream needed (Core NATS). Cluster provisioning is SIENTIAPDE-1985; this container is only for the session
`nats` CLI	natscli or `docker exec` a `nats-box`
`grpcurl`	for the ReadTag call
Mongo/Redis/Postgres	same instances the app already uses (`.env`)
MinIO policy (pre-req if testing security)	frontend user needs write on bucket `sientia-dataops`; gateway reads it. The 1569 CRUD now defaults to `sientia-dataops` (`MINIO_BUCKET` env overrides)

1. Seed test data

python scripts/seed_e2e.py --print-only   # inspect first
python scripts/seed_e2e.py                # seed Mongo + Redis slot v1 + Postgres table

Creates: connector e2e-kepserver (9001), 2 tags with metadata: {}, slot slot:opc_tags:1 (v1), and sientia_data.e2e_predictions with the write_status column + 1 row (model_id='9001', timestamp='2026-07-15T12:00:00Z').

Adjust the two tag address values to real KepServer node ids if the Simulation Examples channel differs.

2. Start the gateway

# .env: MONGODB_*/REDIS_* as today, plus NATS_URL=nats://localhost:4222, GRPC_ENABLED=true
python -m ingestor.app

Expect in logs: NatsWriteConsumer started: subject=opc.write queue=gateway-writers, GrpcServer started on port 50051, GatewayRuntime started: nats_started=True grpc_enabled=True.

Health probe (what k8s will run): curl -sf http://localhost:9090/metrics | grep '^app_up' → app_up{...} 1.0.

3. Subscription (legacy v1 path)

The slot from step 1 subscribes e2e_ramp. Expect within ~1 cycle:

Mongo collection raw_e2e-gateway-test receiving documents {tag, name, timestamp, value}.
Metrics: opc_connection_status{...} 1, opc_tags_subscribed_current ≥ 1.

4. Write via NATS (C2 → OPC → C3)

The seed prints the exact command; it targets the seeded predictions row:

nats pub opc.write '<json printed by the seed script>'

Expect:

KepServer: e2e_write_target (User1) becomes 42.5.
Postgres: SELECT write_status FROM sientia_data.e2e_predictions WHERE model_id='9001'; → ok.
Metrics: opc_write_total{result="success"} +1, opc_write_latency_seconds observed, opc_write_status_by_model{status="ok"} 1.

1879 flag checks (repeat the publish varying the payload):

"write_timestamp": false / "write_quality": false → KepServer must still accept (or reject — that's the point of testing against the real server; compare behaviors).
Bad value "value": "abc" → no crash; write_status='error', opc_error_by_type_total{error_type="conversion"} +1.
Malformed JSON (nats pub opc.write 'not-json') → dropped + logged, app alive.
Missing field (remove "addr") → write_status='error' (bad_command), no crash.
Two rapid publishes with WRITE_MIN_PAUSE_S=1 set → second write delayed ≥1s (rate-limit).

5. gRPC on-demand read

grpcurl -plaintext -d '{"server_id":"9001","addr":"ns=2;s=Simulation Examples.Functions.Ramp1"}' \
  localhost:50051 opcgateway.OpcGateway/ReadTag

Expect: value, opc_status_code: "Good", timestamps, persisted: false (correct until 1974). Metrics: opc_active_read_total{result="success"} +1, opc_read_lag_seconds set. Error case: bad addr → gRPC INTERNAL, opc_active_read_total{result="error"} +1, app alive.

6. Certificates (round 2 — security on)

Upload platform cert via the connectors CRUD (writes to sientia-dataops now) — or place objects under platform/certificates/ + the single platform_certificates Mongo doc manually.
Flip the seeded connector/slot to use_security: true, restart the gateway.
Expect Basic256Sha256 session on KepServer (check its connection log); reads/writes as in steps 3–5.

7. Resilience

NATS down: docker stop nats → app keeps reading (subscription + gRPC fine); ONE notification GATEWAY_NATS_CONSUMER_START_FAILED; docker start nats → consumer reconnects on next loop (~5s); publish works again.
MinIO down at boot (security round): gateway logs GatewayRuntime failed to initialize, continuing without gateway and the read path still runs.
KepServer restart: subscription reconnect kicks in after ~15 idle cycles (opc_reconnections_tried_total).

8. Wrap-up checks

curl -s localhost:9090/metrics | grep -E 'opc_(write|active_read|error|tag_value|read_lag)' — all families populated.
No kafka anywhere: grep -ri kafka ingestor/ → empty.
Postgres row still has exactly the statuses produced by step 4 (no stray writes).

Ledger: phases/commits in docs/superpowers/plans/2026-07-13-opc-gateway.md; deploy details in docs/deploy-notes.md.

5.5 KiB Raw Blame History Unescape Escape