SIENTIAPDE-1974: feat(read): slot v2 reads via SubscriptionEngine, OPC cert from MinIO
Swaps the legacy OpcManager read path for the gateway's SubscriptionEngine over the shared OpcConnectionPool. The OPC client certificate now always comes from MinIO + Mongo (CertProvider, Basic256Sha256) — no more certs/ shipped in the Gitea repository, so a new version branch no longer breaks the reads. - io/slots.py: parse_slot accepts slot v2 (names[]) and converts the legacy v1 (tag_name + topics; cert_path => use_security, path dropped), so gateway and Orchestrator can be deployed in any order. - managers/server_reader.py: new per-server reader with the OpcManager interface the IngestorManager already drives (connect, create_subscription, subscribe, unsubscribe, check_cycles, check_opc_listenning, shutdown). - IngestorManager owns the read path: lazy-MinIO CertProvider, the pool and a read DataSink, built without touching the network. Slots from Redis are normalized on lease and refresh; a malformed slot is notified and the last good config is kept. - GatewayRuntime borrows the pool (writes/gRPC) instead of building it, and no longer closes it on stop: a Postgres/NATS failure can't stop the reads. - pool: pin/unpin per active subscription so release_idle never evicts a subscribed connection; public evict() for lost servers / changed config. - SubscriptionEngine pins on subscribe, unpins on unsubscribe, and drops a subscription locally even when the server-side delete fails. - DataSink: model_id only written when the name has one, keeping v1 documents identical to the legacy ones. - config: CERT_BUCKET default is the legacy streamlit-connectors bucket (D7). - Removes OpcManager, its tests and scripts/generate-and-push-opc-cert.sh; README/CLAUDE.md document the MinIO certificate and the slot v2 format. Tests: 218 unit tests pass; ruff check/format clean. Local end-to-end against an asyncua server: v1 slot with two topics -> one doc per collection, legacy doc shape, idle sweep keeps the subscription, lost-server shutdown and re-initialization work. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Eduardo Rios
@@ -1,95 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Gera o certificado X.509 de cliente OPC UA do ingestor e sobe pro Gitea
|
||||
# in-cluster (repo que o pod clona no boot — nada muda no values/deployment,
|
||||
# só rollout restart depois do push).
|
||||
#
|
||||
# Uso:
|
||||
# GITEA_PASSWORD='<senha>' ./scripts/generate-and-push-opc-cert.sh
|
||||
#
|
||||
# Vars (defaults pro cluster dev/gcp):
|
||||
# GITEA_URL default https://git.sientia.ai
|
||||
# GITEA_USER default gitea_admin
|
||||
# GITEA_PASSWORD obrigatória
|
||||
# GITEA_REPO default gitea_admin/sientia-dataops-opc-ingestor
|
||||
# BRANCH default main
|
||||
# APP_URI default urn:sientia:opc-ingestor (SAN URI do cert — TEM que
|
||||
# ser igual ao campo `uri` do documento na collection
|
||||
# OPC_servers: o ingestor usa esse valor como application_uri
|
||||
# e o KEPServer rejeita com BadCertificateUriInvalid se divergir)
|
||||
# DAYS default 3650
|
||||
# FORCE=1 sobrescreve cert já existente no repo
|
||||
set -euo pipefail
|
||||
|
||||
GITEA_URL=${GITEA_URL:-https://git.sientia.ai}
|
||||
GITEA_USER=${GITEA_USER:-gitea_admin}
|
||||
GITEA_PASSWORD=${GITEA_PASSWORD:?export GITEA_PASSWORD com a senha do Gitea}
|
||||
GITEA_REPO=${GITEA_REPO:-gitea_admin/sientia-dataops-opc-ingestor}
|
||||
BRANCH=${BRANCH:-main}
|
||||
APP_URI=${APP_URI:-urn:sientia:opc-ingestor}
|
||||
CN=${CN:-sientia-opc-ingestor}
|
||||
DAYS=${DAYS:-3650}
|
||||
|
||||
CERT_DIR_IN_REPO="certs"
|
||||
CERT_NAME="opc-ingestor-cert.pem"
|
||||
KEY_NAME="opc-ingestor-key.pem"
|
||||
DER_NAME="opc-ingestor-cert.der"
|
||||
|
||||
workdir=$(mktemp -d)
|
||||
trap 'rm -rf "$workdir"' EXIT
|
||||
|
||||
echo ">> Gerando certificado (CN=${CN}, URI=${APP_URI}, ${DAYS} dias)..."
|
||||
# Extensões exigidas por OPC UA (KEPServer valida keyUsage/EKU e o SAN URI).
|
||||
openssl req -x509 -newkey rsa:2048 -sha256 -nodes -days "$DAYS" \
|
||||
-keyout "$workdir/$KEY_NAME" -out "$workdir/$CERT_NAME" \
|
||||
-subj "/CN=${CN}/O=Aignosi/OU=Sientia" \
|
||||
-addext "subjectAltName=URI:${APP_URI},DNS:${CN}" \
|
||||
-addext "keyUsage=critical,digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment" \
|
||||
-addext "extendedKeyUsage=critical,clientAuth,serverAuth" \
|
||||
-addext "basicConstraints=critical,CA:FALSE"
|
||||
|
||||
# Cópia DER — só referência p/ import manual no KEPServer (o trust normal é
|
||||
# aceitar o cert rejeitado na 1ª conexão).
|
||||
openssl x509 -in "$workdir/$CERT_NAME" -outform der -out "$workdir/$DER_NAME"
|
||||
|
||||
echo ">> Clonando ${GITEA_REPO}@${BRANCH} do Gitea..."
|
||||
encoded_pass=$(python3 - <<EOF
|
||||
import urllib.parse; print(urllib.parse.quote('''${GITEA_PASSWORD}''', safe=''))
|
||||
EOF
|
||||
)
|
||||
clone_url="${GITEA_URL%%/}"
|
||||
clone_url="${clone_url/:\/\//:\/\/${GITEA_USER}:${encoded_pass}@}/${GITEA_REPO}.git"
|
||||
git clone --depth 1 --branch "$BRANCH" "$clone_url" "$workdir/repo" 2>&1 | sed "s#${encoded_pass}#***#g"
|
||||
|
||||
if [[ -f "$workdir/repo/$CERT_DIR_IN_REPO/$CERT_NAME" && "${FORCE:-0}" != "1" ]]; then
|
||||
echo "ERRO: $CERT_DIR_IN_REPO/$CERT_NAME já existe no repo. Re-rodar com FORCE=1 sobrescreve" >&2
|
||||
echo " (o cert antigo deixa de valer — retrust no KEPServer necessário)." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p "$workdir/repo/$CERT_DIR_IN_REPO"
|
||||
cp "$workdir/$CERT_NAME" "$workdir/$KEY_NAME" "$workdir/$DER_NAME" "$workdir/repo/$CERT_DIR_IN_REPO/"
|
||||
|
||||
cd "$workdir/repo"
|
||||
git add "$CERT_DIR_IN_REPO"
|
||||
git -c user.name="opc-cert-script" -c user.email="platform@aignosi.com.br" \
|
||||
commit -m "Add OPC UA client certificate for KEPServer connection (URI ${APP_URI})"
|
||||
git push origin "$BRANCH" 2>&1 | sed "s#${encoded_pass}#***#g"
|
||||
|
||||
fingerprint=$(openssl x509 -in "$CERT_DIR_IN_REPO/$CERT_NAME" -noout -fingerprint -sha1)
|
||||
|
||||
cat <<EOF
|
||||
|
||||
== PRONTO ==
|
||||
Cert no Gitea: ${GITEA_REPO}@${BRANCH} -> ${CERT_DIR_IN_REPO}/{${CERT_NAME},${KEY_NAME},${DER_NAME}}
|
||||
${fingerprint}
|
||||
|
||||
Paths dentro do pod (o entrypoint clona o repo em /app):
|
||||
cert_path: /app/${CERT_DIR_IN_REPO}/${CERT_NAME}
|
||||
private_key_path: /app/${CERT_DIR_IN_REPO}/${KEY_NAME}
|
||||
|
||||
Próximos passos:
|
||||
1. Documento na collection OPC_servers com "uri": "${APP_URI}" (ver README/PR).
|
||||
2. kubectl -n sientia rollout restart deployment/opc-ingestor # re-clona o repo
|
||||
3. Pod tenta conectar -> no KEPServer: OPC UA Configuration Manager ->
|
||||
Trusted Clients -> cert '${CN}' rejeitado -> Trust -> restart runtime.
|
||||
EOF
|
||||
Reference in New Issue
Block a user