gitea_admin/sientia-dataops-opc-ingestor
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
252 Commits 16 Branches 0 Tags
0d60a3fb67c573283d16fce474a5b3f9a639b03c
Commit Graph

252 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
PedroHMCosme
0d60a3fb67 feat(gateway): optional .env loading for local dev (app + seed); document in e2e guide 2026-07-17 09:44:38 -03:00
PedroHMCosme
0e29cc4054 docs(gateway): e2e validation guide + seed script for the KepServer session 2026-07-15 21:45:06 -03:00
PedroHMCosme
e0d8dd3729 SIENTIAPDE-1972: F4 close-out - ledger de commits + gates de integracao (Task 21.end) 2026-07-15 15:33:03 -03:00
PedroHMCosme
cebb5215a3 chore(gateway): final sweep - README/functional Kafka cleanup, asyncua pin, 1879 checklist verification (Task 21) 2026-07-15 15:31:37 -03:00
PedroHMCosme
21d0b37105 docs(gateway): deploy notes for helm/images/platform repos (Task 19+20) 2026-07-15 14:52:50 -03:00
PedroHMCosme
3f413f8d6f fix(gateway): app survives runtime init failure; dedupe NATS-down notifications; fix stale slot_lookup note 2026-07-15 14:17:21 -03:00
PedroHMCosme
391268776c feat(gateway): wire engines+adapters via GatewayRuntime (write/gRPC live; subscription v2 gated on 1974) 
GatewayRuntime builds and owns the F4 object graph (CertProvider/pool ->
WriteEngine/NatsWriteConsumer -> ActiveReadEngine -> optional GrpcServer),
reusing the existing ingestor's logger/notification_handler/metrics_controller
and DataManager (CertProvider shares its Mongo connection instead of opening
a second one). app.py starts it after prepare_ingestor, sweeps idle OPC
connections each loop iteration, and stops it on shutdown. A NATS connect
failure at startup is logged/notified but never crashes the app -- the read
path keeps working and idle_sweep() retries the consumer start.

ingestor.py needed no changes: data_manager is already reachable via
ingestor.ingestor_manager.data_manager.

Also documents the new MinIO/Postgres env vars this wiring reads (via
sientia_do's build_postgres_config for Postgres) in .env.example, and
updates test_app.py's fixture to mock GatewayConfig/GatewayRuntime so the
existing app tests keep exercising only what they mocked before.
 2026-07-15 13:51:18 -03:00
PedroHMCosme
ff605eea93 feat(gateway): GatewayConfig env (NATS/gRPC/certs/write flags) 2026-07-15 08:59:19 -03:00
PedroHMCosme
e9ce9e50e2 SIENTIAPDE-1972: F3 close-out - ledger de commits + carry-forward notes (Task 16.end) 2026-07-15 08:50:59 -03:00
PedroHMCosme
52223e6c09 feat(gateway): gRPC server ReadTag (C4) 2026-07-15 08:39:54 -03:00
PedroHMCosme
1798b69c8b feat(gateway): gRPC proto + generated stubs (C4) 2026-07-15 08:36:08 -03:00
PedroHMCosme
99a715cc16 feat(gateway): ActiveReadEngine (scheduled + read_now) 2026-07-14 22:30:06 -03:00
PedroHMCosme
4855205aa4 SIENTIAPDE-1972: adiciona commit ponytail ao ledger F2 2026-07-14 22:24:28 -03:00
PedroHMCosme
f7f356fbd5 refactor(gateway): ponytail F2 - drop dead metadata, unreachable variant fallback, redundant except member 2026-07-14 22:24:02 -03:00
PedroHMCosme
ce0db62848 SIENTIAPDE-1972: F2 close-out - ledger de commits + carry-forward notes (Task 13.end) 2026-07-14 22:18:28 -03:00
PedroHMCosme
165ac79783 feat(gateway): NatsWriteConsumer (Core NATS, queue group) (C2) 2026-07-14 21:31:53 -03:00
PedroHMCosme
abff80e0d3 fix(gateway): tighten SQL identifier validation to fullmatch (reject trailing newline) 2026-07-14 19:31:11 -03:00
PedroHMCosme
905da5151b feat(gateway): DataSink.update_write_status Postgres by timestamp+model_id (C3); drop unused metadata param 2026-07-14 19:24:24 -03:00
PedroHMCosme
86d0f6cf23 fix(gateway): make WriteEngine + converter raise-safe against malformed write commands 2026-07-14 19:18:52 -03:00
PedroHMCosme
dcf8369638 feat(gateway): WriteEngine (status_code/timestamp/flags, safe converter, rate-limit) 2026-07-14 16:59:43 -03:00
PedroHMCosme
2a9af05464 SIENTIAPDE-1972: F1 close-out - ledger de commits + carry-forward notes (Task 10.end) 2026-07-14 16:33:42 -03:00
PedroHMCosme
00c74e5606 feat(gateway): SubscriptionEngine multi-name fan-out (C1) 2026-07-14 16:17:13 -03:00
PedroHMCosme
323bf24a0f refactor(gateway): remove legacy Kafka egress (MongoDB-only) 2026-07-14 15:58:55 -03:00
PedroHMCosme
ae88ac662e feat(gateway): DataSink.publish_reading multi-name fan-out 2026-07-14 10:51:52 -03:00
PedroHMCosme
44df022e32 feat(gateway): slot v2 loader (contract C1) 2026-07-14 10:47:33 -03:00
PedroHMCosme
00f919cec6 SIENTIAPDE-1972: plano - modelo de PR por fase + ledger de commits (F0 registrado) + close-out por fase 2026-07-14 10:37:09 -03:00
PedroHMCosme
b57be9f5fe refactor(gateway): drop dead error_stack and unused metadata from OpcConnectionPool (ponytail) 2026-07-14 10:06:59 -03:00
PedroHMCosme
0b29857089 fix(gateway): per-server lock in OpcConnectionPool to avoid duplicate connects 2026-07-14 09:50:14 -03:00
PedroHMCosme
a08fd88c80 test(ingestor_manager): force gc.collect() after each test to avoid __del__ flake 
IngestorManager.__del__ calls asyncio.run(self.shutdown()). Adding
tests/unit/connection/test_pool.py shifted GC timing enough that a
cyclic-referenced IngestorManager instance from an earlier test was
swept mid test___del__, double-calling the patched asyncio mock and
failing the assertion. Deterministic gc.collect() after each test in
this file keeps finalizers scoped to their own test.
 2026-07-14 09:36:01 -03:00
PedroHMCosme
5d30a5c547 feat(gateway): OpcConnectionPool (per-server, reuse, idle-evict) 2026-07-14 09:35:53 -03:00
PedroHMCosme
09e42901d1 SIENTIAPDE-1972: remover egress Kafka do Gateway (Mongo-only) no spec/plano 2026-07-14 09:06:53 -03:00
PedroHMCosme
bbe10a4f73 SIENTIAPDE-1972: resolve chamador do GRPC (sem consumidor hoje; interno, sem auth) 2026-07-14 08:51:21 -03:00
PedroHMCosme
ec2a324a84 feat(gateway): cert provider from MinIO+Mongo (sientia-dataops) 2026-07-13 19:20:41 -03:00
PedroHMCosme
b05c2fd76a feat(gateway): Basic256Sha256 + channel>session lifetime (1879) 2026-07-13 18:47:33 -03:00
PedroHMCosme
d6cb1e492d feat(gateway): monotonic clock helper (1879) 2026-07-13 18:39:23 -03:00
PedroHMCosme
55f67216ee feat(gateway): safe type converter (1879 BUG-1/BUG-2) 2026-07-13 18:31:07 -03:00
PedroHMCosme
3dd4f60b0e feat(gateway): add write/active-read/error metrics 2026-07-13 18:27:30 -03:00
PedroHMCosme
0694f6ac87 chore(gateway): add nats-py, grpcio deps (Task 1) 2026-07-13 18:24:32 -03:00
PedroHMCosme
dd41145139 SIENTIAPDE-1972: plano de implementacao do OPC Gateway (F0-F4, TDD) + revisao 2026-07-13 18:12:08 -03:00
PedroHMCosme
4225789dc8 SIENTIAPDE-1972: localiza criacao das tabelas de predicao (pandas to_sql), adiciona schema/table_name ao contrato NATS e fixa cert bucket aprovado 2026-07-13 17:46:43 -03:00
PedroHMCosme
7cffc2629d SIENTIAPDE-1972: confirma criacao dinamica das tabelas de predicao (coluna no template + backfill) 2026-07-13 17:25:27 -03:00
PedroHMCosme
c31db70c8a SIENTIAPDE-1972: fixa decisoes (JOIN=tag_id, certs=sientia-dataops, metadata vazio) e coluna de write_status via Flyway em tabelas por-modelo 2026-07-13 17:17:19 -03:00
PedroHMCosme
941c6c0029 SIENTIAPDE-1972: resolve bucket de certs (sientia-dataops), mecanica do JOIN 1974 e discrepancia do campo metadata (1600) 2026-07-13 17:09:07 -03:00
PedroHMCosme
8a05ff70da SIENTIAPDE-1972: resolve stack de migracao (Flyway) no spec 2026-07-13 16:58:54 -03:00
PedroHMCosme
524f4d6d3f SIENTIAPDE-1972: fixa NATS como broker decidido (brief/Vitor) e reflete rename da branch 2026-07-13 16:55:27 -03:00
PedroHMCosme
c980092d95 SIENTIAPDE-1972: enriquece design do OPC Gateway com padroes de plataforma (aig-docs) e corrige contrato de status de escrita 2026-07-13 16:47:59 -03:00
PedroHMCosme
6f6f3bf734 SIENTIAPDE-1972: design doc do OPC Gateway (arquitetura, contratos e faseamento) 2026-07-13 16:29:57 -03:00
Bruno Domingues
498030dd5f Merge pull request #28 from Aignosi/feature/SIENTIAPDE-1945 
SIENTIAPDE-1945: Remove values.yaml
 2026-07-10 11:50:21 -03:00
Bruno Domingues
878644c953 SIENTIAPDE-1945: Remove values.yaml. 2026-07-08 22:12:59 -03:00
Bruno Domingues
1e0c83828e chore(sonar): Update project key 2026-07-01 21:11:01 -03:00