SIENTIAPDE-1974: fix(read): code-review fixes for the slot v2 read path

- config: GatewayConfig.certs_from_env() reads only CERT_BUCKET/CERT_CACHE_DIR;
  IngestorManager uses it, so a malformed write/gRPC env var (GRPC_PORT,
  WRITE_MIN_PAUSE_S, IDLE_CONN_MAX_S) can no longer stop the reads.
- certs: CertProvider raises CertificateNotConfiguredError with an actionable
  message when no platform_certificates document exists; downloads once under
  an asyncio.Lock, caches the paths, writes atomically (tmp + os.replace);
  invalidate() forces a fresh download.
- pool: server_id normalized to str (1 and '1' share a session); warns when a
  get() asks for other url/server_uri/use_security than the pooled client was
  opened with (client reused, the slot config wins); invalidates the cert
  cache when a secured connect fails.
- SubscriptionEngine: tracks nodes per subscription and forgets them on
  unsubscribe (no more cycle alarms / tag count for removed tags), keeping
  addresses still monitored by another slot.
- ServerReader.shutdown(evict) / IngestorManager.remove_server(evict): a
  server that only left this pod's slots keeps the shared connection for
  NATS writes/gRPC; lost servers and config changes still evict.
- IngestorManager warns once per server when a slot still sets
  server_cert_path (server certificate pinning is no longer applied).
- slots: CONNECTION_FIELDS exported and reused by ServerReader.

Accepted, not changed: read metrics label server_name with the server_id
(same as the write path, which only knows the id); a malformed slot is
notified each lease attempt (alerts are deduplicated by notification_id).

Tests: 236 unit tests pass (+18); ruff clean; local e2e against an asyncua
server incl. slot moved (connection kept and reused by the write path) and
lost-server eviction/re-initialization.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Eduardo Rios
2026-10-06 13:54:04 -03:00
parent 50578fdba8
commit 3d8ee1c2ed
13 changed files with 421 additions and 31 deletions

View File

@@ -314,7 +314,9 @@ async def test_update_opc_servers(metrics, ingestor_manager):
assert ingestor_manager.opc_managers['server2'].config == {'config': 'config2'}
assert ingestor_manager.opc_managers['server3'].config == {'config': 'config3'}
ingestor_manager.remove_server.assert_any_call('server4')
# server4 left this pod's slots: unsubscribe only, the connection stays for writes
ingestor_manager.remove_server.assert_any_call('server4', evict=False)
# server5 failed to initialize: removed with the default (evict)
ingestor_manager.remove_server.assert_any_call('server5')
assert ingestor_manager.remove_server.call_count == 2
@@ -738,3 +740,55 @@ async def test_update_slot_config_keeps_last_good_config_when_malformed(ingestor
assert ingestor_manager.managed_tags == {'slot1': good}
_, kwargs = ingestor_manager.send_notification.call_args
assert kwargs['notification_id'] == 'OPC_SLOT_CONFIG_ERROR_slot1'
@mark.asyncio
async def test_remove_server_forwards_evict(ingestor_manager):
reader = AsyncMock()
ingestor_manager.opc_managers = {'server1': reader}
ingestor_manager.managed_tags = {'slot1': {'server1': {}}}
await ingestor_manager.remove_server('server1', evict=False)
reader.shutdown.assert_awaited_once_with(evict=False)
assert 'server1' not in ingestor_manager.opc_managers
assert ingestor_manager.managed_tags == {'slot1': {}}
@patch('ingestor.managers.ingestor_manager.DataManager')
@patch('ingestor.managers.ingestor_manager.ResourceManager')
def test___init___ignores_malformed_write_env(
resource_manager_mock, data_manager_mock, monkeypatch
):
"""A malformed write/gRPC-only env var must not stop the read path (review #1)."""
monkeypatch.setenv('GRPC_PORT', '')
monkeypatch.setenv('WRITE_MIN_PAUSE_S', '0,5')
monkeypatch.setenv('CERT_BUCKET', 'streamlit-connectors')
ingestor = IngestorManager(
redis_data={'host': 'localhost', 'port': 6379},
lease_ttl=60,
heartbeat_ttl=60,
poll_interval=5,
mongo_connection_string='mongodb://localhost:27017',
mongo_database='sientia',
logger=MagicMock(),
notification_handler=MagicMock(),
metadata=metadata['metadata'],
metrics_controller=MagicMock(),
)
assert ingestor.cert_provider.bucket == 'streamlit-connectors'
def test_parse_slot_config_warns_once_about_server_cert_path(ingestor_manager):
raw = raw_slot_v1()
raw['server1']['server_cert_path'] = '/app/certs/kepserver.der'
ingestor_manager.parse_slot_config('1', raw)
ingestor_manager.parse_slot_config('1', raw) # every loop refreshes the slot
warnings = [
c for c in ingestor_manager.logger.warning.call_args_list if 'server_cert_path' in str(c)
]
assert len(warnings) == 1