SIENTIAPDE-1974: fix(slots): unsubscribe a slot from the readers that hold it
The main loop snapshots managed_tags, then two steps remove a slot from
managed_tags before update_ingestor_manager compares the snapshot with the
current state and calls unsubscribe_slot for every slot that disappeared:
- manage_leases drops an extra lease ("Extra slots available"): it already
unsubscribed the slot, so the second call is redundant;
- update_slot_config finds the slot gone from Redis (deleted by the
orchestrator): nothing unsubscribed it yet.
unsubscribe_slot looked the slot up in managed_tags[slot] to find its servers,
so both cases raised KeyError, which stopped the main loop and restarted the
container (seen on dev on 2026-10-06 18:30 during a rollout, ~40 s without
reads). In the second case the subscription also stayed open on the OPC server.
The same code is on main, so production has it too.
unsubscribe_slot now asks the readers themselves: every reader (ServerReader or
the legacy OpcManager, both keep `subscriptions` by slot) holding a
subscription for the slot unsubscribes it. A slot no reader holds is a no-op.
This also covers a resubscription whose new config dropped a server: the old
server's subscription for that slot is closed too.
Tests: unit tests for the new lookup and the double call, plus two regressions
at the Ingestor level with the real unsubscribe_slot (dropped extra lease;
slot removed from Redis). Without the fix they fail with the same KeyError at
ingestor_manager.py:558. 280 unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -538,26 +538,28 @@ class IngestorManager(SientiaMonitoring):
|
||||
|
||||
async def unsubscribe_slot(self, slot: str):
|
||||
"""
|
||||
Unsubscribes a specific slot from all associated OPC servers.
|
||||
Unsubscribes a specific slot from every OPC reader that holds a subscription for it.
|
||||
|
||||
This method removes all subscriptions for a given slot across all
|
||||
OPC servers that were managing it. It ensures clean cleanup of
|
||||
resources when slots are released or reconfigured.
|
||||
The readers' own subscriptions are the source of truth, not managed_tags: the
|
||||
main loop removes a slot from managed_tags before update_ingestor_manager asks
|
||||
to unsubscribe it, both when manage_leases drops an extra lease (which already
|
||||
unsubscribed it) and when update_slot_config finds the slot gone from Redis
|
||||
(which did not). Looking the slot up in managed_tags raised KeyError in both
|
||||
cases, which stopped the main loop and restarted the container; the second case
|
||||
also left the subscription open on the OPC server. Unsubscribing a slot that no
|
||||
reader holds is a no-op.
|
||||
|
||||
Args:
|
||||
slot (str): The name of the slot to unsubscribe.
|
||||
|
||||
Raises:
|
||||
KeyError: If the specified slot does not exist in the managed tags.
|
||||
|
||||
Side Effects:
|
||||
- Removes subscriptions from all OPC servers for the specified slot
|
||||
- Removes the slot's subscription from every OPC reader that has one
|
||||
- Cleans up subscription resources on the OPC servers
|
||||
"""
|
||||
|
||||
for server in self.managed_tags[slot].keys():
|
||||
if server in self.opc_managers:
|
||||
await self.opc_managers[server].unsubscribe(slot)
|
||||
for reader in self.opc_managers.values():
|
||||
if slot in reader.subscriptions:
|
||||
await reader.unsubscribe(slot)
|
||||
|
||||
async def update_slot_config(self):
|
||||
"""
|
||||
|
||||
Reference in New Issue
Block a user