gitea_admin/sientia-dataops-orchestrator_temporal
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

SIENTIAPDE-2011: normalize notification timestamp in format_log_report

Browse Source 
notification['timestamp'] crosses at least one Temporal activity
boundary (load_latest_data -> filter_notification_reports/alerts)
before reaching format_log_report. Temporal's default data converter
serializes datetime values to plain ISO-8601 strings without
restoring them on decode, and pymongo returns naive datetimes for
BSON Date fields, so the string can carry no UTC offset at all. That
broke export_data_to_postgres's strict to_datetime(..., format=
DATETIME_FORMAT_WITH_TZ) parsing downstream with:
  ValueError: time data "2024-06-01T12:00:00" doesn't match format
  "%Y-%m-%d %H:%M:%S%z"

Same underlying issue as SIENTIAPDE-2007/2009, fixed here using the
same pattern as SlotManager.put_last_data_timestamp: normalize with
pandas.to_datetime(..., utc=True) and re-format to
DATETIME_FORMAT_WITH_TZ before the value crosses the next boundary.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Eduardo Rios
2026-09-15 09:35:25 -03:00
parent ed666c260d
commit 58c0bd4f2d
2 changed files with 23 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
orchestrator/activities/formatters.py
View File

@@ -7,11 +7,15 @@ with workflow.unsafe.imports_passed_through():
from math import ceil
from typing import Any, TypedDict, cast
from pandas import DataFrame
from pandas import DataFrame, to_datetime
from sientia_do.notifications.handlers import CoreNotificationHandler as NotificationHandler
from sientia_do.notifications.models import NotificationLevel
from sientia_do.observability.sientia_monitoring import MetricsController, SientiaMonitoring
from sientia_do.temporal.constants import DATETIME_FORMAT_MS_WITH_TZ, now
from sientia_do.temporal.constants import (
DATETIME_FORMAT_MS_WITH_TZ,
DATETIME_FORMAT_WITH_TZ,
now,
)
from orchestrator.utils.orchestrator_functions import (
build_tag_config,
@@ -755,9 +759,24 @@ class Formatters(SientiaMonitoring):
key = f'{notification_id}:{trigger}'
if key not in data:
# ``notification['timestamp']`` started out as a native
# ``datetime``/BSON Date in Mongo, but by the time it gets here
# it has crossed at least one Temporal activity boundary:
# Temporal's default data converter serializes ``datetime``
# values to plain ISO-8601 strings and does not restore them on
# decode, and pymongo returns naive datetimes for BSON Date
# fields (no tzinfo), so the ISO string may carry no UTC offset
# at all. Normalizing with ``utc=True`` and re-formatting to
# ``DATETIME_FORMAT_WITH_TZ`` (same pattern as
# ``SlotManager.put_last_data_timestamp``) keeps the value a
# fixed-format string that ``export_data_to_postgres``'s
# ``timestamp_conversion`` can parse after the next boundary
# crossing, regardless of what shape it arrived in here.
data[key] = {
'status': group_config['status'],
'timestamp': notification['timestamp'],
'timestamp': to_datetime(notification['timestamp'], utc=True).strftime(
DATETIME_FORMAT_WITH_TZ
),
'groups': [group_name],
'message': notification['message'],
'level': notification['level'],