SIENTIAPDE-1478

Enhance PI Web API Scouter Workflow and Documentation

- Added support for PI Web API data ingestion, including real-time and historical data retrieval.
- Implemented timestamp normalization to ensure consistency across records.
- Updated README.md to reflect new features and detailed workflow execution flow.
- Enhanced API class with comprehensive error handling and cleanup operations.
- Improved CoreScouter workflow with early exit conditions and detailed processing steps.
- Updated configuration parameters for better clarity and usability.
This commit is contained in:
vitor-aignosi
2026-01-09 09:49:55 -03:00
parent b75f499436
commit 4352c67f3a
5 changed files with 150 additions and 15 deletions

View File

@@ -32,10 +32,14 @@ class Scouter:
This method orchestrates the complete data ingestion process:
1. Retrieves the last processed timestamp from Redis
2. Loads new data from MongoDB since the last timestamp
3. Updates the last processed timestamp
2. Loads new data from MongoDB since the last timestamp using collection name
format: `raw_{schedule_name}`
3. Updates the last processed timestamp with the most recent data point
4. Delegates data processing to the CoreScouter workflow
If no new data is found in MongoDB, the workflow exits early without updating
the timestamp or invoking the CoreScouter workflow.
Args:
input_data (dict[str, Any]): Configuration and parameters for the workflow execution.
Required fields: