Commit Graph

6 Commits

Author SHA1 Message Date
vitor-aignosi
4a90b77786 SIENTIAPDE-1110
Update sientia-dataops-library version to 1.2.0 in requirements.txt; refactor logging in activities to use a unified Logger instance and include metadata in log messages across various activities.
2025-06-26 15:15:31 -03:00
vitor-aignosi
cefb5e48ff SIENTIAPDE-1005
Unit test fixes
2025-05-19 15:52:39 -03:00
vitor-aignosi
df8a0fc706 SIENTIAPDE-1005
Enhance README and configuration files; add Redis authentication and update workflows
2025-05-16 16:30:43 -03:00
vitor-aignosi
e252ca7962 SIENTIAPDE-1005
Full coverage

Refactor code structure and improve logging configuration; update tests for activities and connectors
2025-05-16 12:58:36 -03:00
vitor-aignosi
b203b7d22c SIENTIAPDE-1005
Implement workflows for fake data generation, scouter processing, and core scouter operations

- Added `FakeData` workflow to generate random data and send it to a Kafka topic.
- Implemented `Scouter` workflow to load data from Kafka and trigger the core scouter workflow.
- Created `CoreScouter` workflow to process data through quality gates, aggregation, and export to PostgreSQL.
- Developed comprehensive unit tests for activities and workflows, ensuring proper functionality and error handling.
- Enhanced Redis and Postgres activities with robust testing for data handling and error notifications.
- Introduced quality filters for data validation and implemented tests to verify their functionality.
2025-05-15 16:53:24 -03:00
vitor-aignosi
4e579dd5bd SIENTIAPDE-1005
Add initial implementation of data processing activities and filters
2025-05-13 16:58:03 -03:00