b203b7d22c
Implement workflows for fake data generation, scouter processing, and core scouter operations - Added `FakeData` workflow to generate random data and send it to a Kafka topic. - Implemented `Scouter` workflow to load data from Kafka and trigger the core scouter workflow. - Created `CoreScouter` workflow to process data through quality gates, aggregation, and export to PostgreSQL. - Developed comprehensive unit tests for activities and workflows, ensuring proper functionality and error handling. - Enhanced Redis and Postgres activities with robust testing for data handling and error notifications. - Introduced quality filters for data validation and implemented tests to verify their functionality.
8 lines
172 B
Plaintext
8 lines
172 B
Plaintext
temporalio
|
|
psycopg2-binary
|
|
sqlalchemy
|
|
asyncua
|
|
redis
|
|
git+ssh://git@github.com/Aignosi/sientia-dataops-library.git
|
|
git+ssh://git@github.com/Aignosi/sientia-mlops-library.git
|