Remove deprecated files and configurations, including .env, Dockerfile, docker-compose.yml, and client-schedule.py. Update README.md to reflect new architecture and features, enhancing clarity on system capabilities and workflows. Adjust values.yaml for image tag and replica count, and improve code documentation across various modules for better maintainability.
Update dependencies and improve observability by changing logger imports. Bump sientia-dataops-library version to 1.4.1 and update image tag to 0.4.2 in values.yaml.
Enable SDK metrics service in values.yaml and update worker.py to initialize SDK metrics server with Prometheus configuration, enhancing observability of the application.
Update image tag to 0.4.0 in values.yaml, modify GITHUB_BRANCH for SIENTIAPDE-1169, and enhance Redis activity to handle model tags for improved data retention.
Update requirements for sientia-dataops-library to version 1.3.7, add ServiceMonitor configuration for Prometheus in values.yaml, and refactor pod_id usage in Gates and Redis activities for improved consistency in metric tracking.
Update replica count in values.yaml, add application status metrics in metrics.py, and implement Prometheus server in worker.py for enhanced observability.
Update sientia-dataops-library dependency version from 1.3.0 to 1.3.3 in requirements.txt and change GITHUB_BRANCH in values.yaml to SIENTIAPDE-1163-alterar-dinamica-de-notificacoes-para-usar-o-mongodb-ao-inves-do-kafka for improved notification handling.
Update GITHUB_BRANCH in values.yaml to SIENTIAPDE-1154-otimizar-conexao-com-banco-de-dados-e-paralelismo for enhanced database connection optimization and parallelism.
Update values.yaml to change GITHUB_BRANCH to SIENTIAPDE-1148-separar-scouter-laborious-e-orchestrator-por-namespaces and set TEMPORAL_NAMESPACE to 'scouter' for improved environment configuration.
Update values.yaml to increase replicaCount from 1 to 5 and change GITHUB_BRANCH value to SIENTIAPDE-1141-criar-testes-de-carga for enhanced testing and scalability.
Refactor Activities class to remove Kafka and Druid dependencies, simplifying initialization. Update values.yaml to set replica count to 1 for reduced resource usage. Adjust Redis activity to set TTL to None for better data retention. Remove unused Kafka and Druid activity files and their associated tests, streamlining the codebase.
Update test.ipynb to generate and save multiple pipeline specifications and input samples in JSON format. Modify values.yaml to increase replica count from 1 to 3 and adjust PostgreSQL max connections from 20 to 40 for improved scalability.
Update Druid configuration in values.yaml to change port from 8081 to 8888. Refactor pydruid.py to use druid_engine for SQLAlchemy connections, enhancing clarity and consistency in data loading operations.
Enhance Druid activity by adding configuration for Druid host and port in values.yaml. Refactor data loading logic to utilize SQLAlchemy for querying Druid, improving query efficiency and readability. Update timestamp handling to streamline data retrieval process.
Update Druid activity to handle None values for last data timestamp, ensuring robust date parsing. Bump image tag in values.yaml to 0.2.3 for version consistency.
Update environment configuration and refactor activity imports
- Changed Kafka, Redis, and Temporal host configurations to use localhost.
- Updated the version reference for the sientia-dataops-library in requirements.txt.
- Refactored import paths for activities to align with new module structure.
- Removed unused base.py and postgres.py files.
- Updated logger and policies imports to reflect new module locations.
- Adjusted values.yaml for branch and log level settings.
