Update project configuration and dependencies
- Added .mypy_cache and .cursor to .gitignore.
- Changed asyncio_default_fixture_loop_scope and asyncio_default_test_loop_scope to "session" in pyproject.toml.
- Updated e2e testing dependencies in requirements-dev.txt, replacing fakeredis and mongomock with pytest-httpserver.
- Updated requirements.txt to use sientia_do instead of a specific git commit.
- Modified sonar-project.properties to remove a file from coverage exclusions.
- Enhanced E2E test fixtures in e2e/conftest.py for better container management.
- Cleaned up e2e test files related to CoreScouter and PIWebAPIScouter workflows.
Update tests.ipynb and API class in scouter module
- Reset execution counts in tests.ipynb for reproducibility.
- Removed unnecessary outputs and adjusted execution counts for clarity.
- Enhanced API class to accept 'end_time' parameter for improved data retrieval flexibility.
- Commented out timestamp formatting and normalization for future adjustments.
SIENTIAPDE-1441: Refactor worker configuration in worker.py to utilize a new prepare_worker function for cleaner setup. Update environment variable names in values.yaml for consistency and clarity, adjusting memory requests and enhancing health check commands for improved application monitoring.
Update values.yaml to modify liveness and readiness probe configurations, enhancing application health checks with improved metrics validation and adjusted timing parameters for better responsiveness.
Update README.md and values.yaml to enhance system description and worker configuration. Added ML model orchestration to the system overview, introduced new debug mode features, and updated environment variable values to be strings for consistency.
SIENTIAPDE-1442: Enhance Temporal worker configuration by adding environment variables for concurrency limits and poller behavior settings in values.yaml and worker.py for optimized performance.
Remove encrypt.py and test.ipynb files; update values.yaml to set image tag to 1.0.0 and change GITHUB_BRANCH to release/SIENTIAPDE-1441 for version consistency.
Update GITHUB_BRANCH in values.yaml to feature/SIENTIAPDE-1273, increment Helm chart version to 0.6.0, and modify metrics registration in Gates class to skip None values. Update test cases to reflect changes in tag values.
Update requirements to use the remote sientia-dataops-library repository and increment image tag in values.yaml from 0.4.9 to 0.5.0 for version consistency.
Update environment configuration and refactor activities to include metrics controller. Remove Kafka settings and adjust Redis and MongoDB initialization. Update tests to reflect changes in initialization and metrics tracking.
Update GITHUB_BRANCH in values.yaml to feature/SIENTIAPDE-1325 for specific external operation metrics. Refactor shutdown methods in Activities class to ensure proper closure of MongoDB, Redis, and Gates connections. Adjust tests to reflect these changes and enhance repository pattern implementation for MongoDB and Redis activities.
Remove deprecated files and configurations, including .env, Dockerfile, docker-compose.yml, and client-schedule.py. Update README.md to reflect new architecture and features, enhancing clarity on system capabilities and workflows. Adjust values.yaml for image tag and replica count, and improve code documentation across various modules for better maintainability.
Update dependencies and improve observability by changing logger imports. Bump sientia-dataops-library version to 1.4.1 and update image tag to 0.4.2 in values.yaml.
Enable SDK metrics service in values.yaml and update worker.py to initialize SDK metrics server with Prometheus configuration, enhancing observability of the application.
Update image tag to 0.4.0 in values.yaml, modify GITHUB_BRANCH for SIENTIAPDE-1169, and enhance Redis activity to handle model tags for improved data retention.
Update requirements for sientia-dataops-library to version 1.3.7, add ServiceMonitor configuration for Prometheus in values.yaml, and refactor pod_id usage in Gates and Redis activities for improved consistency in metric tracking.
Update replica count in values.yaml, add application status metrics in metrics.py, and implement Prometheus server in worker.py for enhanced observability.
Update sientia-dataops-library dependency version from 1.3.0 to 1.3.3 in requirements.txt and change GITHUB_BRANCH in values.yaml to SIENTIAPDE-1163-alterar-dinamica-de-notificacoes-para-usar-o-mongodb-ao-inves-do-kafka for improved notification handling.
Update GITHUB_BRANCH in values.yaml to SIENTIAPDE-1154-otimizar-conexao-com-banco-de-dados-e-paralelismo for enhanced database connection optimization and parallelism.
Update values.yaml to change GITHUB_BRANCH to SIENTIAPDE-1148-separar-scouter-laborious-e-orchestrator-por-namespaces and set TEMPORAL_NAMESPACE to 'scouter' for improved environment configuration.
Update values.yaml to increase replicaCount from 1 to 5 and change GITHUB_BRANCH value to SIENTIAPDE-1141-criar-testes-de-carga for enhanced testing and scalability.
Refactor Activities class to remove Kafka and Druid dependencies, simplifying initialization. Update values.yaml to set replica count to 1 for reduced resource usage. Adjust Redis activity to set TTL to None for better data retention. Remove unused Kafka and Druid activity files and their associated tests, streamlining the codebase.
Update test.ipynb to generate and save multiple pipeline specifications and input samples in JSON format. Modify values.yaml to increase replica count from 1 to 3 and adjust PostgreSQL max connections from 20 to 40 for improved scalability.
Update Druid configuration in values.yaml to change port from 8081 to 8888. Refactor pydruid.py to use druid_engine for SQLAlchemy connections, enhancing clarity and consistency in data loading operations.
Enhance Druid activity by adding configuration for Druid host and port in values.yaml. Refactor data loading logic to utilize SQLAlchemy for querying Druid, improving query efficiency and readability. Update timestamp handling to streamline data retrieval process.
Update Druid activity to handle None values for last data timestamp, ensuring robust date parsing. Bump image tag in values.yaml to 0.2.3 for version consistency.
