Commit Graph

163 Commits

Author SHA1 Message Date
vitor-aignosi
799aa5af3e SIENTIAPDE-1199
Refactor logging in worker.py to use custom_info and custom_error methods, enhancing log metadata with pod and workflow details for improved observability.
2025-08-21 11:36:37 -03:00
vitor-aignosi
76d96c2c13 Merge branch 'main' into SIENTIAPDE-1199-revisar-e-testar-observabilidade 2025-08-21 08:36:38 -03:00
Matheus Demoner
2637e7dc97 Merge pull request #15 from Aignosi/SIENTIAPDE-1169-pensar-e-projetar-testes-de-breakdown-e-performance
Sientiapde 1169 pensar e projetar testes de breakdown e performance
2025-08-20 12:49:07 -03:00
vitor-aignosi
2a76f2e3c6 SIENTIAPDE-1199
Enhance logging in Gates, MongoDB, and Redis activities by replacing debug statements with info level logs, improving observability of data processing steps. Update worker configuration to adjust concurrency settings and enable autoscaling for task polling.
2025-08-20 12:39:43 -03:00
vitor-aignosi
3bbc2c4993 SIENTIAPDE-1199
Update dependencies and improve observability by changing logger imports. Bump sientia-dataops-library version to 1.4.1 and update image tag to 0.4.2 in values.yaml.
2025-08-19 16:29:30 -03:00
vitor-aignosi
552f58ca86 SIENTIAPDE-1169
SIENTIAPDE-1169 Update Redis and core scouter tests to include model_tags in data structures, enhancing data organization and consistency across test cases.
2025-08-19 09:29:46 -03:00
vitor-aignosi
2c0f5a67e6 Merge branch 'main' into SIENTIAPDE-1169-pensar-e-projetar-testes-de-breakdown-e-performance 2025-08-18 15:20:23 -03:00
vitor-aignosi
9480495155 SIENTIAPDE-1169
Enable SDK metrics service in values.yaml and update worker.py to initialize SDK metrics server with Prometheus configuration, enhancing observability of the application.
2025-08-18 13:45:20 -03:00
vitor-aignosi
4bbd937b7f SIENTIAPDE-1169
Enhance Redis activity logging by adding debug statements for tag management and metrics registration, improving traceability of data processing.
2025-08-14 16:33:39 -03:00
vitor-aignosi
9303158a9d SIENTIAPDE-1169
Update image tag to 0.4.0 in values.yaml, modify GITHUB_BRANCH for SIENTIAPDE-1169, and enhance Redis activity to handle model tags for improved data retention.
2025-08-14 16:15:20 -03:00
Bruno Domingues
77b65bf740 Merge pull request #14 from Aignosi/SIENTIAPDE-1174-mapear-e-implementar-metricas-a-serem-criadas
Sientiapde 1174 mapear e implementar metricas a serem criadas
2025-08-13 16:34:49 -03:00
vitor-aignosi
ba994f9bb4 Merge branch 'main' into SIENTIAPDE-1174-mapear-e-implementar-metricas-a-serem-criadas 2025-08-05 14:04:46 -03:00
vitor-aignosi
c6fca2a8c2 SIENTIAPDE-1174
Update image tag to 0.3.2 in values.yaml for version consistency in deployment.
2025-08-04 15:26:21 -03:00
vitor-aignosi
d4718c7d95 SIENTIAPDE-1174
Update Helm upgrade command in values.yaml to use version 0.4.0, ensuring consistency in deployment instructions.
2025-08-04 10:04:01 -03:00
Bruno Domingues
07c54e9edd Merge pull request #13 from Aignosi/SIENTIAPDE-1172-criar-pipeline-de-alertas-orquestrador
SIENTIAPDE-1172
2025-08-04 10:02:39 -03:00
vitor-aignosi
25f9d89317 SIENTIAPDE-1174
Update image tag to 0.3.1 in values.yaml and modify write_metrics method signature in Gates activity for improved type clarity.
2025-08-01 08:36:08 -03:00
vitor-aignosi
40c57ed23d SIENTIAPDE-1174
Implement async test for write_metrics method in Gates activity, enhancing test coverage for metric logging functionality.
2025-08-01 07:40:06 -03:00
vitor-aignosi
fd740ca451 SIENTIAPDE-1174
Update Gates activity to use labels in LABORIOUS_DATA_WRITTEN_COUNT metric, improving clarity in metric tracking.
2025-07-31 16:34:55 -03:00
vitor-aignosi
7ea9c1ce65 SIENTIAPDE-1174
Update Gates and Redis activities to use 'workflow_name' instead of 'pipeline_name' in metric tracking, ensuring consistency across the codebase.
2025-07-31 16:31:38 -03:00
vitor-aignosi
b9c004827f SIENTIAPDE-1174
Update Redis activity to use metadata for model and pipeline names, improving consistency in metric tracking.
2025-07-31 16:27:18 -03:00
vitor-aignosi
a8222359cc SIENTIAPDE-1174
Refactor Scouter class to remove pod_id reference, enhancing code clarity and consistency in data handling.
2025-07-31 16:25:02 -03:00
vitor-aignosi
205810e882 SIENTIAPDE-1174
Update requirements for sientia-dataops-library to version 1.3.7, add ServiceMonitor configuration for Prometheus in values.yaml, and refactor pod_id usage in Gates and Redis activities for improved consistency in metric tracking.
2025-07-31 15:45:18 -03:00
vitor-aignosi
c69ae5df03 SIENTIAPDE-1174
Remove unused getenv import from scouter.py to enhance code clarity and maintainability.
2025-07-31 14:59:12 -03:00
vitor-aignosi
5d3eb74022 SIENTIAPDE-1174
SIENTIAPDE-1174 Add pod_id initialization in Activities class and update Scouter to reference it for improved consistency in environment variable handling.
2025-07-31 14:57:22 -03:00
vitor-aignosi
ce20d86328 SIENTIAPDE-1174
Remove unused metrics import from core_scouter.py to streamline code and improve clarity.
2025-07-31 14:48:00 -03:00
vitor-aignosi
b4827935a7 SIENTIAPDE-1174
Integrate write_metrics activity into CoreScouter for enhanced metric logging and update worker.py to include write_metrics in workflow tasks.
2025-07-31 14:43:34 -03:00
vitor-aignosi
1c78cfb558 SIENTIAPDE-1174
Add write_metrics activity to Gates for database metric logging and import metrics module
2025-07-31 14:39:58 -03:00
vitor-aignosi
7925b8a2ae SIENTIAPDE-1174
Update replica count in values.yaml, add application status metrics in metrics.py, and implement Prometheus server in worker.py for enhanced observability.
2025-07-31 13:01:39 -03:00
vitor-aignosi
2efad30304 SIENTIAPDE-1174
SIENTIAPDE-1174 Update dependencies, modify image tag, and enhance metrics tracking in various activities for improved observability and performance.
2025-07-31 12:52:07 -03:00
vitor-aignosi
37d196be14 SIENTIAPDE-1172
Update dependencies in requirements.txt and change image tag and GITHUB_BRANCH in values.yaml for versioning and pipeline adjustments.
2025-07-29 13:21:01 -03:00
Bruno Domingues
65efef796f Merge pull request #12 from Aignosi/SIENTIAPDE-1151-criar-testes-de-stress
Sientiapde 1151 criar testes de stress
2025-07-22 12:22:15 -03:00
vitor-aignosi
3b02a819ad Merge branch 'main' into SIENTIAPDE-1151-criar-testes-de-stress 2025-07-22 11:17:04 -03:00
Bruno Domingues
64a5bb0bbb Merge pull request #11 from Aignosi/SIENTIAPDE-1163-alterar-dinamica-de-notificacoes-para-usar-o-mongodb-ao-inves-do-kafka
Sientiapde 1163 alterar dinamica de notificacoes para usar o mongodb ao inves do kafka
2025-07-22 10:14:10 -03:00
vitor-aignosi
f6913da932 SIENTIAPDE-1151
Enhance test cases in Gates and Redis activities by adding 'name' field to data and updating Redis TTL to 5 hours for better data retention.
2025-07-22 08:16:21 -03:00
vitor-aignosi
b68b3b403b SIENTIAPDE-1151
SIENTIAPDE-1151 Filter data in Gates activity based on model tags to enhance data accuracy.
2025-07-21 12:03:03 -03:00
vitor-aignosi
aacd52b5c7 SIENTIAPDE-1151
Add debug logging for collected data in MongoDB activity
2025-07-21 11:12:39 -03:00
vitor-aignosi
a9b421b134 SIENTIAPDE-1151
Update GITHUB_BRANCH in values.yaml to SIENTIAPDE-1151-criar-testes-de-stress for stress testing implementation.
2025-07-18 16:21:18 -03:00
vitor-aignosi
b230ad614e SIENTIAPDE-1151
Update Redis set method to use a TTL of 5 hours for improved data retention.
2025-07-18 16:19:06 -03:00
vitor-aignosi
e6d2bac902 SIENTIAPDE-1163
Remove service configuration from values.yaml to streamline deployment settings.
2025-07-17 14:59:14 -03:00
vitor-aignosi
9580b6f8be SIENTIAPDE-1163
SIENTIAPDE-1163 Update test_redis.py to mock existing data in redis_activity.get for improved error handling in group and hold data tests.
2025-07-17 14:30:34 -03:00
vitor-aignosi
839be18644 SIENTIAPDE-1163
Update .gitignore to include .coverage and git_log for better file management
2025-07-17 12:33:27 -03:00
vitor-aignosi
b0d0478b20 SIENTIAPDE-1163
Add PR shortcut command to README for easier log generation
2025-07-17 12:32:38 -03:00
vitor-aignosi
939acb5354 SIENTIAPDE-1163
Update image tag in values.yaml from 0.2.3 to 0.2.4 for version increment.
2025-07-17 12:31:08 -03:00
vitor-aignosi
9329ae206c SIENTIAPDE-1163
Update sientia-dataops-library dependency version from 1.3.0 to 1.3.3 in requirements.txt and change GITHUB_BRANCH in values.yaml to SIENTIAPDE-1163-alterar-dinamica-de-notificacoes-para-usar-o-mongodb-ao-inves-do-kafka for improved notification handling.
2025-07-17 11:00:52 -03:00
vitor-aignosi
03f5518cd1 SIENTIAPDE-1163
Update sientia-dataops-library dependency version from 1.2.1 to 1.3.0 in requirements.txt
2025-07-16 09:23:02 -03:00
vitor-aignosi
95defccbf6 SIENTIAPDE-1163
Refactor notification handler imports across multiple files to use CoreNotificationHandler instead of NotificationHandler for consistency and clarity.
2025-07-16 08:59:21 -03:00
vitor-aignosi
b014375c3f SIENTIAPDE-1154
Update GITHUB_BRANCH in values.yaml to SIENTIAPDE-1154-otimizar-conexao-com-banco-de-dados-e-paralelismo for enhanced database connection optimization and parallelism.
2025-07-15 12:11:20 -03:00
Matheus Demoner
036e74ebad Merge pull request #10 from Aignosi/SIENTIAPDE-1148-separar-scouter-laborious-e-orchestrator-por-namespaces
Sientiapde 1148 separar scouter laborious e orchestrator por namespaces
2025-07-14 10:36:09 -03:00
vitor-aignosi
efeef628cb SIENTIAPDE-1148
Update sonar-project.properties to exclude worker.py from coverage, enhance error handling in worker.py, and add new tests for error scenarios in test_redis.py and test_mongo.py.
2025-07-11 12:32:23 -03:00
vitor-aignosi
72c87dfb51 SIENTIAPDE-1148
Update values.yaml to change GITHUB_BRANCH to SIENTIAPDE-1148-separar-scouter-laborious-e-orchestrator-por-namespaces and set TEMPORAL_NAMESPACE to 'scouter' for improved environment configuration.
2025-07-11 11:41:59 -03:00