gitea_admin/sientia-dataops-laborious_temporal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
50 Commits 8 Branches 0 Tags
6d3cbb1a1bcaa59eb12ed3100d4a589b2c52b837
Commit Graph

50 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Matheus Demoner
6d3cbb1a1b Merge pull request #6 from Aignosi/SIENTIAPDE-1102-ajustar-escrita-postgres-do-laborious-para-ocorrer-apos-a-escrita-opc 
SIENTIAPDE-1102
 2025-06-12 16:09:49 -03:00
vitor-aignosi
781e3a43bb SIENTIAPDE-1102 
Refactor OPC activity to enhance data writing and confidence processing; update tests accordingly
 2025-06-12 14:57:18 -03:00
Matheus Demoner
e3ef4853dd Merge pull request #5 from Aignosi/SIENTIAPDE-1100-rever-erros-nao-lancados-lancando-os-se-necessario 
Sientiapde 1100 rever erros nao lancados lancando os se necessario
 2025-06-12 10:41:53 -03:00
vitor-aignosi
7c4f3936e8 SIENTIAPDE-1100 
Add test for handling invalid data type in write_data method
 2025-06-12 09:14:15 -03:00
vitor-aignosi
1057db1d73 SIENTIAPDE-1100 
Improve error handling in OPC activities; raise exceptions instead of logging errors and add data type validation in OpcRepository
 2025-06-12 09:06:31 -03:00
vitor-aignosi
e8afa2673e SIENTIAPDE-1100 
Refactor error handling in Gates and OPC activities; replace raise statements with logging for better traceability
 2025-06-12 08:56:10 -03:00
vitor-aignosi
ba576636a0 SIENTIAPDE-1100 
Fix formatting in values.yaml by adding a newline at the end of the SSH secret creation comment
 2025-06-11 18:00:38 -03:00
vitor-aignosi
72876e3598 [SIENTIAPDE-1100] Fix formatting in values.yaml by ensuring consistent comment style for SSH secret creation 2025-06-11 17:59:38 -03:00
vitor-aignosi
fd9131bdd8 testing commit message 2025-06-11 17:56:58 -03:00
vitor-aignosi
fc1cd056af Refactor error handling in Gates and OPC activities; update OPC_NAME to OPC_ID in values.yaml 2025-06-11 17:55:27 -03:00
Matheus Demoner
d57ecc3041 Merge pull request #4 from Aignosi/SIENTIAPDE-1097-realizar-testes-basicos-no-cluster-suse-linux 
Sientiapde 1097 realizar testes basicos no cluster suse linux
 2025-06-10 13:50:31 -03:00
vitor-aignosi
948fecc444 SIENTIAPDE-1097 
Refactor filter configuration keys from 'POLICY' to 'policy' for consistency
 2025-06-10 11:51:27 -03:00
vitor-aignosi
98a931393b SIENTIAPDE-1097 
Update library versions in requirements.txt and bump image tag in values.yaml to 0.1.1
 2025-06-10 11:38:17 -03:00
vitor-aignosi
9430bc9424 Remove unused last_timestamp from input_data in prediction_process.py 2025-06-10 09:10:09 -03:00
vitor-aignosi
c02dae19a0 SIENTIAPDE-1097 
Update input filter function call and add last_timestamp to prediction_process
 2025-06-10 09:07:36 -03:00
vitor-aignosi
ba96ea988e Update model_id key to model in prediction_process.py 2025-06-10 08:53:05 -03:00
vitor-aignosi
881e7232e9 SIENTIAPDE-1094 
fix: update variable names for consistency and bump image tag version
 2025-06-10 08:48:44 -03:00
vitor-aignosi
17b98392a7 SIENTIAPDE-1094 
Update notification handler parameters and change task queue name
 2025-06-10 08:39:19 -03:00
Bruno Domingues
ce92fddbab Merge pull request #3 from Aignosi/SIENTIAPDE-1094-atualizar-versao-da-lib 
Sientiapde 1094 atualizar versao da lib
 2025-06-09 13:34:49 -03:00
vitor-aignosi
497e7a2eaa conflicts 2025-06-09 12:44:39 -03:00
vitor-aignosi
2f72ce1cef Merge branch 'main' into SIENTIAPDE-1094-atualizar-versao-da-lib 2025-06-09 12:41:42 -03:00
vitor-aignosi
10081b71d2 SIENTIAPDE-1081 
Enhance documentation across multiple modules with detailed parameter descriptions and usage examples
 2025-06-09 12:05:26 -03:00
vitor-aignosi
5326051714 SIENTIAPDE-1094 
Refactor activity imports and remove unused base and logger files; update requirements for library versioning
 2025-06-09 11:08:07 -03:00
vitor-aignosi
11f41f358a values fix 2025-05-28 09:59:22 -03:00
vitor-aignosi
36a2fbbc13 name and variable fixes 2025-05-28 09:40:05 -03:00
vitor-aignosi
cfb217963b SIENTIAPDE-994 
Add shutdown tests for Activities and OPC classes; improve test assertions
 2025-05-28 09:24:16 -03:00
vitor-aignosi
b3db8d1190 ignore keys 2025-05-28 08:54:16 -03:00
vitor-aignosi
325d22f241 SIENTIAPDE-994Add shutdown methods for Activities and OPC classes; enhance worker error handling 2025-05-28 08:23:50 -03:00
vitor-aignosi
6f81ce7d03 SIENTIAPDE-994 
Refactor logging in MLFlow and Gates activities; remove print statement in FormatAndExportPrediction; update data handling in PredictionProcess; delete unused redis-feeder script.
 2025-05-27 16:49:31 -03:00
Matheus Demoner
dcb3e651ca Merge pull request #2 from Aignosi/SIENTIAPDE-1081-desenvolver-a-documentacao-do-modulo 
SIENTIAPDE-1081 Enhance documentation across multiple modules with detailed parameter descriptions and usage examples
 2025-05-26 17:07:54 -03:00
vitor-aignosi
49fa60c66f SIENTIAPDE-1081 
Enhance documentation across multiple modules with detailed parameter descriptions and usage examples
 2025-05-26 16:45:50 -03:00
vitor-aignosi
bc3f26c280 Merge pull request #1 from Aignosi/SIENTIAPDE-994-implementar-os-workflows-mapeados-utilizando-as-workers-e-activities-apropriadas 
Sientiapde 994 implementar os workflows mapeados utilizando as workers e activities apropriadas
 2025-05-26 14:32:12 -03:00
vitor-aignosi
35c552e04f sonar issues 2025-05-26 13:41:08 -03:00
vitor-aignosi
01823554e0 test and coverage fixes 2025-05-26 12:25:21 -03:00
vitor-aignosi
7600d22a61 Merge branch 'main' into SIENTIAPDE-994-implementar-os-workflows-mapeados-utilizando-as-workers-e-activities-apropriadas 2025-05-26 12:19:35 -03:00
vitor-aignosi
09c0fc4043 fix 2025-05-26 12:09:07 -03:00
Bruno Domingues
d700244d56 Add quality gate workflow. 2025-05-26 12:07:42 -03:00
vitor-aignosi
ad00661516 SIENTIAPDE-994 
Refactor tests for Postgres activities and improve error handling

- Updated test_postgres.py to enhance the testing of load_custom_query method, including cases for None data and date conversion.
- Refactored repeat_last_prediction tests to use mocks for SQLAlchemy session execution.
- Added tests for export_data_to_postgres method, covering both success and error scenarios.
- Improved the initialization tests for Activities class to ensure proper instantiation of dependencies.
- Enhanced test coverage for OPC repository connection validation.
- Updated tests for prediction workflows to streamline input handling and improve clarity.
- Introduced tests for connectors configuration to validate environment variable handling for MLFlow, OPC, and Postgres.
- Added tests for logger utility to ensure default settings are correctly applied.
 2025-05-26 11:22:50 -03:00
vitor-aignosi
67fe4afaa6 SIENTIAPDE-994 
Refactor and enhance the laborious workflow and utilities

- Removed outdated test file `test_predictions_batch.py` from workflows.
- Added `input_sample.json` for standardized input configuration.
- Introduced `connectors_config.py` to manage database and service configurations.
- Implemented a logging utility in `logger.py` for consistent logging across the application.
- Created `policies.py` to define retry policies for workflows.
- Developed comprehensive tests for `MLFlowRepository` in `test_model_repository.py`.
- Added extensive tests for `OpcRepository` in `test_opc_repository.py`.
- Updated `test_predictions_batch.py` to reflect new workflow structure and testing methodology.
 2025-05-23 17:34:47 -03:00
Bruno Domingues
77b3859be1 Add quality gate workflow 2. 2025-05-14 17:19:05 -03:00
Bruno Domingues
239886a263 Add quality gate workflow. 2025-05-14 16:17:54 -03:00
vitor-aignosi
5fe552410b SIENTIAPDE-994 
Worker fixes
 2025-05-13 12:01:12 -03:00
vitor-aignosi
1156037e08 SIENTIAPDE-994 
Refactor activity preparation methods and update Docker configuration for simulator
 2025-05-13 10:17:39 -03:00
vitor-aignosi
d22545ba88 SIENTIAPDE-994 
Finishing base pipelines, preparing to testing
 2025-05-12 14:50:20 -03:00
vitor-aignosi
eb6d2dd79c SIENTIAPDE-994 
Implement new get_last_timestamp method in Gates class, refactor MLFlow activity methods to return only transformed data, and update PredictionsBatch and PredictionProcess workflows to utilize Activities module. Add detailed docstrings for new methods and enhance test coverage for get_last_timestamp functionality.
 2025-05-12 11:36:21 -03:00
vitor-aignosi
d09fb6ac5e SIENTIAPDE-994 
Refactor activity methods and update requirements.txt to enhance functionality and remove deprecated filters. Added detailed docstrings for clarity and improved error handling in data processing workflows.
 2025-05-09 16:27:57 -03:00
vitor-aignosi
43f19ed93a SIENTIAPDE-994 
Update requirements.txt with new dependencies and refactor activity methods for improved functionality and error handling
 2025-05-08 17:01:40 -03:00
vitor-aignosi
e7f214b144 SIENTIAPDE-994 
Remove unused utility files and update requirements.txt to include new dependencies for data processing and database interaction.
 2025-05-07 17:39:45 -03:00
Matheus
7fb63778d4 Atualiza arquivos de configuração do projeto, ajustando o Dockerfile e requirements.txt para melhor compatibilidade e desempenho. 2025-04-15 10:53:56 -03:00
Matheus
ca9f5488bf Adiciona arquivos de configuração do projeto, incluindo .gitignore, Dockerfile, Makefile e requirements.txt 2025-04-15 10:28:47 -03:00