Bruno Domingues
11aaa0780f
SIENTIAPDE-1350: Improve Docker image build context and fix queue names
...
This commit refactors the .dockerignore file to use a whitelist approach, significantly reducing the Docker image size by excluding unnecessary files. It also fixes the queue names in values.yaml and adds TEMPORAL_USE_TLS to the test environment.
2025-11-26 17:52:44 -03:00
Bruno Domingues
c281969381
SIENTIAPDE-1350: Add test to ensure worker starts despite cleanup schedule creation failure.
2025-11-26 11:30:22 -03:00
Bruno Domingues
bc917e97d8
SIENTIAPDE-1350: Add cleanup schedule tests and configure worker with task queues from env vars
2025-11-25 19:21:55 -03:00
Bruno Domingues
b028dc1b7d
SIENTIAPDE-1350: Implement scheduled cleanup workflow using Temporal schedules. Adds schedule creation to worker startup and configures environment variables.
2025-11-25 18:12:53 -03:00
Bruno Domingues
9ecac9c8fd
SIENTIAPDE-1350: Update values.yaml with new environment variables and remove outdated task from todo-list.txt
2025-11-25 17:03:06 -03:00
Bruno Domingues
823df2d65d
SIENTIAPDE-1350: Refactor: Replace AsyncMock with MagicMock in cleanup tests for synchronous calls
2025-11-25 16:54:45 -03:00
Bruno Domingues
33480fa108
SIENTIAPDE-1350: Add unit tests for the CleanupFiles workflow, covering both input bucket and default bucket scenarios.
2025-11-25 16:44:51 -03:00
Bruno Domingues
d0b5b74f88
SIENTIAPDE-1350: Add tests for close method and paginated list_bucket_objects
2025-11-25 16:04:59 -03:00
Bruno Domingues
d2c4803a75
SIENTIAPDE-1350: Add unit tests for the Cleanup activity, ensuring 100% code coverage, and update todo-list.
2025-11-25 16:01:10 -03:00
Bruno Domingues
f3c88885ea
SIENTIAPDE-1350: Configure cleanup test script and improve test isolation. This commit configures the cleanup test script to load environment variables from a .env file and use them for Temporal connection. It also adds a fixture to clean up temporary directories created by tests, ensuring better test isolation and preventing potential conflicts. Additionally, it adds the 'uri' property to the MongoDB config.
2025-11-25 00:44:55 -03:00
Bruno Domingues
463906073e
SIENTIAPDE-1350: Refactor: Improve logging, configuration, and resource management. Includes .env updates, client initialization logging, and resource closing.
2025-11-24 23:08:29 -03:00
Bruno Domingues
2ee66552d2
SIENTIAPDE-1350: Configure separate task queues for train and cleanup workers, update sientia-dataops-library to 1.6.1 and refactor prometheus server startup to use logger.
2025-11-24 11:21:43 -03:00
Bruno Domingues
53f49d7a97
SIENTIAPDE-1350: Implement file cleanup workflow and activities, including MinIO and local directory cleanup, configuration, and metrics.
2025-11-19 18:38:00 -03:00
Bruno Domingues
cea3ef60cb
Merge pull request #14 from Aignosi/feature/SIENTIAPDE-1349
...
SIENTIAPDE-1349: Add Grafana dashboard configuration
2025-11-11 11:46:49 -03:00
Bruno Domingues
731a23a92e
SIENTIAPDE-1349: Add Grafana dashboard configuration to values.yaml
2025-11-11 11:37:28 -03:00
Bruno Domingues
e51daca073
Merge pull request #13 from Aignosi/feature/SIENTIAPDE-1309
...
SIENTIAPDE-1309: Enhance Project Configuration, Documentation, and Deployment
2025-11-07 16:32:36 -03:00
Bruno Domingues
f232406bd3
SIENTIAPDE-1309: Improve .gitignore to preserve .gitkeep in model_manager reports temp directory.
2025-11-07 15:25:10 -03:00
Bruno Domingues
18103f2436
SIENTIAPDE-1309: Refine .gitignore to ignore test run reports within the model_manager/reports/temp directory, while preserving the temp directory itself.
2025-11-07 15:19:21 -03:00
Bruno Domingues
7ff54a1112
SIENTIAPDE-1309: Improve Dockerignore, Gitignore, Dockerfile, README, model repository, tests, todo list and values.yaml files.
2025-11-07 15:08:50 -03:00
Bruno Domingues
258bad84c6
SIENTIAPDE-1309: Add ignore rule for test run reports in model_manager/reports
2025-11-06 15:59:44 -03:00
Bruno Domingues
c7f44a2423
SIENTIAPDE-1309: Update README with Helm instructions and refactor experiment status messages. Also, update values.yaml with new image and configurations.
2025-11-06 15:34:03 -03:00
Bruno Domingues
a625ef6c19
SIENTIAPDE-1309: Add Helm chart documentation to README and update image tag in values.yaml
2025-11-05 21:26:58 -03:00
Bruno Domingues
005c33151c
Merge pull request #12 from Aignosi/feature/SIENTIAPDE-1308
...
SIENTIAPDE-1308: Dockerize project, update documentation, and improve dependency management
2025-11-04 15:42:06 -03:00
Bruno Domingues
d9d3df5723
SIENTIAPDE-1308: Initial Docker setup, improved .dockerignore, updated README, and minor fixes.
2025-11-04 14:07:37 -03:00
Bruno Domingues
9f56a128e6
SIENTIAPDE-1308: Update documentation, remove install_dependencies.sh script from README, correct Training class initialization, and update sientia-dataops-library version in requirements.txt.
2025-11-03 17:35:33 -03:00
Bruno Domingues
6ff2d9857d
feat: update todo list
...
This commit updates the todo list with new tasks and removes outdated ones.
The changes include:
- Adding tasks related to Grafana dashboard creation, library updates, and workflow improvements.
- Removing outdated tasks about test refactoring and readme updates.
2025-11-03 15:36:35 -03:00
Bruno Domingues
12f8b21625
Merge pull request #11 from Aignosi/feature/SIENTIAPDE-1307
...
SIENTIAPDE-1307: Implement and Integrate Metrics Tracking for Training Activities and Workflows
2025-11-03 15:26:58 -03:00
Bruno Domingues
f32d647d1c
SIENTIAPDE-1307: Implement metrics for training activities and workflow executions, tracking success/failure status.
2025-11-03 14:41:37 -03:00
Bruno Domingues
d8e8d3c481
SIENTIAPDE-1307: Fix PytestUnraisableExceptionWarning in activity and experiment tracking tests by controlling exception raising in __del__ mocks.
2025-10-31 17:21:27 -03:00
Bruno Domingues
a258036397
SIENTIAPDE-1307: Refactor: Remove redundant patching of SientiaMonitoring and Postgres init in tests, and add shutdown mock to metrics controller.
2025-10-31 17:15:59 -03:00
Bruno Domingues
db581f60f9
SIENTIAPDE-1307: Integrate metrics controller and update sientia-dataops-library to 1.5.1. This change adds metrics collection capabilities to activities and updates the dataops library dependency. (18 files changed, 143 insertions(+), 16 deletions(-))
2025-10-31 17:06:26 -03:00
Bruno Domingues
326f9f9dc6
Merge pull request #10 from Aignosi/feature/SIENTIAPDE-1241
...
Feature/SIENTIAPDE-1241: Corrigir bug ao treinar modelos com a flag de normalizar dados ativa
2025-10-30 17:04:51 -03:00
Bruno Domingues
5b14f1016b
SIENTIAPDE-1241: Suppress PytestUnraisableExceptionWarning and update test data in training repository tests.
2025-10-30 15:22:59 -03:00
Bruno Domingues
bb25a19d8c
SIENTIAPDE-1241: Simplify worker shutdown logic by removing conditional checks.
2025-10-30 15:14:43 -03:00
Bruno Domingues
f5c1ac3615
SIENTIAPDE-1241: Remove model equation implementation documentation and add test coverage for _generate_artifacts in ModelRepository.
2025-10-30 14:56:10 -03:00
Kou-Kinoshita
a5c0cb6e47
SIENTIAPDE-1241: Fixed formatting and linted code
2025-10-30 10:38:29 -03:00
Kou-Kinoshita
17a551d6f9
SIENTIAPDE-1241: Added remaining training repository tests
2025-10-30 10:29:51 -03:00
Kou-Kinoshita
7e9d6683e4
SIENTIAPDE-1241: Added worker test
2025-10-30 10:21:39 -03:00
Kou-Kinoshita
c63389620a
SIENTIAPDE-1241: Train_model tests
2025-10-30 10:11:07 -03:00
Kou-Kinoshita
928c679223
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-30 09:58:46 -03:00
Kou-Kinoshita
c699a47261
SIENTIAPDE-1241: worker tests
2025-10-30 09:52:34 -03:00
Kou-Kinoshita
94a407f1b8
SIENTIAPDE-1241: Storage repository tests
2025-10-30 09:49:10 -03:00
Kou-Kinoshita
87c64eac2e
SIENTIAPDE-1241: Metrics test
2025-10-30 09:47:23 -03:00
Kou-Kinoshita
16fba46520
SIENTIAPDE-1241: Deleted accidental .md
2025-10-29 16:47:42 -03:00
Kou-Kinoshita
26fca36d82
SIENTIAPDE-1241: Refactored method into smaller ones
2025-10-29 16:45:30 -03:00
Kou-Kinoshita
63fa264ec4
SIENTIAPDE-1241: Updated readme (removed port foward script))
2025-10-29 16:27:56 -03:00
Kou-Kinoshita
1f6765dc86
SIENTIAPDE-1241: Linted files
2025-10-29 11:57:40 -03:00
Kou-Kinoshita
5e763a8a64
SIENTIAPDE-1241: Formatted files
2025-10-29 11:55:03 -03:00
Kou-Kinoshita
e8250d8dd7
SIENTIAPDE-1241: fixed test
2025-10-29 11:49:08 -03:00
Kou-Kinoshita
64590032f3
SIENTIAPDE-1241: Added remaining model repository tests
2025-10-29 11:46:29 -03:00
