f55604c7db
Refactor tests for async compatibility and enhance IngestorManager functionality - Updated test cases in `test_app.py`, `test_ingestor.py`, and `test_ingestor_manager.py` to use async/await syntax for improved concurrency. - Refactored methods in IngestorManager and related classes to support asynchronous operations, ensuring non-blocking behavior during execution. - Enhanced mock setups in tests to accommodate async methods, improving test reliability and performance.
sientia-dataops-opc-ingestor
OPC gateway to manage Scouter pipelines
Local tests
Generate your ssh key to Docker
ssh-keygen -t ed25519 -C "docker-access" -f ~/.ssh/id_ed25519_docker
Add the public key to yout Git SSH keys
Enable Docker BuildKit
export DOCKER_BUILDKIT=1
or make it permanent:
echo '{ "features": { "buildkit": true } }' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker
Run docker compose
docker compose down -v
docker compose build --ssh default=$HOME/.ssh/id_ed25519_docker
docker compose up -d
Run Ingestor
The ingestor can be run as a docker container or as a python script.
As a docker container: Uncomment the ingestor service in docker-compose.yaml
docker compose up -d ingestor
As a python script:
python -m ingestor.app
Populate redis server
Create venv with python3.11
python3.11 -m venv venv
source ./venv/bin/activate
Install requirements
pip install -r requirements.txt
Run feeder
python simulator/redis-feeder.py
Unit tests
Install pytest
pip install pytest
Run pytest
pytest
Get current coverage
pip install pytest-cov
pytest --cov=ingestor
Generate complete report
pytest --cov=ingestor --cov-report=html
#PR shortcut
git log origin/main..HEAD --no-merges > git_log
Prompt: Write a summary of PR changes in markdown. Be objective and direct. Write to file