sientia-dataops-opc-ingestor

OPC gateway to manage Scouter pipelines

Local tests

Generate your ssh key to Docker

ssh-keygen -t ed25519 -C "docker-access" -f ~/.ssh/id_ed25519_docker

Add the public key to yout Git SSH keys

Enable Docker BuildKit

export DOCKER_BUILDKIT=1

or make it permanent:

echo '{ "features": { "buildkit": true } }' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker

Run docker compose

docker compose down -v
docker compose build --ssh default=$HOME/.ssh/id_ed25519_docker
docker compose up -d

Run Ingestor

The ingestor can be run as a docker container or as a python script.

As a docker container: Uncomment the ingestor service in docker-compose.yaml

docker compose up -d ingestor

As a python script:

python -m ingestor.app

Populate redis server

Create venv with python3.11

python3.11 -m venv venv
source ./venv/bin/activate

Install requirements

pip install -r requirements.txt

Run feeder

python simulator/redis-feeder.py

Unit tests

Install pytest

pip install pytest

Run pytest

pytest

Get current coverage

pip install pytest-cov
pytest --cov=ingestor

Generate complete report

pytest --cov=ingestor --cov-report=html