Merge branch 'feature/SIENTIAPDE-1972-gateway-write-read' into feature/SIENTIAPDE-1972-gateway-wiring
# Conflicts: # README.md # requirements.txt
This commit is contained in:
61
README.md
61
README.md
@@ -46,7 +46,7 @@ The OPC Ingestor uses a modular, manager-based architecture designed for scalabi
|
||||
- **Ingestor**: Main application orchestrator managing the overall lifecycle
|
||||
- **IngestorManager**: Coordinates slot allocation, OPC server management, and load balancing
|
||||
- **OPC Manager**: Handles individual OPC UA server connections and tag subscriptions
|
||||
- **Data Manager**: Manages data streaming (MongoDB)
|
||||
- **Data Manager**: Manages data persistence (MongoDB)
|
||||
- **Resource Manager**: Coordinates resource allocation and instance coordination via Redis
|
||||
|
||||
## 🚪 OPC Gateway (SIENTIAPDE-1972)
|
||||
@@ -123,6 +123,10 @@ health probe, NetworkPolicy, cert cache) and the `# OPC Gateway` block in
|
||||
# Ensure services are accessible on localhost with appropriate ports
|
||||
```
|
||||
|
||||
6. **Generate an OPC UA client certificate** (only if connecting to a secured OPC UA server)
|
||||
|
||||
See [Certificate Generation](#-certificate-generation) below.
|
||||
|
||||
## Usage
|
||||
|
||||
### Running the Ingestor
|
||||
@@ -173,7 +177,7 @@ pytest
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=ingestor --cov-report=html
|
||||
|
||||
```
|
||||
|
||||
### Populating Redis with OPC Configuration
|
||||
|
||||
@@ -189,6 +193,38 @@ pytest --cov=ingestor --cov-report=html
|
||||
|
||||
The feeder creates sample OPC tag configurations in Redis that the ingestor can discover and manage.
|
||||
|
||||
## 🔐 Certificate Generation
|
||||
|
||||
OPC UA servers that require secure connections (e.g. KEPServer with `Basic256` policy) validate the ingestor via an X.509 client certificate. Use `scripts/generate-and-push-opc-cert.sh` to generate that certificate and push it to the Gitea repo the pod clones on boot.
|
||||
|
||||
```bash
|
||||
GITEA_PASSWORD='<senha-do-gitea>' ./scripts/generate-and-push-opc-cert.sh
|
||||
```
|
||||
|
||||
Environment variables (all optional except `GITEA_PASSWORD`):
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `GITEA_URL` | Gitea in-cluster URL | `https://git.sientia.ai` |
|
||||
| `GITEA_USER` | Gitea user | `gitea_admin` |
|
||||
| `GITEA_PASSWORD` | Gitea password (**required**) | — |
|
||||
| `GITEA_REPO` | Repo the ingestor pod clones | `gitea_admin/sientia-dataops-opc-ingestor` |
|
||||
| `BRANCH` | Branch to push the cert to | `main` |
|
||||
| `APP_URI` | SAN URI of the cert — **must match** the `uri` field of the corresponding document in the `OPC_servers` Mongo collection, since the ingestor uses it as `application_uri` (KEPServer rejects with `BadCertificateUriInvalid` on mismatch) | `urn:sientia:opc-ingestor` |
|
||||
| `CN` | Certificate common name | `sientia-opc-ingestor` |
|
||||
| `DAYS` | Certificate validity (days) | `3650` |
|
||||
| `FORCE` | Set to `1` to overwrite an existing cert already pushed to the repo | unset |
|
||||
|
||||
What the script does:
|
||||
1. Generates a self-signed RSA-2048 X.509 cert + key with `openssl`, with the SAN URI, `keyUsage`, and `extendedKeyUsage` extensions OPC UA / KEPServer require.
|
||||
2. Clones the ingestor's Gitea repo, copies the cert/key (`.pem`) and a `.der` copy (for manual import) into `certs/`, commits, and pushes.
|
||||
3. Prints the paths the pod will see after cloning (`/app/certs/opc-ingestor-cert.pem`, `/app/certs/opc-ingestor-key.pem`) and the next manual steps.
|
||||
|
||||
After pushing, you still need to:
|
||||
- Set `cert_path` / `private_key_path` (and `uri`) on the server's document in the `OPC_servers` collection so `OpcManager` picks them up (see [OPC Server Configuration](#opc-server-configuration)).
|
||||
- Restart the ingestor deployment so it re-clones the repo: `kubectl -n sientia rollout restart deployment/opc-ingestor`.
|
||||
- On first connection, the server rejects the new cert — trust it in the OPC UA server's console (e.g. KEPServer: *OPC UA Configuration Manager → Trusted Clients → Trust*) and restart its runtime.
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Unit Tests
|
||||
@@ -270,7 +306,7 @@ gateway-specific environment variables.
|
||||
|
||||
### OPC Server Configuration
|
||||
|
||||
OPC servers are configured through Redis with the following structure:
|
||||
OPC servers are configured through Redis with the following structure. `cert_path` / `private_key_path` are only required for secured connections (see [Certificate Generation](#-certificate-generation)):
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -280,6 +316,8 @@ OPC servers are configured through Redis with the following structure:
|
||||
"server_id": "1",
|
||||
"url": "opc.tcp://localhost:4841",
|
||||
"server_uri": "http://opcua-server.simulator",
|
||||
"cert_path": "/app/certs/opc-ingestor-cert.pem",
|
||||
"private_key_path": "/app/certs/opc-ingestor-key.pem",
|
||||
"tags": {
|
||||
"ns=2;i=2": {
|
||||
"aggr_func": "avg",
|
||||
@@ -316,8 +354,8 @@ sientia-dataops-opc-ingestor/
|
||||
│ ├── ingestor.py # Core ingestor logic
|
||||
│ └── metrics.py # Prometheus metrics definitions
|
||||
├── simulator/ # OPC simulation and testing tools
|
||||
├── scripts/ # Ops scripts (e.g. OPC UA cert generation)
|
||||
├── tests/ # Test suite
|
||||
├── docker-compose.yaml # Infrastructure services
|
||||
└── requirements.txt # Python dependencies
|
||||
```
|
||||
|
||||
@@ -387,10 +425,6 @@ export LOG_LEVEL=DEBUG
|
||||
- Use type hints where appropriate
|
||||
- Follow the established architectural patterns
|
||||
|
||||
## 📄 License
|
||||
|
||||
This project is licensed under the terms specified in the LICENSE file.
|
||||
|
||||
## 🆘 Support
|
||||
|
||||
For support and questions:
|
||||
@@ -402,14 +436,3 @@ For support and questions:
|
||||
---
|
||||
|
||||
**Note**: This OPC Ingestor is designed for production use in industrial environments. Ensure proper security configuration and network isolation for production deployments.
|
||||
|
||||
This comprehensive documentation provides:
|
||||
|
||||
1. **Complete feature overview** with architectural details
|
||||
2. **Detailed installation and setup instructions**
|
||||
3. **Comprehensive configuration documentation**
|
||||
4. **Performance tuning and troubleshooting guides**
|
||||
5. **Development guidelines and contribution standards**
|
||||
6. **Updated docstrings** for all major classes and methods
|
||||
|
||||
The documentation now serves as a complete reference for users, developers, and operators of the OPC Ingestor system.
|
||||
Reference in New Issue
Block a user