SIENTIAPDE-1231
Enhance README and code documentation for clarity and structure - Added a comprehensive Table of Contents to the README for easier navigation. - Introduced a new section on Code Quality & Validation, detailing the validation workflow and tools used. - Improved descriptions and consistency in docstrings across various modules, including metrics, formatters, and MongoDB activities. - Clarified function return types and arguments in the MongoDB and TemporalManager classes for better understanding.
This commit is contained in:
97
README.md
97
README.md
@@ -2,6 +2,36 @@
|
||||
|
||||
A high-performance, scalable workflow orchestration system built on Temporal.io for automated pipeline management, notification delivery, and resource coordination. The Orchestrator provides enterprise-grade workflow automation, real-time alerting, and comprehensive monitoring capabilities for the SIENTIA platform.
|
||||
|
||||
## 📑 Table of Contents
|
||||
|
||||
- [Features](#features)
|
||||
- [Core Functionality](#core-functionality)
|
||||
- [Advanced Capabilities](#advanced-capabilities)
|
||||
- [Architecture](#architecture)
|
||||
- [Architecture Principles](#architecture-principles)
|
||||
- [Task Queue Isolation](#2-task-queue-isolation)
|
||||
- [Workflows](#-workflows)
|
||||
- [Orchestrator Workflow](#1-orchestrator-workflow-orchestratorpy)
|
||||
- [Alerts Workflow](#2-alerts-workflow-alertspy)
|
||||
- [Reports Workflow](#3-reports-workflow-reportspy)
|
||||
- [Subworkflows](#subworkflows)
|
||||
- [Load Notification Package](#1-load-notification-package-load_notification_packagepy)
|
||||
- [Process Notifications](#2-process-notifications-process_notificationspy)
|
||||
- [Notification Filtering System](#-notification-filtering-system)
|
||||
- [Prerequisites](#-prerequisites)
|
||||
- [Installation](#-installation)
|
||||
- [How to Run](#-how-to-run)
|
||||
- [Configuration](#-configuration)
|
||||
- [Monitoring and Metrics](#-monitoring-and-metrics)
|
||||
- [Testing](#-testing)
|
||||
- [Code Quality & Validation](#-code-quality--validation)
|
||||
- [Development](#-development)
|
||||
- [Troubleshooting](#-troubleshooting)
|
||||
- [Performance Tuning](#-performance-tuning)
|
||||
- [Contributing](#-contributing)
|
||||
- [License](#-license)
|
||||
- [Support](#-support)
|
||||
|
||||
## Features
|
||||
|
||||
### Core Functionality
|
||||
@@ -646,6 +676,73 @@ pytest tests/activities/test_mongo_db.py
|
||||
pytest tests/workflows/test_orchestrator.py
|
||||
```
|
||||
|
||||
## 🛡️ Code Quality & Validation
|
||||
|
||||
### Overview
|
||||
|
||||
Since Python is not compiled, this project ships a validation workflow to catch issues early. Use the `validate.sh` script to run formatting, linting, type checks, security analysis, and tests in one command.
|
||||
|
||||
### Validation Tools
|
||||
|
||||
- **Ruff**: formatting and linting (fast, replaces Black/Flake8)
|
||||
- **mypy**: static typing checks
|
||||
- **Bandit**: security static analysis
|
||||
- **pytest**: unit/integration tests with coverage
|
||||
|
||||
### Tools Installation
|
||||
|
||||
```bash
|
||||
pip install -r requirements-dev.txt
|
||||
```
|
||||
|
||||
### Complete Validation (recommended)
|
||||
|
||||
```bash
|
||||
./validate.sh
|
||||
```
|
||||
|
||||
What `validate.sh` does:
|
||||
1. Checks formatting with Ruff
|
||||
2. Lints code with Ruff
|
||||
3. Runs mypy type checking
|
||||
4. Runs Bandit security analysis
|
||||
5. Executes pytest with coverage (generates HTML report)
|
||||
|
||||
Exit codes are propagated so CI can fail fast when quality gates are not met.
|
||||
|
||||
### Individual Commands
|
||||
|
||||
```bash
|
||||
# 1) Format check
|
||||
ruff format --check orchestrator/ tests/
|
||||
|
||||
# 2) Lint
|
||||
ruff check orchestrator/ tests/
|
||||
|
||||
# 3) Type check
|
||||
mypy orchestrator/
|
||||
|
||||
# 4) Security
|
||||
bandit -r orchestrator/ -ll
|
||||
|
||||
# 5) Tests with coverage
|
||||
pytest tests/ --cov=orchestrator --cov-report=html
|
||||
```
|
||||
|
||||
### Automatic Fixes
|
||||
|
||||
```bash
|
||||
# Apply formatting
|
||||
ruff format orchestrator/ tests/
|
||||
|
||||
# Autofix common lint issues
|
||||
ruff check --fix orchestrator/ tests/
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Tooling is configured in `pyproject.toml` (lint rules, formatting, typing). Adjust thresholds and rules there as needed.
|
||||
|
||||
## 🔧 Development
|
||||
|
||||
### Project Structure
|
||||
|
||||
Reference in New Issue
Block a user