Updated the IngestorManager class to ensure that OPC managers are properly shut down and removed from the management list when servers are lost or removed. This change enhances the integrity checks for OPC server management by ensuring asynchronous handling of shutdown operations.
Updated the Ingestor and IngestorManager classes to use async methods for checking OPC server integrity and shutting down OPC managers. Adjusted related tests to ensure proper mocking of asynchronous behavior.
Update requirements and simplify quality gate workflow
- Updated the sientia-dataops-library dependency version from 1.4.6 to 1.4.7 in requirements.txt.
- Replaced the existing quality gate workflow with a simplified version that utilizes a template for better maintainability.
SIENTIAPDE-1312: Enhance quality gate workflow to determine next version based on branch type
- Added steps to fetch the branch name and calculate the next version based on the latest tag.
- Implemented versioning logic to increment major, minor, or patch numbers depending on the branch type (release, feature, fix).
- Updated SonarQube analysis step to use the dynamically calculated version.
Update image tag in values.yaml and refactor OpcManager error handling
- Updated the image tag in values.yaml from "0.4.5" to "0.4.9" for the latest version.
- Refactored error handling in OpcManager to improve logging and maintain code clarity.
- Adjusted unit tests for better readability and consistency in assertions.
Refactor OpcManager connection error handling and enhance unit tests
- Simplified error handling during connection attempts in OpcManager by removing redundant disconnection logic.
- Updated unit tests to include subscription_period_ms in server configuration and adjusted connection assertions to include a timeout parameter.
- Added new tests for disconnection fallback functionality to ensure robust error handling during disconnect attempts.
Update OpcManager to set a timeout for the client connection
- Added a timeout parameter to the Client initialization in OpcManager to enhance connection reliability and prevent indefinite blocking during connection attempts.
Fix pod URI formatting and enhance disconnection error logging in OpcManager
- Updated pod URI construction to replace hyphens with colons for correct formatting.
- Improved error logging during disconnection attempts to include the attempt number for better traceability.
- Changed disconnection_fallback method call to be awaited for proper asynchronous handling.
SIENTIAPDE-1318 Enhance OpcManager disconnection handling
- Introduced a disconnection_fallback method to attempt multiple disconnects from the OPC UA server, improving reliability.
- Updated error handling during connection attempts to ensure proper logging and notification of disconnection failures.
- Refactored disconnect method to utilize the new fallback mechanism and maintain metrics accurately.
Enhance OpcManager to accept subscription period from configuration
- Updated OpcManager to include subscription_period_ms as a parameter for better configurability.
- Refactored create_subscription method to utilize the new subscription period parameter.
- Adjusted unit tests to reflect changes in subscription handling and ensure correct functionality.
Refactor app.py for consistency in comment formatting
- Updated the main asynchronous function and loop condition comments for improved readability and consistency.
Refactor app.py and enhance unit tests for better clarity and structure
- Updated `app.py` to improve async compatibility with a clearer comment.
- Added blank lines in `test_ingestor.py` for improved readability and organization of test cases.
Refactor async handling and enhance unit tests for ingestor manager
- Updated `app.py` to improve async compatibility by adding a comment for clarity.
- Added new unit tests to handle scenarios where the ingestor manager is None, ensuring robustness in `manage_leases` and `loop` methods.
- Enhanced existing tests to cover edge cases related to the ingestor manager's state.
Enhance GitHub Actions workflow for quality gate
- Added steps to generate a GitHub App token for authentication.
- Prepared `requirements.txt` to use HTTPS for GitHub dependencies.
- Configured Git to utilize the generated App token for repository access.
- Updated caching strategy to use the processed requirements file for improved dependency management.
Refactor project structure and update configurations
- Deleted the empty `__init__.py` file to clean up the project structure.
- Renamed the project from "laborious" to "ingestor" in `pyproject.toml`, updating the description accordingly.
- Improved type hints and conditional checks in the `Ingestor`, `DataManager`, `IngestorManager`, and `OpcManager` classes for better code clarity and type safety.
- Enhanced error handling and assertions in various methods to ensure robustness.
- Updated unit tests to reflect changes in class names and error handling improvements.
- Updated the `sientia-dataops-library` dependency version from 1.4.3 to 1.4.6 in `requirements.txt`.
- Modified the GitHub Actions workflow to install development and runtime dependencies separately, improving clarity and organization.
- Added code formatting and linting checks using Ruff, along with type checking using mypy, to ensure code quality.
- Updated `.gitignore` to include additional cache directories and log files.
- Refactored code in various files for consistency in string formatting and improved logging messages.
Enhance documentation and improve shutdown methods for DataManager and IngestorManager
- Updated README.md to clarify persistent storage capabilities.
- Enhanced docstrings in DataManager and IngestorManager for shutdown processes, detailing resource cleanup and error handling.
- Improved comments in various methods to provide clearer explanations of functionality and side effects.
Update README.md to clarify data streaming and storage features
- Revised descriptions for Real-time Data Streaming and Persistent Storage to emphasize historical data streaming capabilities.
- Updated Data Manager section to reflect the integration of both MongoDB and Kafka for data management.
- Enhanced configuration example in the prerequisites section with additional fields for server configuration.
Remove deprecated files and enhance documentation
- Deleted `coverage.sh`, `docker-compose.yaml`, `Dockerfile`, and simulator-related files to streamline the project structure.
- Updated `README.md` to provide a comprehensive overview of the OPC Ingestor, including features, architecture, installation, usage, and troubleshooting.
- Enhanced docstrings across various classes and methods in the `ingestor` module for better clarity and maintainability.
- Improved Prometheus metrics documentation in `metrics.py` to ensure proper monitoring and observability of the system.
Enhance unit tests for async shutdown and improve assertions
- Updated `test_app.py` to verify multiple calls to `asyncio_sleep` and ensure proper handling of sleep intervals.
- Added new tests in `test_ingestor.py` and `test_opc_manager.py` to validate shutdown behavior and error handling for async operations.
- Improved assertions in existing tests to enhance reliability and clarity of test outcomes.
Refactor tests for async compatibility and enhance IngestorManager functionality
- Updated test cases in `test_app.py`, `test_ingestor.py`, and `test_ingestor_manager.py` to use async/await syntax for improved concurrency.
- Refactored methods in IngestorManager and related classes to support asynchronous operations, ensuring non-blocking behavior during execution.
- Enhanced mock setups in tests to accommodate async methods, improving test reliability and performance.
