Enhance release workflow to trigger only on merged pull requests and streamline data insertion in DataManager
- Updated the release workflow to execute only when a pull request is merged.
- Simplified the document insertion logic in DataManager by consolidating the dictionary construction.
- Refactored OPC Manager tests to utilize asynchronous mocks for improved accuracy in testing.
Refactor OpcManager to emit connection metrics immediately after connection attempt
- Moved the emission of OPC_CONNECTIONS_TOTAL metric to occur right after the connection attempt, ensuring accurate tracking of connection status.
- Removed redundant metric emission logic that was previously placed after a successful connection.
Refactor OpcManager to emit connection metrics after successful connection establishment
- Moved the metric emission for OPC_CONNECTIONS_TOTAL to occur after a successful connection.
- Removed redundant metric increment logic to streamline the connection process.
Update requirements.txt to use version 1.5.3 of sientia-dataops-library and enhance DataManager and ResourceManager with improved logging and timestamping for data insertion and retrieval.
Enhance Ingestor Class with Asynchronous Metrics and Monitoring Integration
- Refactored Ingestor class to inherit from SientiaMonitoring for improved observability.
- Updated methods to utilize asynchronous operations for declaring active ingestors and managing slots.
- Integrated pod_id label into various metrics for better tracking.
- Adjusted unit tests to reflect changes in the Ingestor initialization and asynchronous behavior.
Refactor Ingestor and Metrics Classes for Enhanced Asynchronous Operations
- Updated IngestorManager, OpcManager, and ResourceManager to utilize asynchronous methods for improved performance.
- Integrated MetricsController into various classes for better observability and monitoring.
- Adjusted unit tests to accommodate the new asynchronous behavior, ensuring proper mocking of async methods.
- Removed deprecated Redis metrics and streamlined resource management logic.
SIENTIAPDE-1325: Refactor Ingestor and Manager Classes for Enhanced Asynchronous Operations
- Introduced asynchronous methods across Ingestor, IngestorManager, DataManager, and OpcManager to improve performance and responsiveness.
- Integrated MetricsController into various classes for better observability and monitoring.
- Updated Redis and MongoDB interactions to support asynchronous operations, enhancing data handling efficiency.
- Removed deprecated Redis metrics and streamlined resource management logic.
- Adjusted unit tests to accommodate the new asynchronous behavior and ensure proper mocking of async methods.
Refactor IngestorManager to streamline server disconnection process
- Simplified the logic in the `update_opc_servers` method by directly appending servers to the disconnection list and utilizing the new `remove_server` method for handling server shutdowns.
- Removed redundant code related to managing the `managed_tags` during server disconnection, enhancing code clarity and maintainability.
Refactor IngestorManager and OpcManager for improved server management
- Introduced a new method `remove_server` in IngestorManager to encapsulate server removal logic, enhancing code clarity and reusability.
- Updated `update_opc_servers` to utilize the new `remove_server` method for better maintainability.
- Refactored security settings in OpcManager to use direct attribute assignments instead of method calls, streamlining the code.
- Adjusted unit tests to reflect changes in the IngestorManager and OpcManager, ensuring proper asynchronous behavior and mocking.
Updated the IngestorManager class to ensure that OPC managers are properly shut down and removed from the management list when servers are lost or removed. This change enhances the integrity checks for OPC server management by ensuring asynchronous handling of shutdown operations.
Updated the Ingestor and IngestorManager classes to use async methods for checking OPC server integrity and shutting down OPC managers. Adjusted related tests to ensure proper mocking of asynchronous behavior.
Update image tag in values.yaml and refactor OpcManager error handling
- Updated the image tag in values.yaml from "0.4.5" to "0.4.9" for the latest version.
- Refactored error handling in OpcManager to improve logging and maintain code clarity.
- Adjusted unit tests for better readability and consistency in assertions.
Refactor OpcManager connection error handling and enhance unit tests
- Simplified error handling during connection attempts in OpcManager by removing redundant disconnection logic.
- Updated unit tests to include subscription_period_ms in server configuration and adjusted connection assertions to include a timeout parameter.
- Added new tests for disconnection fallback functionality to ensure robust error handling during disconnect attempts.
Update OpcManager to set a timeout for the client connection
- Added a timeout parameter to the Client initialization in OpcManager to enhance connection reliability and prevent indefinite blocking during connection attempts.
Fix pod URI formatting and enhance disconnection error logging in OpcManager
- Updated pod URI construction to replace hyphens with colons for correct formatting.
- Improved error logging during disconnection attempts to include the attempt number for better traceability.
- Changed disconnection_fallback method call to be awaited for proper asynchronous handling.
SIENTIAPDE-1318 Enhance OpcManager disconnection handling
- Introduced a disconnection_fallback method to attempt multiple disconnects from the OPC UA server, improving reliability.
- Updated error handling during connection attempts to ensure proper logging and notification of disconnection failures.
- Refactored disconnect method to utilize the new fallback mechanism and maintain metrics accurately.
Enhance OpcManager to accept subscription period from configuration
- Updated OpcManager to include subscription_period_ms as a parameter for better configurability.
- Refactored create_subscription method to utilize the new subscription period parameter.
- Adjusted unit tests to reflect changes in subscription handling and ensure correct functionality.
Refactor app.py for consistency in comment formatting
- Updated the main asynchronous function and loop condition comments for improved readability and consistency.
Refactor app.py and enhance unit tests for better clarity and structure
- Updated `app.py` to improve async compatibility with a clearer comment.
- Added blank lines in `test_ingestor.py` for improved readability and organization of test cases.
Refactor async handling and enhance unit tests for ingestor manager
- Updated `app.py` to improve async compatibility by adding a comment for clarity.
- Added new unit tests to handle scenarios where the ingestor manager is None, ensuring robustness in `manage_leases` and `loop` methods.
- Enhanced existing tests to cover edge cases related to the ingestor manager's state.
Refactor project structure and update configurations
- Deleted the empty `__init__.py` file to clean up the project structure.
- Renamed the project from "laborious" to "ingestor" in `pyproject.toml`, updating the description accordingly.
- Improved type hints and conditional checks in the `Ingestor`, `DataManager`, `IngestorManager`, and `OpcManager` classes for better code clarity and type safety.
- Enhanced error handling and assertions in various methods to ensure robustness.
- Updated unit tests to reflect changes in class names and error handling improvements.
- Updated the `sientia-dataops-library` dependency version from 1.4.3 to 1.4.6 in `requirements.txt`.
- Modified the GitHub Actions workflow to install development and runtime dependencies separately, improving clarity and organization.
- Added code formatting and linting checks using Ruff, along with type checking using mypy, to ensure code quality.
- Updated `.gitignore` to include additional cache directories and log files.
- Refactored code in various files for consistency in string formatting and improved logging messages.
Enhance documentation and improve shutdown methods for DataManager and IngestorManager
- Updated README.md to clarify persistent storage capabilities.
- Enhanced docstrings in DataManager and IngestorManager for shutdown processes, detailing resource cleanup and error handling.
- Improved comments in various methods to provide clearer explanations of functionality and side effects.
Remove deprecated files and enhance documentation
- Deleted `coverage.sh`, `docker-compose.yaml`, `Dockerfile`, and simulator-related files to streamline the project structure.
- Updated `README.md` to provide a comprehensive overview of the OPC Ingestor, including features, architecture, installation, usage, and troubleshooting.
- Enhanced docstrings across various classes and methods in the `ingestor` module for better clarity and maintainability.
- Improved Prometheus metrics documentation in `metrics.py` to ensure proper monitoring and observability of the system.
Refactor tests for async compatibility and enhance IngestorManager functionality
- Updated test cases in `test_app.py`, `test_ingestor.py`, and `test_ingestor_manager.py` to use async/await syntax for improved concurrency.
- Refactored methods in IngestorManager and related classes to support asynchronous operations, ensuring non-blocking behavior during execution.
- Enhanced mock setups in tests to accommodate async methods, improving test reliability and performance.
