Refactor OpcManager connection error handling and enhance unit tests
- Simplified error handling during connection attempts in OpcManager by removing redundant disconnection logic.
- Updated unit tests to include subscription_period_ms in server configuration and adjusted connection assertions to include a timeout parameter.
- Added new tests for disconnection fallback functionality to ensure robust error handling during disconnect attempts.
Update OpcManager to set a timeout for the client connection
- Added a timeout parameter to the Client initialization in OpcManager to enhance connection reliability and prevent indefinite blocking during connection attempts.
Fix pod URI formatting and enhance disconnection error logging in OpcManager
- Updated pod URI construction to replace hyphens with colons for correct formatting.
- Improved error logging during disconnection attempts to include the attempt number for better traceability.
- Changed disconnection_fallback method call to be awaited for proper asynchronous handling.
SIENTIAPDE-1318 Enhance OpcManager disconnection handling
- Introduced a disconnection_fallback method to attempt multiple disconnects from the OPC UA server, improving reliability.
- Updated error handling during connection attempts to ensure proper logging and notification of disconnection failures.
- Refactored disconnect method to utilize the new fallback mechanism and maintain metrics accurately.
Enhance OpcManager to accept subscription period from configuration
- Updated OpcManager to include subscription_period_ms as a parameter for better configurability.
- Refactored create_subscription method to utilize the new subscription period parameter.
- Adjusted unit tests to reflect changes in subscription handling and ensure correct functionality.
Refactor app.py for consistency in comment formatting
- Updated the main asynchronous function and loop condition comments for improved readability and consistency.
Refactor app.py and enhance unit tests for better clarity and structure
- Updated `app.py` to improve async compatibility with a clearer comment.
- Added blank lines in `test_ingestor.py` for improved readability and organization of test cases.
Refactor async handling and enhance unit tests for ingestor manager
- Updated `app.py` to improve async compatibility by adding a comment for clarity.
- Added new unit tests to handle scenarios where the ingestor manager is None, ensuring robustness in `manage_leases` and `loop` methods.
- Enhanced existing tests to cover edge cases related to the ingestor manager's state.
Enhance GitHub Actions workflow for quality gate
- Added steps to generate a GitHub App token for authentication.
- Prepared `requirements.txt` to use HTTPS for GitHub dependencies.
- Configured Git to utilize the generated App token for repository access.
- Updated caching strategy to use the processed requirements file for improved dependency management.
Refactor project structure and update configurations
- Deleted the empty `__init__.py` file to clean up the project structure.
- Renamed the project from "laborious" to "ingestor" in `pyproject.toml`, updating the description accordingly.
- Improved type hints and conditional checks in the `Ingestor`, `DataManager`, `IngestorManager`, and `OpcManager` classes for better code clarity and type safety.
- Enhanced error handling and assertions in various methods to ensure robustness.
- Updated unit tests to reflect changes in class names and error handling improvements.
- Updated the `sientia-dataops-library` dependency version from 1.4.3 to 1.4.6 in `requirements.txt`.
- Modified the GitHub Actions workflow to install development and runtime dependencies separately, improving clarity and organization.
- Added code formatting and linting checks using Ruff, along with type checking using mypy, to ensure code quality.
- Updated `.gitignore` to include additional cache directories and log files.
- Refactored code in various files for consistency in string formatting and improved logging messages.
Enhance documentation and improve shutdown methods for DataManager and IngestorManager
- Updated README.md to clarify persistent storage capabilities.
- Enhanced docstrings in DataManager and IngestorManager for shutdown processes, detailing resource cleanup and error handling.
- Improved comments in various methods to provide clearer explanations of functionality and side effects.
Update README.md to clarify data streaming and storage features
- Revised descriptions for Real-time Data Streaming and Persistent Storage to emphasize historical data streaming capabilities.
- Updated Data Manager section to reflect the integration of both MongoDB and Kafka for data management.
- Enhanced configuration example in the prerequisites section with additional fields for server configuration.
Remove deprecated files and enhance documentation
- Deleted `coverage.sh`, `docker-compose.yaml`, `Dockerfile`, and simulator-related files to streamline the project structure.
- Updated `README.md` to provide a comprehensive overview of the OPC Ingestor, including features, architecture, installation, usage, and troubleshooting.
- Enhanced docstrings across various classes and methods in the `ingestor` module for better clarity and maintainability.
- Improved Prometheus metrics documentation in `metrics.py` to ensure proper monitoring and observability of the system.
Enhance unit tests for async shutdown and improve assertions
- Updated `test_app.py` to verify multiple calls to `asyncio_sleep` and ensure proper handling of sleep intervals.
- Added new tests in `test_ingestor.py` and `test_opc_manager.py` to validate shutdown behavior and error handling for async operations.
- Improved assertions in existing tests to enhance reliability and clarity of test outcomes.
Refactor tests for async compatibility and enhance IngestorManager functionality
- Updated test cases in `test_app.py`, `test_ingestor.py`, and `test_ingestor_manager.py` to use async/await syntax for improved concurrency.
- Refactored methods in IngestorManager and related classes to support asynchronous operations, ensuring non-blocking behavior during execution.
- Enhanced mock setups in tests to accommodate async methods, improving test reliability and performance.
Update values.yaml and refactor shutdown methods for improved configuration and async handling
- Changed replica count from 2 to 1 in values.yaml for deployment optimization.
- Updated image tag from 0.4.3 to 0.4.5 in values.yaml for version consistency.
- Refactored shutdown method in IngestorManager to use await server.shutdown() for better async operation.
- Removed __del__ method in OpcManager to prevent blocking during cleanup.
Refactor Ingestor and OPC Manager for Asynchronous Operations
- Updated main function to be asynchronous and integrated asyncio for better concurrency.
- Refactored Ingestor methods to support async operations, including prepare_ingestor, loop, and shutdown.
- Enhanced IngestorManager and OpcManager with async methods for improved performance and responsiveness.
- Replaced blocking calls with await statements to ensure non-blocking behavior during operations.
- Added a new run_async_main function to handle the async event loop setup.
SIENTIAPDE-1193 Update values.yaml to include new environment variables for enhanced configuration
- Added new environment variables for improved deployment flexibility.
- Updated existing configurations to align with recent changes in the application structure.
Update requirements and refactor timestamp handling in DataManager
- Updated sientia-dataops-library version to 1.4.3 in requirements.txt.
- Refactored timestamp insertion in DataManager to use the now() function for improved consistency.
Update requirements and refactor datetime handling in OpcManager for timezone support
- Commented out the specific version of sientia-dataops-library in requirements.txt and added a local path.
- Updated GITHUB_BRANCH in values.yaml for consistency with current development.
- Refactored source_timestamp handling in OpcManager to include timezone information.
- Adjusted timestamp formatting in test cases to reflect the new datetime format with timezone.
Refactor logging levels in Ingestor and IngestorManager for improved clarity
- Changed debug log statements to info level in Ingestor class for better visibility of subscription and management actions.
- Updated log level for initializing OPC manager in IngestorManager to info for consistent logging practices.
Update Ingestor project name and metadata keys to lowercase for consistency
- Changed project_name, workflow_name, and schema_name from uppercase to lowercase in Ingestor class and corresponding unit tests.
- Enabled metrics service with updated configuration for Prometheus ServiceMonitor.
- Removed disabled API and OPC service configurations to streamline values.yaml.
- Updated helm upgrade command version to 0.5.0.
- Updated sientia-dataops-library version to 1.4.1 in requirements.txt.
- Changed image tag from 0.3.2 to 0.4.2 in values.yaml.
- Refactored logger imports from sientia_do.temporal.utils.logger to sientia_do.observability.logger in multiple manager files for improved observability.
Refactor test assertions in test_opc_manager.py for improved readability
- Updated the assertion syntax for notification calls in the test_check_opc_listenning_error_notification_and_retry function to use dictionary literals for clarity.
