# Worker Parameters Documentation This document explains each configuration parameter used in the `prepare_worker.py` file for configuring Temporal workers. ## Overview All parameters can be configured via environment variables using the pattern: `{WORKFLOW_NAME}_{PARAMETER_NAME}`. If not set, default values are used as specified below. ## Concurrency Parameters ### MAX_CONCURRENT_WORKFLOW_TASKS - **Default**: `200` - **Description**: Maximum number of concurrent workflow tasks that can be processed simultaneously by the worker. This controls how many workflow executions can be actively running at the same time. - **Usage**: Set via `max_concurrent_workflow_tasks` in the Worker configuration. - **Impact**: Higher values allow more workflows to run concurrently but consume more resources. Lower values provide better resource control but may limit throughput. ### MAX_CONCURRENT_ACTIVITIES - **Default**: `200` - **Description**: Maximum number of concurrent activity tasks that can be executed simultaneously by the worker. Activities are the actual work units that perform business logic. - **Usage**: Set via `max_concurrent_activities` in the Worker configuration. - **Impact**: Controls the parallelism of activity execution. Higher values increase throughput but require more system resources (CPU, memory, network connections). ### MAX_CONCURRENT_LOCAL_ACTIVITIES - **Default**: `200` - **Description**: Maximum number of concurrent local activity tasks that can be executed simultaneously. Local activities run in the same process as the workflow, without requiring a separate activity worker. - **Usage**: Set via `max_concurrent_local_activities` in the Worker configuration. - **Impact**: Similar to regular activities, but local activities have lower latency and overhead since they don't require network round-trips. Useful for lightweight operations. ## Caching Parameters ### MAX_CACHED_WORKFLOWS - **Default**: `200` - **Description**: Maximum number of workflow instances that can be cached in memory by the worker. Cached workflows allow faster resumption of execution without reloading state. - **Usage**: Set via `max_cached_workflows` in the Worker configuration. - **Impact**: Higher values improve performance for frequently accessed workflows but consume more memory. Lower values reduce memory usage but may require more frequent state reloads. ## Understanding Pollers in Temporal **Pollers** are components of Temporal Workers that continuously request tasks from the Temporal service's Task Queues via synchronous RPCs. There are separate pollers for workflow tasks and activity tasks. ### How Pollers Work Pollers send requests to the Temporal service to retrieve tasks from Task Queues. When a task is available, the poller retrieves it and the Worker processes it using registered Workflow or Activity handlers. This architecture provides: - **Load Balancing**: Workers only poll when they have capacity, distributing load across multiple processes - **Fault Tolerance**: Tasks persist in queues if a Worker fails, allowing recovery - **Task Routing**: Tasks can be routed to specific Worker processes ### Autoscaling Poller Behavior Temporal supports autoscaling that dynamically adjusts the number of concurrent pollers based on workload. The system scales up during high load and down during low load, maintaining a baseline for responsiveness. Autoscaling is configured with `minimum`, `initial`, and `maximum` parameters that define the scaling bounds. ## Workflow Poller Behavior (Autoscaling) These parameters control the autoscaling behavior of the workflow task poller, which retrieves workflow tasks from the Temporal server. ### WORKFLOW_POLLER_BEHAVIOUR_MINIMUM - **Default**: `10` - **Description**: Minimum number of concurrent pollers for workflow tasks. The poller count will never go below this value. - **Usage**: Set via `minimum` in `PollerBehaviorAutoscaling` for `workflow_task_poller_behavior`. - **Impact**: Ensures a baseline level of polling activity even during low load periods. ### WORKFLOW_POLLER_BEHAVIOUR_INITIAL - **Default**: `100` - **Description**: Initial number of concurrent pollers for workflow tasks when the worker starts. - **Usage**: Set via `initial` in `PollerBehaviorAutoscaling` for `workflow_task_poller_behavior`. - **Impact**: Determines the starting point for poller scaling. Higher values provide faster initial task acquisition but consume more resources. ### WORKFLOW_POLLER_BEHAVIOUR_MAXIMUM - **Default**: `200` - **Description**: Maximum number of concurrent pollers allowed for workflow tasks. The poller count will not exceed this value even under high load. - **Usage**: Set via `maximum` in `PollerBehaviorAutoscaling` for `workflow_task_poller_behavior`. - **Impact**: Caps the resource consumption for workflow task polling. Prevents excessive polling that could overwhelm the Temporal server or worker. ## Activity Poller Behavior (Autoscaling) These parameters control the autoscaling behavior of the activity task poller, which retrieves activity tasks from the Temporal server. ### ACTIVITY_POLLER_BEHAVIOUR_MINIMUM - **Default**: `10` - **Description**: Minimum number of concurrent pollers for activity tasks. The poller count will never go below this value. - **Usage**: Set via `minimum` in `PollerBehaviorAutoscaling` for `activity_task_poller_behavior`. - **Impact**: Ensures a baseline level of polling activity even during low load periods. ### ACTIVITY_POLLER_BEHAVIOUR_INITIAL - **Default**: `100` - **Description**: Initial number of concurrent pollers for activity tasks when the worker starts. - **Usage**: Set via `initial` in `PollerBehaviorAutoscaling` for `activity_task_poller_behavior`. - **Impact**: Determines the starting point for poller scaling. Higher values provide faster initial task acquisition but consume more resources. ### ACTIVITY_POLLER_BEHAVIOUR_MAXIMUM - **Default**: `200` - **Description**: Maximum number of concurrent pollers allowed for activity tasks. The poller count will not exceed this value even under high load. - **Usage**: Set via `maximum` in `PollerBehaviorAutoscaling` for `activity_task_poller_behavior`. - **Impact**: Caps the resource consumption for activity task polling. Prevents excessive polling that could overwhelm the Temporal server or worker. ## Configuration Example To override these parameters, set environment variables using the pattern: ``` {WORKFLOW_NAME}_{PARAMETER_NAME}={value} ``` For example, if your workflow is named `ScouterWorkflow`: ```bash SCOUTERWORKFLOW_MAX_CONCURRENT_ACTIVITIES=500 SCOUTERWORKFLOW_WORKFLOW_POLLER_BEHAVIOUR_MAXIMUM=300 ``` ## Notes - All parameter values are converted to integers before use. - The autoscaling poller behavior dynamically adjusts the number of pollers between the minimum and maximum values based on workload. - These parameters should be tuned based on your specific workload characteristics, available resources, and performance requirements.