Configuration¶

LeasedExecutorManager¶

manager = LeasedExecutorManager(
    backend="thread",
    max_pools=4,
    min_pools=1,
    units_per_pool=10,
    size_provider=lambda: len(connected_devices),
    check_interval=120.0,
    default_lease_seconds=300.0,
    lease_grace_seconds=15.0,
    workers_per_pool=4,
    name_prefix="leasepool",
)

backend: "thread", "process", or "interpreter". You may also pass the matching ExecutorBackend enum value.
max_pools: Maximum number of executor objects owned by this manager. Required.
min_pools: Minimum desired number of executor objects. The manager creates this many on startup unless adaptive sizing asks for more.
units_per_pool: Adaptive sizing ratio used with size_provider.
size_provider: Optional callable returning a current unit count. Units can be connected devices, tenants, queues, shards, customers, or any custom runtime signal.
check_interval: How often the background checker wakes when no scale or expiry event occurs. Expiring leases and notify_scale_changed() can wake it earlier.
default_lease_seconds: Default lease duration when callers do not specify lease_seconds in acquire().
lease_grace_seconds: Extra time after soft expiry before hard revocation.
workers_per_pool: Worker count passed to the executor constructor as max_workers.
name_prefix: Prefix for worker thread names on backends that support thread_name_prefix.
logger: Optional logger used by the manager for lifecycle, lease, and checker logs.
process_logging: Optional ProcessLoggingConfig for process-worker log forwarding.
forward_process_logs: Convenience flag for enabling process-worker log forwarding without creating a ProcessLoggingConfig manually.
process_log_level: Child-process root logger level when process log forwarding is enabled.
process_log_target_logger: Parent-process logger that receives forwarded worker log records.
clear_process_log_handlers: Whether to remove existing child-process root handlers before installing the queue handler. Defaults to True to avoid duplicate logs after fork.
**executor_kwargs: Extra keyword arguments passed to the underlying executor constructor. For the process backend this can include options such as initializer, initargs, mp_context, or max_tasks_per_child.

Process logging configuration¶

Use the explicit configuration object when you want documented, reusable configuration:

import logging

from leasepool import LeasedExecutorManager, ProcessLoggingConfig


manager = LeasedExecutorManager(
    backend="process",
    max_pools=1,
    min_pools=1,
    workers_per_pool=2,
    process_logging=ProcessLoggingConfig(
        enabled=True,
        level="INFO",
        target_logger=logging.getLogger("myapp.process-workers"),
    ),
)

Use the convenience arguments for a shorter setup:

manager = LeasedExecutorManager(
    backend="process",
    max_pools=1,
    min_pools=1,
    workers_per_pool=2,
    forward_process_logs=True,
    process_log_level="INFO",
)

Do not pass process_logging and the forward_process_logs / process_log_* convenience arguments together.

Acquire options¶

lease = await manager.acquire(
    lease_seconds=30.0,
    owner="request:123",
    wait=True,
    timeout=2.0,
)

lease_seconds: Overrides default_lease_seconds for this lease.
owner: Optional diagnostic label.
wait: If True, wait for capacity when every pool is leased. If False, raise LeaseUnavailableError immediately.
timeout: Maximum time to wait for capacity. Raises built-in TimeoutError if the timeout is reached.

WorkGrinder¶

grinder = WorkGrinder(
    executor_manager=manager,
    max_wait_seconds=10.0,
    batch_size_threshold=20,
    lease_seconds=60.0,
    owner_prefix="work-grinder",
)

executor_manager: Started LeasedExecutorManager used for leases.
max_wait_seconds: Maximum age of the oldest pending item before a batch is processed.
batch_size_threshold: Pending item count that triggers immediate batch processing.
lease_seconds: Lease duration requested for each batch.
owner_prefix: Prefix used to label batch leases in manager diagnostics.

max_pools is per manager¶

max_pools is not global across all backend types.

thread_manager = LeasedExecutorManager(
    backend="thread",
    max_pools=5,
    workers_per_pool=4,
)

process_manager = LeasedExecutorManager(
    backend="process",
    max_pools=1,
    workers_per_pool=4,
)

This allows up to five thread executors and one process executor. Each manager enforces its own limit.