batched — Grouping items from an iterable

This module groups the items of any iterable into tuples of a fixed size, which is useful when an API accepts at most a certain number of records in one request.

batched(iterable, n)

Yield tuples of n items from iterable. The last tuple may be shorter when the items run out before it fills.

>>> list(batched('ABCDEFG', 3))
[('A', 'B', 'C'), ('D', 'E', 'F'), ('G',)]

Note

Passing n less than one raises ValueError.

Recipes

Batching pairs well with a thread pool when you want to send records to a slow service a few hundred at a time without holding them all in memory.