Metadata-Version: 2.4
Name: bloomfilter-py
Version: 1.3.0
Summary: Yet another bloomfilter implementation in Python
Author-email: OldPanda <me@old-panda.com>
License: MIT License
        
        Copyright (c) 2020 OldPanda
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Documentation, https://github.com/OldPanda/bloomfilter-py/blob/master/README.md
Project-URL: Homepage, https://github.com/OldPanda/bloomfilter-py
Project-URL: Repository, https://github.com/OldPanda/bloomfilter-py.git
Keywords: bloomfilter
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: bitarray==3.11.0
Requires-Dist: mmh3==5.0.1; python_version < "3.9"
Requires-Dist: mmh3==5.2.0; python_version >= "3.9" and python_version < "3.10"
Requires-Dist: mmh3==5.3.1; python_version >= "3.10"
Dynamic: license-file

# bloomfilter-py
![](https://img.shields.io/pypi/v/bloomfilter-py.svg)
![](https://img.shields.io/pypi/pyversions/bloomfilter-py.svg)
[![codecov](https://codecov.io/gh/OldPanda/bloomfilter-py/branch/master/graph/badge.svg?token=RBX1JK7P7O)](https://codecov.io/gh/OldPanda/bloomfilter-py)
[![Downloads](https://pepy.tech/badge/bloomfilter-py)](https://pepy.tech/project/bloomfilter-py)

## Overview
Yet another Bloomfilter implementation in Python, compatible with Java's Guava library.

Compatibility targets Guava's compact `BloomFilter.writeTo` / `readFrom` format.
Java `ObjectOutputStream` / `ObjectInputStream` serialization is a different
format and is not supported. The compact format does not include the funnel;
callers must provide the matching encoder when loading a filter.

I was looking for a Python library which is capable of reading what Bloomfilter of Java's Guava library serializes and is also able to output byte array which is recognizable by Java. But unfortunately failed. Hence I developed this library by borrowing how Guava implements Bloomfilter serialization/deserialization a lot to deal with Bloomfilters on both Python and Java sides.

As for Bloomfilter usage in Java world, please refer to [this post](https://www.baeldung.com/guava-bloom-filter).

Here's a brief [introduction](https://en.wikipedia.org/wiki/Bloom_filter) to Bloomfilter.

## Requirements
* Python 3.8+

## Install
```
pip install bloomfilter-py
```

## Usage Examples

### Basic Usage
```Python
>>> from bloomfilter import BloomFilter
>>> bloom_filter = BloomFilter(expected_insertions=500, err_rate=0.01)
>>> for i in range(100):
...     bloom_filter.put(i)
...
>>> 1 in bloom_filter
True
>>> 100 in bloom_filter
False
>>>
```

### Guava Funnel Compatibility

Guava's compact BloomFilter format stores the hash strategy and bit array, but
does not store its `Funnel`. To query a filter produced by Guava, select the
same funnel that was used in Java:

```Python
from bloomfilter import BloomFilter, INTEGER_FUNNEL

with open("guava-filter.out", "rb") as f:
    bloom_filter = BloomFilter.loads(f.read(), funnel=INTEGER_FUNNEL)
```

The built-in mappings are:

| Java funnel | bloomfilter-py funnel |
| --- | --- |
| `Funnels.integerFunnel()` | `INTEGER_FUNNEL` |
| `Funnels.longFunnel()` | `LONG_FUNNEL` |
| `Funnels.stringFunnel(StandardCharsets.UTF_8)` | `UTF8_STRING_FUNNEL` |
| `Funnels.stringFunnel(StandardCharsets.UTF_16)` | `StringFunnel("UTF-16")` |
| `Funnels.stringFunnel(StandardCharsets.UTF_16BE)` | `StringFunnel("UTF-16BE")` |
| `Funnels.stringFunnel(StandardCharsets.UTF_16LE)` | `StringFunnel("UTF-16LE")` |
| `Funnels.byteArrayFunnel()` | `BYTE_ARRAY_FUNNEL` |

`StringFunnel` supports Java's six standard charsets: US-ASCII, ISO-8859-1,
UTF-8, UTF-16, UTF-16BE, and UTF-16LE. Custom Guava funnels are not encoded in
the serialized filter; use a matching custom Python `Funnel` implementation
when reading them.

Use the same funnel when creating a filter that Java will read:

```Python
from bloomfilter import BloomFilter, LONG_FUNNEL

bloom_filter = BloomFilter(500, 0.01, funnel=LONG_FUNNEL)
bloom_filter.put(1)  # Encoded as a Java long even though 1 fits in an int.
```

Calls that omit `funnel` retain the historical bloomfilter-py behavior, which
chooses integer width from each value and therefore is not suitable for all
Guava interoperability scenarios.

### Serialize Bloomfilter
You can easily serialize `BloomFilter` instance to a byte array
```Python
>>> dumps = bloom_filter.dumps()
>>> with open("dumps.out", "wb") as f:
...     f.write(dumps)
...
>>>
```

or to a hex string
```Python
>>> hex_str = bloom_filter.dumps_to_hex()
```

or to a base64 encoded bytes
```Python
base64_bytes = bloom_filter.dumps_to_base64()
```

### Deserialize Bloomfilter
And you can easily initialize a `BloomFilter` instance from a byte array
```Python
>>> with open("dumps.out", "rb") as f:
...     bf = BloomFilter.loads(f.read())
...
>>> 1 in bf
True
>>> 100 in bf
False
>>>
```

or from a hex string
```Python
>>> bf = BloomFilter.loads_from_hex(hex_str)
>>> 1 in bf
True
>>> 100 in bf
False
```

or from a base64 encoded bytes
```Python
>>> bf = BloomFilter.loads_from_base64(base64_bytes)
>>> 100 in bf
False
>>> 200 in bf
False
>>> 1 in bf
True
>>> 99 in bf
True
```

Deserialization limits the backing bit array to 2³⁰ bits by default. For a
trusted larger Guava filter, pass an explicit limit to any loading method:

```Python
bf = BloomFilter.loads(dumps, funnel=INTEGER_FUNNEL, max_num_bits=2**31)
```

### Stream I/O

Read and write compact Guava data directly with binary streams:

```Python
with open("guava-filter.out", "rb") as stream:
    bf = BloomFilter.read_from(stream, funnel=INTEGER_FUNNEL)

with open("python-filter.out", "wb") as stream:
    bf.write_to(stream)

byte_count = bf.serialized_size()
```

`read_from` reads exactly one filter, leaving any following stream data unread.
It accepts the same `max_num_bits` limit as `loads`, and validates the header
before reading the payload. `loads` requires exactly one filter with no trailing
bytes. Neither stream method closes the caller's stream. Both handle partial
reads or writes on blocking binary streams.

### Copies, Merging, and Statistics

```Python
snapshot = bf.copy()
other = BloomFilter(500, 0.01, funnel=INTEGER_FUNNEL)
other.put(123)

if bf.is_compatible(other):
    bf.put_all(other)

false_positive_probability = bf.expected_fpp()
estimated_distinct_elements = bf.approximate_element_count()
```

`copy` duplicates the backing bit array and preserves the loading limit. It
shares the funnel and stateless strategy, as Guava does; custom funnels should
be immutable. `put_all` performs an in-place union without modifying the source.
Compatibility requires distinct instances with the same bit-array size, number
of hashes, strategy, and equal funnels. Custom funnels must define equality
when separate instances represent the same encoding.

Statistics use the current bit occupancy, so they also work on loaded and merged
filters. A fully saturated filter has an expected FPP of `1.0`; its element count
cannot be estimated and raises `OverflowError`, corresponding to Guava's
`ArithmeticException`. Filter equality compares configuration, funnel, and bit
contents. Mutable filters are unhashable in Python.

Unlike Guava's atomic implementation, this library does not guarantee safe
concurrent mutations. Use an external lock around mutations, queries, copies,
statistics, and serialization when sharing a filter between threads. Java's
deprecated predicate aliases and stream collectors are language-specific APIs;
use `might_contain` (or `in`) and a loop calling `put` in Python.

## Development and Testing

From the repository root, create an isolated virtual environment and install the
project in editable mode with pytest:

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e . pytest
```

Run the complete test suite:

```bash
python -m pytest -v
```

The bidirectional Java integration test requires a JDK and a Guava JAR. Point
`GUAVA_JAR` at the JAR to compile the included Java helper and verify both
Guava-to-Python and Python-to-Guava serialization:

```bash
GUAVA_JAR=/path/to/guava-33.7.2-jre.jar \
  python -m pytest -v tests/test_guava_interop.py
```

The test is skipped when those prerequisites are unavailable. CI downloads the
pinned Guava release, verifies its SHA-256 checksum, and runs the test as a
required separate job.

To run the same formatting, type, and security checks used during development,
install the additional tools and execute:

```bash
python -m pip install black mypy bandit
python -m black --check bloomfilter tests
python -m mypy bloomfilter --strict
python -m bandit -r bloomfilter
```

On Windows PowerShell, activate the virtual environment with
`.venv\Scripts\Activate.ps1` instead of `source .venv/bin/activate`.
