Metadata-Version: 2.5
Name: truhe
Version: 1.0.0
Summary: Encrypted blob stores for S3, the filesystem and memory, synchronous and asynchronous.
Project-URL: Homepage, http://github.com/conestack/truhe
Author-email: Conestack Contributors <dev@conestack.org>
License: MIT
License-File: LICENSE.rst
Keywords: aes-gcm,blob,encryption,s3,sigv4,storage
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: AsyncIO
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Security :: Cryptography
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: cryptography
Provides-Extra: httpx
Requires-Dist: httpx<1; extra == 'httpx'
Provides-Extra: test
Requires-Dist: httpx<1; extra == 'test'
Requires-Dist: pytest; extra == 'test'
Requires-Dist: urllib3>=2; extra == 'test'
Provides-Extra: urllib3
Requires-Dist: urllib3>=2; extra == 'urllib3'
Description-Content-Type: text/x-rst

.. image:: https://img.shields.io/pypi/v/truhe.svg
    :target: https://pypi.python.org/pypi/truhe
    :alt: Latest PyPI version

.. image:: https://img.shields.io/pypi/dm/truhe.svg
    :target: https://pypi.python.org/pypi/truhe
    :alt: Number of PyPI downloads

.. image:: https://github.com/conestack/truhe/actions/workflows/test.yml/badge.svg
    :target: https://github.com/conestack/truhe/actions/workflows/test.yml
    :alt: Test truhe


truhe
=====

Encrypted blob stores. An application puts its bytes into a store and gets
them back; the store, S3, a directory or memory, only ever sees ciphertext.

- AES-256-GCM in 64 KiB blocks, so objects of any size stream in and out
  without being held in memory.
- Stores for S3 compatible object storage, the filesystem and memory.
- Synchronous over ``urllib3``, asynchronous over ``httpx``.
- AWS Signature Version 4 of its own, no SDK.
- Keys rotate without rewriting the store.

.. contents::
    :depth: 2


Installation
------------

The S3 store needs an HTTP client, chosen with an extra:

.. code-block:: shell

    pip install truhe[urllib3]   # synchronous
    pip install truhe[httpx]     # asynchronous

Without extra, the filesystem and the memory store are available.


Usage
-----

A store wrapped in ``EncryptedBlobStore`` encrypts on the way in and decrypts
on the way out:

.. code-block:: python

    import os

    from truhe import EncryptedBlobStore
    from truhe import Keyring
    from truhe.s3 import S3BlobStore
    from truhe.sigv4 import Credentials

    credentials = Credentials(
        access_key='...',
        secret_key='...',
        region='fsn1',
    )
    store = EncryptedBlobStore(
        S3BlobStore(credentials, 'https://fsn1.your-objectstorage.com', 'bucket'),
        Keyring.from_setting(keys),
        magic=b'MYAPP',
    )

    # bytes
    store.put('reports/2026/q3.pdf', data, 'application/pdf')

    # a stream, with its length up front
    with open('intro.mp4', 'rb') as f:
        store.put(
            'videos/intro.mp4',
            iter(lambda: f.read(65536), b''),
            'video/mp4',
            content_length=os.fstat(f.fileno()).st_size,
        )

    with store.open('videos/intro.mp4') as stream:
        for chunk in stream.chunks:
            ...

    store.head('videos/intro.mp4')    # StoredBlob(size=..., content_type=...)
    list(store.list('videos/'))       # ListedBlob(key, size, modified)
    store.delete('videos/intro.mp4')

The asynchronous stores have the same operations as coroutines, ``open`` is an
async context manager, ``list`` an async iterator, a stream to ``put`` an
async iterable:

.. code-block:: python

    import httpx

    from truhe.aio import EncryptedBlobStore
    from truhe.aio.s3 import S3BlobStore

    async with httpx.AsyncClient() as client:
        store = EncryptedBlobStore(
            S3BlobStore(credentials, endpoint, 'bucket', client),
            keyring,
        )
        await store.put('k', data, 'application/octet-stream')
        async with store.open('k') as stream:
            async for chunk in stream.chunks:
                ...

Sizes are always plaintext sizes, the framing stays inside
``EncryptedBlobStore``.


Streams
-------

``put`` takes bytes, or an iterable of bytes together with ``content_length``.
A stream which delivers more or fewer bytes than announced fails, and nothing
is stored: the piece which would complete the object is only passed on once
the stream is known to end there.

Bytes go to S3 with their SHA-256 in the signature and ``Content-MD5``, so the
store verifies what arrived. A stream goes out with ``UNSIGNED-PAYLOAD`` and
its ``Content-Length``; hashing it first would mean holding it. One request
per object, so an object is at most 5 GiB, the limit of a single ``PUT``.


Keys
----

A ``Keyring`` holds 32 byte keys, newest first. New objects are sealed with the
first, every object names its key in its header. To rotate, put a new key in
front, old objects stay readable. Dropping a key makes its objects
unreadable.

.. code-block:: shell

    openssl rand -hex 32

``Keyring.from_setting`` reads a comma separated list of hex keys, the form of
an environment variable. ``Keyring.primary_id`` is the id of the current key,
worth recording next to every object, so the progress of a rotation is a
query.

**The key is the data.** Lose it, and every object is gone while the store is
perfectly healthy.


S3 compatible stores
--------------------

``S3BlobStore`` addresses the bucket virtual hosted,
``{bucket}.{endpoint host}``, the form most providers document. Pass
``path_style=True`` for ``{endpoint}/{bucket}``, e.g. MinIO or Garage without
a domain. The region is the location code of the provider.

The stores do not retry. A retried upload after a lost answer is the
application's decision.


Format
------

An object is a header and its sealed blocks:

=============== ======= =====================================================
Field           Bytes   Content
=============== ======= =====================================================
magic           5       names the application, ``TRUHE`` by default
version         1       ``1``
key id          8       truncated SHA-256 of the key
nonce prefix    8       random per object
blocks          n       64 KiB plaintext each, the last one shorter, sealed
                        with AES-256-GCM, 16 byte tag
=============== ======= =====================================================

The nonce of a block is the prefix and the block counter, the associated data
is the header and a flag for the last block. So a block cannot be reordered
or moved to another object, a truncated object does not verify, and a header
cannot be swapped. An empty object has one sealed empty block.

``truhe.format`` has the format without I/O: ``Sealer`` and ``Opener`` take
bytes as they arrive and return complete blocks, ``ciphertext_length`` gives
the stored size before the first byte.

Not supported: HTTP ``Range``, multipart uploads, presigned URLs.


Tests
-----

.. code-block:: shell

    make install
    make test

The S3 stores are tested offline. The tests against a store are opt-in, ``make
test`` skips them. To run them against a throwaway MinIO:

.. code-block:: shell

    make test-s3

This starts the container from ``testing/compose.minio.yml``, runs the S3 tests
and discards the container, also when a test fails. To keep it running while
working on the tests:

.. code-block:: shell

    ./scripts/minio.sh up
    eval "$(./scripts/minio.sh env)"
    make test
    ./scripts/minio.sh down

The store is selected by ``TRUHE_TEST_S3_ENDPOINT``,
``TRUHE_TEST_S3_ACCESS_KEY``, ``TRUHE_TEST_S3_SECRET_KEY``,
``TRUHE_TEST_S3_REGION`` and ``TRUHE_TEST_S3_BUCKET``, plus
``TRUHE_TEST_S3_PATH_STYLE=1`` if needed. **The bucket is emptied before each
test**, so it must be one which holds nothing else.


Contributors
------------

- Robert Niederreiter


Changes
=======

1.0.0 (2026-11-04)
------------------

- Initial release. Encrypted blob store with stores for memory, the
  filesystem and S3, synchronous over ``urllib3``, asynchronous over
  ``httpx``. AES-256-GCM in 64 KiB blocks, streaming in both directions.
  [rnix]


License
=======

MIT License

Copyright (c) 2026 Conestack

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
