Metadata-Version: 2.5
Name: chunkedte
Version: 0.2.0
Summary: Strict, incremental, typed, zero-dependency codec for HTTP/1.1 chunked transfer coding
Project-URL: Homepage, https://lianes8studio.com
Project-URL: Repository, https://github.com/JaviLianes8/chunkedte
Project-URL: Issues, https://github.com/JaviLianes8/chunkedte/issues
Project-URL: Changelog, https://github.com/JaviLianes8/chunkedte/blob/main/CHANGELOG.md
Project-URL: LinkedIn, https://www.linkedin.com/in/jlianes/
Project-URL: Funding, https://buymeacoffee.com/jlianesglrs
Author-email: Javier Lianes <javier@lianes8studio.com>
License-Expression: MIT
License-File: LICENSE
Keywords: chunked,codec,http,request-smuggling,rfc9112,trailers,transfer-encoding
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Description-Content-Type: text/markdown

# chunkedte

Strict, incremental, typed, zero-dependency codec for HTTP/1.1 chunked transfer coding
(RFC 9112 section 7.1).

## Install

```
pip install chunkedte
```

Python 3.10 or newer. No dependencies.

## Usage

Decode a complete message in one call. `decode` raises `ChunkedError` if the message is
malformed, unfinished, or followed by more bytes:

```python
from chunkedte import ChunkedError, decode

print(decode(b"5\r\nhello\r\n0\r\n\r\n"))
try:
    decode(b"0x5\r\nhello\r\n0\r\n\r\n")
except ChunkedError as error:
    print(error)
# b'hello'
# malformed chunk-size line: b'0x5'
```

Decode incrementally as bytes arrive off a socket. `Decoder.feed` takes any split and
returns the body pieces available so far; trailer fields and whatever follows the message
(a pipelined request, say) are kept apart:

```python
from chunkedte import Decoder

decoder = Decoder()
body = b""
for part in [b"4\r\nwi", b"ki\r\n5;sig=ab\r\npedia\r\n0\r\nDigest: x\r\n", b"\r\nGET /"]:
    for piece in decoder.feed(part):
        body += piece
print(body, decoder.is_complete)
print(decoder.trailers, decoder.unused_data)
# b'wikipedia' True
# [('Digest', 'x')] b'GET /'
```

Bound what a peer can make you buffer. Passing a bound raises `LimitError`, a subclass of
`ChunkedError`, so a server can answer 413 or 431 instead of 400:

```python
from chunkedte import Decoder, LimitError, Limits

decoder = Decoder(Limits(max_body_size=1024, max_trailer_size=4096))
try:
    decoder.feed(b"10000\r\n")
except LimitError as error:
    print(error)
print(Limits())
# body exceeds max_body_size
# Limits(max_line_size=4096, max_extensions_size=16384, max_chunk_size=9223372036854775807, max_body_size=None, max_trailer_size=16384)
```

Encode in one call, or stream with `Encoder` and close with optional trailer fields:

```python
from chunkedte import Encoder, encode

print(encode(b"abcd", chunk_size=2))
encoder = Encoder()
print(encoder.write(b"hello") + encoder.finish({"X-Checksum": "42"}))
# b'2\r\nab\r\n2\r\ncd\r\n0\r\n\r\n'
# b'5\r\nhello\r\n0\r\nX-Checksum: 42\r\n\r\n'
```

Decide whether a message body is chunked at all from its Transfer-Encoding and
Content-Length values (`None` when absent). `is_chunked` applies RFC 9112 sections 6.1 and
6.3 and raises `ChunkedError` on the combinations used for request smuggling:

```python
from chunkedte import ChunkedError, is_chunked

print(is_chunked("gzip, chunked"), is_chunked(None, "12"))
print(is_chunked("gzip", request=False))
for te, cl in [("chunked", "12"), ("chunked, chunked", None), ("chunked, gzip", None)]:
    try:
        is_chunked(te, cl)
    except ChunkedError as error:
        print(error)
# True False
# False
# both Transfer-Encoding and Content-Length are present
# chunked is applied more than once
# chunked is not the final transfer coding of a request
```

### Command line

```
$ printf '5\r\nhello\r\n0\r\nX-Sum: 1\r\n\r\n' | chunkedte decode
hello
X-Sum: 1
$ printf 'abc' | chunkedte encode --chunk-size 2 --trailer "X-Sum: 1"
```

`decode` writes the body to stdout and the trailer fields to stderr; `encode` frames stdin
(or a file) as a chunked body. Both exit with status 1 and a message on bad input; body
bytes decoded before the error may already be on stdout, so treat the output of a failed
run as truncated. Trailer bytes outside printable ASCII are written as escapes.
`python -m chunkedte` works too.

## What "strict" means here

Every leniency in a chunked parser is a way for two HTTP processors to disagree on where a
message ends, which is how request smuggling works. So the decoder rejects:

- a chunk size that is not only hex digits (`+5`, `0x5`, `5_0`, ` 5`, `5 `);
- chunk extensions that do not follow the grammar (whitespace is allowed only around `;`
  and `=`, values are tokens or quoted strings);
- a bare LF or a bare CR anywhere a CRLF is expected, including inside an extension;
- anything other than CRLF right after the chunk data;
- trailer lines that are not `name: value`, with whitespace before the colon, with control
  bytes, or folded onto a continuation line (obs-fold).

Extensions are validated and then ignored, as RFC 9112 asks. The default `Limits` cap one
size line at 4 KiB, all extensions of a message at 16 KiB, the trailer section at 16 KiB and
a chunk size below 2**63 (so a forwarded size never wraps in a 64-bit parser); the body is
unbounded unless you set `max_body_size`. A line split over many small reads is scanned
once, so decoding stays linear in the input. When `feed` raises, the pieces decoded in that
call are dropped with it. Received trailers are returned as they came: RFC 9110 section
6.5.2 says not to merge them into the header section, so keep them apart (a `Content-Length`
trailer, say, must never reach your framing logic). The encoder refuses trailer names that
are not tokens, values with control bytes or edge whitespace, and the fields RFC 9110
section 6.5.1 needs before the content: framing and routing (Content-Length,
Transfer-Encoding, Trailer, TE, Host, Connection, Keep-Alive, Upgrade, Max-Forwards),
authentication (Authorization, Proxy-Authorization, WWW-Authenticate, Proxy-Authenticate,
Cookie, Set-Cookie), request modifiers and response controls (Expect, Range,
Cache-Control) and content format (Content-Type, Content-Encoding, Content-Range).
`is_chunked` leaves the rest of RFC 9112 section 6.3 to you: responses to HEAD, 1xx, 204,
304 and 2xx to CONNECT carry no body, and an HTTP/1.0 message with Transfer-Encoding has
faulty framing. There is no lenient mode.

## Why

Chunked transfer coding is simple to frame and easy to get subtly wrong: the terminating
`0` chunk, the closing blank line, extensions and trailers are routinely mishandled, and
the security advisories of the last few years (waitress, h11, aiohttp, netty, Go net/http,
Node.js) are all variations on a parser accepting bytes another parser reads differently.
The existing Python options are tied to one HTTP stack (`h11`, `aiohttp`) or to one server.
This is a standalone, typed, zero-dependency codec with the strict behaviour by default:
reach for it when you move chunked framing yourself and do not want a whole HTTP stack.

## License

MIT. See [LICENSE](LICENSE).

## Links

- Website: https://lianes8studio.com
- GitHub: https://github.com/JaviLianes8
- LinkedIn: https://www.linkedin.com/in/jlianes/
