Metadata-Version: 2.4
Name: earthframe-codec
Version: 0.2.0
Summary: Zero-dependency streaming decoders for environmental binary telemetry
Author: earthframe-codec contributors
License: MIT License
        
        Copyright (c) 2026 Baller300000
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Keywords: grib,bufr,hdf5,netcdf,nexrad,telemetry
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"
Dynamic: license-file

# earthframe-codec

`earthframe-codec` is a zero-runtime-dependency Python library for bounded, incremental inspection of environmental binary frames. It is designed for telemetry gateways and archive readers that cannot load multi-gigabyte products into memory just to validate a header.

The package exposes one status contract for every input buffer:

- `SCIENCE_VALID_FRAME`: structural checks passed.
- `SCIENCE_INCOMPLETE_STREAM`: more bytes are required.
- `SCIENCE_CORRUPTED_FRAME`: the bytes cannot satisfy the selected format rules.

## Scope

The current release provides production-safe structural decoders for GRIB2, BUFR, HDF5 signatures/superblocks, and NEXRAD archive messages. GRIB2 simple packing, BUFR descriptor words, and legacy one-byte radar bins have reusable bitwise primitives. HDF5 object trees and vendor-specific radar message payloads intentionally remain exposed as bounded raw offsets because their layouts depend on version, templates, filters, and local tables.

This is not a claim that one decoder can infer every vendor template or every NetCDF convention. NetCDF4 data is HDF5 storage with NetCDF metadata conventions; callers should use the returned HDF5 superblock information to schedule dataset-specific block reads.

```text
python -m pip install earthframe-codec
```

The current release provides production-safe structural decoders for GRIB2, BUFR, HDF5 signatures/superblocks, NEXRAD archive messages, CCSDS Space Packets, and miniSEED records. GRIB2 simple packing, BUFR descriptor words, legacy one-byte radar bins, CCSDS primary headers, and miniSEED Blockette 1000 records have reusable bitwise/framing primitives. HDF5 object trees and vendor-specific radar message payloads intentionally remain exposed as bounded raw offsets because their layouts depend on version, templates, filters, and local tables.
No NumPy, HDF5 C library, or network service is required at runtime. Python 3.10+ is supported.

## Streaming integration

```python
from earthframe_codec import FrameDecoder, FrameStatus

reader = FrameDecoder(max_frame_size=256 * 1024 * 1024)
with open("telemetry.bin", "rb", buffering=0) as source:
    while block := source.read(1024 * 1024):
        for result in reader.feed(block):
            if result.status is FrameStatus.SCIENCE_VALID_FRAME:
                print(result.protocol, result.consumed, result.data.keys())
            elif result.status is FrameStatus.SCIENCE_CORRUPTED_FRAME:
                raise ValueError(result.errors)
    for result in reader.feed(b"", final=True):
        print(result.status, result.errors)
```

`FrameDecoder` retains a bytearray only for the current incomplete record. `decode_frame(memoryview(...))` accepts a caller-owned buffer and does not copy section payloads while validating them. Results copy only small metadata fields; raw payloads are included as bytes where a protocol has no safe universal template.

## Protocol fields

### GRIB2

The decoder verifies `GRIB`, edition 2, the 8-byte total length, section lengths, section ordering boundaries, and the `7777` end marker. Section summaries expose raw section bytes plus common fields:

```python
from earthframe_codec.grib2 import unpack_simple_packing
values = unpack_simple_packing(payload, count=nx * ny, bits_per_value=12,
                               reference=reference_value,
                               binary_scale=binary_scale,
                               decimal_scale=decimal_scale)
```

Grid schemas are returned as metadata rather than guessed coordinates. Common section 3 fields include `grid_template`, `ni`, and `nj`; use the template-specific Earth model and scan-mode flags to reconstruct longitude/latitude arrays.

### BUFR

BUFR section boundaries and edition are checked. Section 3 descriptor words are decoded into `(F, X, Y)` using `F=(word >> 14) & 3`, `X=(word >> 8) & 63`, and `Y=word & 255`. Table B/C/D interpretation is intentionally supplied by the application or station profile because descriptors and local tables evolve independently of the wire framing.

### HDF5 / NetCDF4

The HDF5 signature and superblock version/address sizes are checked without reading the full file. `root_object_offset`, when present in superblock v0, is an absolute file offset suitable for a block scheduler. Dataset filters, fractal heaps, B-trees, and NetCDF attributes must be interpreted against the file's exact HDF5 version and are not safely interchangeable.

### NEXRAD

The archive signature, message length, type, and channel are checked. `decode_legacy_bins` turns byte-valued REF/VEL/SW bins into scaled values while preserving the configured missing sentinel. Vendor message 1/31/packet-structure templates should be decoded by a product-specific adapter using the returned payload.

## Development

```text
python -m pip install -e ".[test]"
python -m pytest
```

### CCSDS Space Packets

The CCSDS decoder validates the six-byte primary header, packet version, APID, sequence flags/count, and the CCSDS length rule (`total = 6 + packet_length + 1`). Because CCSDS packets have no universal magic prefix, pass `"ccsds"` to `decode_frame` or `FrameDecoder`.

### miniSEED

The miniSEED decoder validates the fixed header's numeric sequence, follows the blockette chain, reads Blockette 1000 encoding/byte order, and derives the power-of-two record size without loading a stream. It exposes station, location, channel, network, sample count, and sample-rate fields; waveform sample decoding remains dependent on the selected encoding and is intentionally left to a codec adapter.

The GitHub Actions workflow builds and publishes distributions for version tags matching `v*.*.*`. Configure PyPI trusted publishing for the repository before pushing a tag.
