Metadata-Version: 2.4
Name: fletchr-core
Version: 0.0.1rc6
Summary: Arrow-backed frame and packet containers, transformers, pipeline, and I/O for binary protocol data.
Project-URL: Homepage, https://github.com/fletchr-labs/fletchr
Project-URL: Repository, https://github.com/fletchr-labs/fletchr
Project-URL: Issues, https://github.com/fletchr-labs/fletchr/issues
Project-URL: Changelog, https://github.com/fletchr-labs/fletchr/blob/main/fletchr-core/CHANGELOG.md
Author-email: Jonathan Olsten <jolsten@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: arrow,binary-protocol,framework,pipeline,pyarrow
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: attrs>=22.1.0
Requires-Dist: escapement>=0.2.0
Requires-Dist: eval-type-backport>=0.2.0; python_version < '3.10'
Requires-Dist: fletchr-measurand>=0.0.1rc4
Requires-Dist: fletchr-uintn>=0.0.1rc3
Requires-Dist: numpy>=2.0
Requires-Dist: ormsgpack>=1.5.0
Requires-Dist: polars>=0.20
Requires-Dist: pyarrow>=16
Requires-Dist: varuintarray>=1.3.0
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == 'pandas'
Description-Content-Type: text/markdown

# fletchr-core

Arrow-backed frame and packet containers, transformers, pipeline, and
I/O for binary protocol data. The integration layer of the
[`fletchr`](https://github.com/fletchr-labs/fletchr) framework — built
on [`fletchr-uintn`](https://github.com/fletchr-labs/fletchr/tree/main/fletchr-uintn)
for bit-precise storage and
[`fletchr-measurand`](https://github.com/fletchr-labs/fletchr/tree/main/fletchr-measurand)
for bit-fragment extraction and decoding.

## Why?

Binary-protocol pipelines repeat the same shape across projects:

- **Containers** for fixed-width frames (rectangular) and variable-length
  packets (jagged).
- **Transformers** to filter, subframe, invert, synchronize, and
  decode-on-the-fly.
- **I/O** for the formats raw data arrives in (CSV, packed bits, msgpack,
  pickle) and the formats it leaves in (Arrow IPC, Parquet).
- **A plugin model** so domain-specific packet types and transformers
  plug in cleanly without forking the framework.

`fletchr-core` provides each of these as one Arrow-native, null-aware
unit. Containers are backed by `pa.Table` with
`fletchr.uintn(bits=N)` columns (no out-of-band schema metadata, nulls
propagate end-to-end); the Measurand build chain consumes those tables
directly; transformers compose into pipelines with provenance tracking
built in.

## Features

- **Containers** — `FrameArray` (column-major fixed-width frames),
  `MuxFrameArray` (multiplexed variants), `PacketList` (variable-length
  packets, homogeneous), `Packet` / `GenericPacket` / `HeaderPacket` /
  `define_packet` for declaring per-protocol packet types.
- **`fields` declarations** — `Packet` and `FrameArray` subclasses
  declare data-derived columns via `fields: ClassVar[dict[str, Field]]`
  using the measurand parameter DSL. The framework parses specs at
  class-definition time, materializes columns at construction, and
  exposes them via `__getattr__`. One-line column additions, no codec
  hooks to write.
- **Transformers + Pipeline** — `FrameFilter`, `MuxFilter`, `MuxSelect`,
  `FrameSynchronizer`, `Subframe`, `Reverse`, `Invert`, `PolarityFix`,
  `PacketFilter`, `PacketSelect`, `PatternPacketExtractor`,
  `DelimitedPacketExtractor`, clock transformers, plus the `Pipeline`
  container for composition.
- **I/O** — `read_arrow` / `write_arrow` (Arrow IPC), `read_parquet` /
  `write_parquet`, `read_csv` / `write_csv`, `read_bits` / `write_bits`,
  `read_txt` / `write_txt`, `read_pickle` / `write_pickle`, plus the
  generic `read_file` / `write_file` dispatch and `FileReader` /
  `FileWriter` extension points.
- **Polars + pandas adapters** — `add_columns`, `merge`, `stack` for
  interop with the common DataFrame libraries.
- **History / provenance** — optional processing-history DAG attached to
  containers; tracks slice / concat / transform operations. Toggle with
  `enable_history` / `disable_history`.
- **Plugin registry** — downstream packages register custom `Packet`
  subclasses, `Transformer`s, and I/O readers/writers via Python entry
  points. `register_plugin_group("my_pkg.plugins")` to add a new
  discovery namespace.
- **Custom exception hierarchy** — `FletchrCoreError` root with subsystem
  branches (`ContainerError`, `TransformError`, `CodecError`,
  `IODispatchError`, `RegistryError`, `PluginError`, `GrammarError`).
  Each leaf also multi-inherits the matching builtin (`ValueError` /
  `TypeError` / `KeyError` / `ImportError`) so existing
  `except ValueError:` callers keep working.

## Install

```bash
uv add fletchr-core                # or: pip install fletchr-core
uv add 'fletchr-core[pandas]'      # with pandas adapter (polars is required)
```

Requires Python 3.9+. Pulls in `fletchr-uintn`, `fletchr-measurand`,
`pyarrow >= 16`, `numpy >= 2.0`, `polars >= 0.20`, `attrs`, `escapement`,
`ormsgpack`, `varuintarray`.

## Quickstart

```python
import numpy as np
from varuintarray import VarUIntArray
from fletchr_core import FrameArray, write_arrow, read_arrow
from fletchr_measurand import Measurand, parameter_parser

# Build a 2-row, 3-word FrameArray with 8-bit words.
time = np.array(["2026-01-01", "2026-01-02"], dtype="datetime64[ns]")
data = VarUIntArray(
    np.array([[10, 20, 30], [40, 50, 60]], dtype=np.uint8),
    word_size=8,
)
frame = FrameArray(time=time, ctime=time, data=data)

frame.shape                  # (2, 3)
frame.table.column_names     # ['time', 'ctime', 'c0', 'c1', 'c2']

# Compute a measurand directly against the FrameArray's Arrow table.
# Parameter "[1+2]" reads word 1 + word 2 as a single 16-bit value.
m = Measurand(parameter=parameter_parser.parse("[1+2]"))
m.build(frame.table).values.to_pylist()
# [2580, 10290]  =  [10*256 + 20, 40*256 + 50]

# Round-trip through Arrow IPC.
write_arrow(frame, "frame.arrow")
back = read_arrow("frame.arrow")
assert back.shape == frame.shape
```

### Declarative fields on a subclass

`FrameArray` (and `Packet`) subclasses can declare data-derived columns
using the [`fletchr-measurand`](https://github.com/fletchr-labs/fletchr/tree/main/fletchr-measurand)
parameter DSL. The framework parses specs at class-definition time,
materializes the columns at construction, and exposes them as
attributes:

```python
from typing import ClassVar
import numpy as np
from varuintarray import VarUIntArray
from fletchr_core import FrameArray, Field

class NumberedFrame(FrameArray):
    fields: ClassVar[dict[str, Field]] = {
        "counter":        Field("[1+2]"),               # raw 16-bit (uintn(bits=16))
        "counter_halved": Field("[1+2];u;EUC[0.5]"),    # decoded float64, scaled ×0.5
    }

time = np.array(["2026-01-01", "2026-01-02"], dtype="datetime64[ns]")
data = VarUIntArray(
    np.array([[0, 1, 100], [1, 0, 200]], dtype=np.uint8),
    word_size=8,
)
fa = NumberedFrame(time=time, ctime=time, data=data)
list(fa.counter)         # [1, 256]
list(fa.counter_halved)  # [0.5, 128.0]
```

Field specs use the [`fletchr-measurand`](https://github.com/fletchr-labs/fletchr/tree/main/fletchr-measurand)
grammar — `parameter ";" encoding ";" euc ";" sampling-strategy`,
encoding/euc/ss independently optional. Parameter-only specs like
`[1+2]` extract raw bits (natural output: `fletchr.uintn(bits=N)`);
full measurand specs like `[1+2];2c;EUC[0.1]` apply Level 1 (encoding)
and Level 2 (engineering-unit conversion) at construction time. When
`dtype=` is omitted, the column's type is whatever the measurand chain
naturally produces.

## Public API

```python
from fletchr_core import (
    # Containers
    FrameArray, MuxFrameArray,
    PacketList, Packet, GenericPacket, HeaderPacket, define_packet,
    # Pipeline
    Transformer, Pipeline, stack,
    # I/O (generic + per-format)
    read_file, write_file, FileReader, FileWriter,
    read_arrow, write_arrow,
    read_parquet, write_parquet,
    read_csv, write_csv,
    read_bits, write_bits,
    read_txt, write_txt,
    read_pickle, write_pickle,
    # DataFrame interop
    add_columns, merge,
    # History
    History, HistoryGraph, HistoryNode,
    enable_history, disable_history, is_history_enabled, reset_history_graph,
    # Plugins
    load_plugins, register_plugin_group,
    # Exceptions
    FletchrCoreError, ContainerError, TransformError,
    HomogeneityError, BitWidthError, SchemaError,
    PipelineError, TransformConfigError,
    CodecError, IODispatchError,
    RegistryError, RegistryCollisionError,
    PluginError, GrammarError,
)
```

Concrete transformers (`FrameFilter`, `Invert`, `MuxSelect`, etc.) live
under `fletchr_core.transform`.

## Extensibility

`fletchr-core` is a **framework**, not a finished application. Domain
packages plug in by registering subclasses through Python entry points:

```toml
# in your downstream package's pyproject.toml
[project.entry-points."fletchr_core.plugins"]
my_protocol = "my_pkg.fletchr_plugins"
```

Modules in that entry-point group are imported lazily on first registry
access; any `Packet` / `FrameArray` / `Transformer` / `FileReader` /
`FileWriter` subclasses they define land in the same global registries
as the built-ins. Use `register_plugin_group("my_namespace.plugins")` to
add a new discovery namespace.

## Claude Code skills

`fletchr-core` ships [Claude Code](https://docs.claude.com/claude-code)
skills as bundled package data. Install them into your user Claude
skills directory:

```bash
python -m fletchr_core.skills list                # show bundled skills
python -m fletchr_core.skills install             # copy to ./.claude/skills/ (project)
python -m fletchr_core.skills install --user      # copy to ~/.claude/skills/
```

Project-level is the default (skills are typically scoped to the
project that needs them); `--user` elevates to user-level.

The shipped skills travel versioned with the code they describe — when
you upgrade `fletchr-core`, re-run `install --force` to refresh.

## Links

- Source: <https://github.com/fletchr-labs/fletchr>
- Issues: <https://github.com/fletchr-labs/fletchr/issues>

## License

MIT.
