Metadata-Version: 2.4
Name: rowver
Version: 15.0.0
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Dist: numpy
Requires-Dist: awkward ; extra == 'all'
Requires-Dist: pyarrow ; extra == 'all'
Requires-Dist: pandas ; extra == 'all'
Provides-Extra: all
License-File: LICENSE
Summary: Read .rvr files as NumPy, Awkward, Arrow and pandas, and make them with rvr.
Keywords: hep,cms,root,columnar,numpy
Author: Derek Grove
License-Expression: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://gitlab.cern.ch/dgrove/rowver/-/blob/main/crates/rowver-py/README.md
Project-URL: Repository, https://gitlab.cern.ch/dgrove/rowver

# rowver (Python)

Read `.rvr` files in Python, through the same Rust reader as the `rvr` command, which this
package also installs: `pip install rowver` (Python 3.11 or later).

```python
import rowver

f = rowver.open("higgs_tautau.rvr")
f                         # <rowver.File 'higgs_tautau.rvr': 2248 rows, 11 columns, 1 derived, 2 samples>
f.samples, f.columns      # names, in the file's order
f["MET_pt"]               # NumPy array, one value per row (no copy, read-only: .copy() to change it)
f["Muon_pt"]              # Awkward array, a list per row
f["sample"], f["entry"]   # each row's sample index, and its entry in its file's tree
f["MET_pt"][f.rows_of("GluGluToHToTauTau")]  # one sample's rows: a slice, as a sample's rows come together
len(f), "MET_pt" in f     # rows; whether it has a column of that name (sample and entry too)
f.keys()                  # every name f[...] reads: sample, entry, the columns, the derived ones, the sample columns
f.show()                  # the same, a line each with its kind (f.kinds) and a derived column's expression
f["xsec"]                 # a sample column (one value per sample), as its sample's value in every row
f.path, f.header          # where it is; its JSON header, as a dict
f.derived                 # the derived columns: kept as expressions, worked out when first asked for
f["mean_jet_pt"]          # a derived column, read as any other, checked against what the file records
f.slice("MET_pt", 100, 200)  # rows 100 to 199 alone: only the chunks of the file that hold them read
f.slice("MET_pt", -100)      # the last 100; f.slice(name, slice(a, b, step)) reads as a list's slice
f.labels                  # each sample's label, what a plot shows
f.sample_columns          # per-sample values: events_read, events_total, metadata
f.recipe                  # the recipe.toml that made it
f.study                   # {"recipe.toml": …, "samples.toml": …, "samples.lock": …}
f.signed_by               # the public key that signed it, or None
f.complete                # whether it holds every entry its press chose (no holes, not empty)
f.empty                   # whether rvr empty let its columns go (rvr refill presses them again)
f.holes                   # what it could not read: [{"sample", "file", "entries", "kind", "why"}, …],
                          # entries (first, last) as the file records them, last not among them

f.arrays()                # every column as one Awkward record array, sample and entry first
f.arrays(["MET_pt"])      # sample, entry, then the columns named
f.arrays(["MET_pt"], sample="GluGluToHToTauTau")  # one sample's rows alone, only its chunks read (to_* too)
f.to_arrow()              # a pyarrow Table (list columns as Arrow lists), sample and entry first
f.to_pandas()             # a pandas DataFrame, sample (a categorical) and entry first
f.to_parquet("out.parquet")

with rowver.open("higgs_tautau.rvr") as f:  # f.close() at the end: what it read stays,
    met = f["MET_pt"]                        # its header still reads, a column then raises Error
```

[The examples](https://gitlab.cern.ch/dgrove/rowver/-/tree/v15.0.0/docs/examples/python) work with
the columns of a file, and plot them with matplotlib, hist and ROOT.

A column is the same array each time it is asked for while anything holds it; the `File` keeps none
in memory itself, so memory is what your own variables hold. A `File` pickles as its path (for
multiprocessing, dask and the like), and a name it has no column of raises `KeyError` naming the
nearest it has. Every column asked for at once (`f.read([...])`, `arrays`, `to_*`) is read in parallel. Only
NumPy is required; Awkward, pyarrow and pandas are used when installed
(`pip install "rowver[all]"`). A file whose header does not check out raises `rowver.Error`, a
path that is not there `FileNotFoundError`; a file changed while it is read raises `rowver.Error`
saying so. `rowver.validate(path)` checks every array and works out every derived column, as
`rvr validate` does, and gives `(rows, columns)`. Reading a column checks its chunk table and
every compressed frame's own checksum, and leaves each chunk's SHA-256 to `rowver.validate`. An open `File` holds one open file, through
which it reads, so a file renamed over it meanwhile is read as it was; dropping the `File`
(`f.close()`, or `del f`) lets it go (a system allows a few hundred open files: 256 in a macOS shell). These, `rowver.open` and `rowver.File` are the
whole package; every other name in it begins with `_`.

## Building

From this folder, with Rust 1.91 or later: `maturin develop --release` installs it into the
active Python; `maturin build --release` writes a wheel (one for every Python from 3.11).
It builds on Linux and macOS.

MIT licensed.

