Metadata-Version: 2.4
Name: gwseq_io
Version: 0.1.15
Summary: Python library for processing bigWig, bigBed, BAM and HiC files
Author-Email: Arthur Gouhier <ajgouhier@gmail.com>
License-Expression: MIT
License-File: LICENSE
Project-URL: Repository, https://github.com/ajgouhier/gwseq_io
Requires-Python: >=3.9
Requires-Dist: numpy
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: scikit-build-core>=1.0.0; extra == "dev"
Requires-Dist: nanobind>=2.11.0; extra == "dev"
Requires-Dist: cmake; extra == "dev"
Requires-Dist: matplotlib; extra == "dev"
Requires-Dist: pandas; extra == "dev"
Requires-Dist: pyBigWig; extra == "dev"
Requires-Dist: pysam; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Description-Content-Type: text/markdown

# gwseq_io

Python library for processing bigWig, bigBed, BAM and HiC files.
Backed by a C++17 core via nanobind.


## Installation

```
pip install gwseq-io
```

Requires numpy, installed automatically as a dependency.


## Usage

### Open bigWig, bigBed, BAM and HiC files for reading

```python
reader = gwseq_io.open(path, mode, parallel, zoom_correction, file_buffer_size, max_file_buffer_count, index_path)

with gwseq_io.open("path/to/file.bigwig") as reader: # .bigbed .bam .hic
    ...
```

Parameters:
- `mode` Opening mode. May be omitted as "r" (read) by default.
- `parallel` Number of parallel file handles and processing threads. Use -1 for recommended (one per core, capped at 12). -1 by default.
- `zoom_correction` Scaling factor for automatic zoom level selection based on bin size. Only for bigWig files. 1/3 by default.
- `file_buffer_size` Size in bytes of each file buffer for caching file reads. Use -1 for recommended (32768 or 1048576 for URLs). -1 by default.
- `max_file_buffer_count` Maximum number of file buffers to keep in cache. Use -1 for recommended (128). -1 by default.
- `index_path` Path of the index. Only for BAM files, where it defaults to the path of the file with ".bai" appended. An index is optional, but reading entries needs one.

Every reader also has:
- `close` Give back the `parallel` file handles and the threads the reader holds for as long as it lives. Calling it twice is harmless, and a reader is a context manager, so leaving the `with` block above closes it. Reading through a closed reader raises, and the headers it read at open stay readable. A reader that is never closed gives everything back when it is collected instead.
- `closed` Whether `close` has run.

Attributes for bigWig and bigBed files:
- `main_header` General file formatting info.
- `zoom_headers` Zooms levels info (reduction level and location).
- `auto_sql` BED entries declaration (only in bigBed).
- `total_summary` Statistical summary of entire file values (coverage, sums and extremes).
- `chr_sizes` Map of chromosome IDs and their sizes.
- `type` Either "bigwig" or "bigbed".

Attributes for BAM files:
- `header` Header lines, each a dict of its "type" (the two letters after the @) and its "fields".
- `chr_sizes` Map of reference IDs and their sizes.
- `is_indexed` Whether the index was found and read. Reading entries needs it.
- `index_error` Why the index is absent, when it is. Empty when it loaded, and empty as well when the file simply has none.

Attributes for HiC files:
- `header` `footer` General file info.
- `chr_sizes` Map of chromosome IDs and their sizes.
- `normalizations` Available normalizations.
- `units` Available units.
- `bin_sizes` Available bin sizes.

### Read bigWig and bigBed values

```python
values = reader.read_values(chr_ids, starts, ends, centers, span, ...)

values = reader.read_values(chr_ids=["chr1", "chr1"], starts=[1000, 1100], ends=[1100, 1200])
values = reader.read_values(chr_ids=["chr1", "chr1"], starts=[1000, 1100], span=100)
values = reader.read_values(chr_ids=["chr1", "chr1"], ends=[1100, 1200], span=100)
values = reader.read_values(chr_ids=["chr1", "chr1"], centers=[1050, 1150], span=100)
values = reader.read_values(chr_ids=["chr1", "chr1"], centers=[1050, 1150], span=100, strands=["+", "-"])
```

Parameters:
- `chr_ids` `starts` `ends` `centers` Chromosome IDs, starts, ends and centers of the locations. Both `starts` `ends`, or one of `starts` `ends` `centers` with `span`, may be specified.
- `span` Reading window in bp relative to `starts`, `ends` or `centers`. Only one of the three may be given with it. Not by default.
- `strands` Strand of each location, as "+" or "-" ("." and "" count as "+"). The values of a "-" location are reversed, so that every location reads from its own start. All "+" by default.
- `bin_size` Reading bin size in bp. May vary in output if locations have variable spans or `bin_count` is specified. 1 by default. A fractional value is accepted and snaps the window to a grid of that width, so the window edges no longer fall on whole bases; pass a whole number unless that is what you want (`iter_all_values` requires one).
- `bin_count` Output bin count. Inferred as max location span / bin size by default.
- `bin_mode` Method to aggregate bin values, all three per base of the bin rather than per record of the file: "mean" is the base-weighted mean, "sum" the value summed over each base it covers, and "count" the bases of the bin carrying data — the bin's width where the file covers it fully. "mean" by default. For a bigBed the value of a bin is the depth of coverage its entries make over that bin.
- `full_bin` Extend locations ends to overlapping bins if true. Not by default.
- `def_value` Default value to use when no data overlap a bin. 0 by default.
- `zoom` BigWig zoom level to use. Use full data if -1, or auto-detect if -2 by taking the coarsest level whose bin size is under `bin_size` times `zoom_correction` (may be the full data). Full data by default.
- `progress` Function called during extraction with the extracted and the total coverage in bp. Use the default callback if true. None by default.

Returns a numpy float32 array of shape (locations, bin count).

### Quantify bigWig and bigBed values

```python
values = reader.quantify(chr_ids, starts, ends, centers, span, ...)
```

Parameters:
- `chr_ids` `starts` `ends` `centers` `span` `bin_size` `full_bin` `def_value` `zoom` `progress` Identical to `read_values` method.
- `reduce` Method to aggregate values over span. Either "mean", "sd", "sem", "sum", "count", "min", "max", "l1norm" or "l2norm". "mean" by default.

Returns a numpy float32 array of shape (locations).

### Profile bigWig and bigBed values

```python
values = reader.profile(chr_ids, starts, ends, centers, span, ...)
```

Parameters:
- `chr_ids` `starts` `ends` `centers` `span` `strands` `bin_size` `bin_count` `bin_mode` `full_bin` `def_value` `zoom` `progress` Identical to `read_values` method. A "-" location takes part in the profile reversed, as it would come out of `read_values`.
- `reduce` Method to aggregate values over locations. Either "mean", "sd", "sem", "sum", "count", "min", "max", "l1norm" or "l2norm". "mean" by default.

Returns a numpy float32 array of shape (bin count).

### Iterate over all bigWig and bigBed values

```python
iterator = reader.iter_all_values(...)

iterator = reader.iter_all_values(bin_size=10)
for values in iterator:
    ...
for (chr_id, start, end), values in zip(iterator.locs, iterator):
    ...
```

Parameters:
- `chr_ids` Only walk these chromosomes. All by default.
- `bin_mode` `full_bin` `def_value` `zoom` `progress` Identical to `read_values` method. `full_bin` decides whether the partial bin a chromosome ends on is walked at all.
- `span` Window in bp for each step, rounded up to a whole number of bins. 1 000 000 by default, a million values at the default bin size.
- `bin_size` Identical to `read_values` method, but must be a whole number of bp: the windows tile the genome on this grid. 1 by default.

Returns an iterator over successive windows, each one a numpy float32 array of shape (bins), in chromosome then coordinate order. `len(iterator)` gives the number of windows, and `iterator.locs` the region of each, so the nth array covers `locs[n]`:

Notes:
- A window never spans two chromosomes and no bin straddles a window boundary, so concatenating the windows of a chromosome gives exactly what `read_values` gives for the whole of it at the same bin size. For a bigBed the values are the pileup of its entries.
- An iterator is exhausted after one pass — `__iter__` returns the iterator itself, so `zip(iterator.locs, iterator)` works once and a second walk yields nothing. Ask for a new one to walk again. The same holds for every `iter_*` method of every reader.

### Read bigBed entries

```python
entries = reader.read_entries(chr_ids, starts, ends, centers, span, ...)
```

Parameters:
- `chr_ids` `starts` `ends` `centers` `span` `progress` Identical to `read_values` method.
- `col_count` Only read this number of columns (eg, 3 for chr, start and end). Must be 0 (all) or at least 3. The columns left out are never parsed, so a narrower read is a cheaper one. All by default.

Returns a list (locations) of list of entries (dict with at least "chr", "start" and "end" keys).

### Read all bigBed entries

```python
entries = reader.read_all_entries(...)
```

Parameters:
- `chr_ids` Only extract data from these chromosomes. All by default.
- `col_count` Identical to `read_entries` method.

Returns a list of entries (as in `read_entries`).

### Iterate over all bigBed entries

```python
iterator = reader.iter_all_entries(...)

iterator = reader.iter_all_entries()
for entries in iterator:
    ...
for (chr_id, start, end), entries in zip(iterator.locs, iterator):
    ...
```

Parameters:
- `chr_ids` `col_count` `progress` Identical to `read_all_entries` method.
- `span` Identical to `iter_all_values` method, but a window is measured in bp alone, there being no bins to round it to.

Returns an iterator over successive windows, each one a list of entries (as in `read_entries`), in chromosome then coordinate order. `len(iterator)` gives the number of windows, and `iterator.locs` the region of each.

Notes:
- A window never spans two chromosomes, and an entry reaching over a window boundary is reported by the window it starts in. Concatenating the windows gives exactly what `read_all_entries` returns, in the same order.

### Convert bigWig to bedGraph or WIG

```python
reader.to_bedgraph(output_path, ...)
reader.to_wig(output_path, ...)
```

Parameters:
- `output_path` Path to output file.
- `chr_ids` Only extract data from these chromosomes. All by default.
- `bin_size` `zoom` `progress` Identical to `read_values` method, except that `bin_size` does not rebin here — every value is written as the file stores it, and `bin_size` only steers which zoom level `zoom=-2` picks.

### Convert bigBed to BED

```python
reader.to_bed(output_path, ...)
```

Parameters:
- `output_path` `chr_ids` `progress` Identical to `to_bedgraph` and `to_wig` methods.
- `col_count` Only write this number of columns (eg, 3 for chr, start and end). All by default.

### Read BAM entries

```python
entries = reader.read_entries(chr_ids, starts, ends, centers, span, ...)
```

Parameters:
- `chr_ids` `starts` `ends` `centers` `span` `progress` Identical to bigWig `read_values` method.
- `filter` Drop unmapped alignments, improperly paired reads, secondary and supplementary records, and anything marked as failing quality control or as a duplicate. True by default.
- `parse_tags` Keep the optional fields of every alignment. They are only decoded when read, so this costs one small copy per alignment. True by default.

Returns a list (locations) of list of `BamEntry`, in the order the locations were given. Each location gets its own full list, so two overlapping locations both report the alignments they share.

Notes:
- Needs an index, as `is_indexed` reports. Reading without one raises, naming either the index that was not found or the error it gave.

### Read all BAM entries

```python
entries = reader.read_all_entries(...)
```

Parameters:
- `chr_ids` Only extract data from these references. All by default.
- `filter` `parse_tags` `progress` Identical to `read_entries` method.

Returns a list of `BamEntry` (as in `read_entries`). Unplaced alignments are left out, the index reaching an alignment only through the reference it sits on.

### Iterate over BAM entries

```python
iterator = reader.iter_entries(chr_ids, starts, ends, centers, span, ...)

iterator = reader.iter_entries(chr_ids=["chr1", "chr1"], starts=[1000, 1100], ends=[1100, 1200])
for entries in iterator:
    ...
for loc_index, entries in zip(iterator.order, iterator):
    ...
```

Parameters:
- `chr_ids` `starts` `ends` `centers` `span` `filter` `parse_tags` `progress` Identical to `read_entries` method.
- `sort_locations` Read the locations in reference and position order, and report them in that order. Locations close together then share the blocks a read decompressed, 4 to 7 times faster on a scattered request. False by default.

Returns an iterator over locations, each one a list of `BamEntry`. `len(iterator)` gives the number of locations, and `iterator.order` the request index of each, so the nth list belongs to location `order[n]`:

### Iterate over all BAM entries

```python
iterator = reader.iter_all_entries(...)

for entries in reader.iter_all_entries():
    ...
```

Parameters:
- `chr_ids` `filter` `parse_tags` `progress` Identical to `read_all_entries` method.
- `span` Window in bp for each step. 1000000 by default.

Returns an iterator over successive windows, each one a list of `BamEntry`, in reference then coordinate order. `len(iterator)` gives the number of windows.

Notes:
- A window never spans two references, and an alignment reaching over a window boundary is reported by the window it starts in. Concatenating the windows gives exactly what `read_all_entries` returns, in the same order.

### BAM entries

A `BamEntry` is one alignment.

Attributes:
- `chr` (str) Reference the alignment sits on.
- `start` `end` (int) 0-based half-open span, the end derived from the cigar. Equal for an alignment covering no reference.
- `read_name` (str) QNAME.
- `flag` (int) FLAG, as the raw bitfield.
- `mapping_quality` (int) MAPQ.
- `cigar` (str) CIGAR, eg "10S80M10S". A cigar of more than 65 535 operations does not fit the record's own field, so it is stored in a `CG` optional field and read from there; the placeholder the record carries in its place is never returned, and the `CG` field is left in `tags`.
- `sequence` (str) SEQ, unpacked from its 4-bit encoding, or "*" when the record carries none.
- `qualities` (str) QUAL as phred+33, or "*" when the record carries no qualities at all. A record with only some of them missing spells those "*" in place.
- `next_chr` (str) RNEXT, "*" when the mate sits on no reference.
- `next_start` (int) PNEXT.
- `template_length` (int) TLEN.
- `bai_bin` (int) Index bin the record declares itself in.
- `reference_length` `query_length` (int) Bases of the reference the alignment covers, and of the read its cigar consumes.
- `tags` (dict) Optional fields by two-letter tag, in the order the record stores them. Typed as the file types them: character, integer, float, string, or list of integers or floats. Empty when `parse_tags` is off.
- `is_paired` `is_proper_pair` `is_mapped` `is_next_mapped` `is_reverse` `is_next_reverse` `is_first_in_pair` `is_last_in_pair` `is_secondary_or_supplementary` `is_failed_qc_or_duplicate` (bool) The `flag` bits, decoded. Finer ones are yours to mask off `flag`.

Notes:
- `cigar`, `sequence`, `qualities` and `tags` are decoded the first time they are read and kept afterwards, so an alignment read for its coordinates never pays for the rest of it. The optional fields are only walked when `tags` is read, so a record with malformed ones reads fine and raises there.
- `to_dict()` returns every field as a plain dict under the same names, for pandas, for serialising, or for sending to another process, an alignment itself not being picklable. It holds a "tags" key only when `parse_tags` was on.

### Read HiC values

```python
values = reader.read_values(chr_ids, starts, ends, ...)
```

Parameters:
- `chr_ids` `starts` `ends` Chromosome IDs, starts and ends of the two locations.
- `bin_size` Input bin size or -1 to use the smallest. Must be available in the file. Smallest by default.
- `bin_count` Approximate output bin count. Takes precedence over `bin_size` if specified by selecting the closest bin size resulting in `bin_count`. Not specified by default.
- `exact_bin_count` Resize output to match `bin_count` (if specified). Not by default.
- `full_bin` Extend locations ends to overlapping bins if true. Not by default.
- `def_value` Default value to use when no data overlap a bin. 0 by default. A bin holding a contact the file cannot value — one the chosen `normalization` has no factor for, or an expected value of zero — comes back NaN instead, that being a different answer from not having been observed at all.
- `triangle` Skip symmetrical data if true. Not by default. On one chromosome a hic file stores one side of the diagonal only, and `triangle` reads that side alone rather than mirroring it, so a window lying entirely on the *other* side comes back empty — `starts=[15_000_000, 10_000_000]` gives `def_value` throughout where the mirrored request gives the data. Leave it off unless you know which side your window is on.
- `min_distance` `max_distance` Min and max distance in bp from diagonal for contacts to be reported. All by default.
- `normalization` Either "none" or any normalization available in the file, such as "kr", "vc" or "vc_sqrt". "none" by default.
- `mode` Either "observed", "oe" (observed/expected) or "expected". "observed" by default.
- `unit` Either "bp" or "frag". "bp" by default.
- `save_to` Save output to this .npz path (under "values" key) and return nothing. Not by default.

Returns a numpy float32 array of shape (loc 1 bins, loc 2 bins).

### Read HiC sparse values

```python
values = reader.read_sparse_values(chr_ids, starts, ends, ...)
```

Parameters:
- `chr_ids` `starts` `ends` `bin_size` `bin_count` `exact_bin_count` `full_bin` `triangle` `min_distance` `max_distance` `normalization` `mode` `unit` `save_to` Identical to `read_values` method. There is no `def_value`: a cell this does not list is one no contact reached, which is what a sparse matrix says by leaving it out.

Returns a COO sparse matrix as a dict with keys:
- `values` Values as a numpy float32 array.
- `row` Values rows indices as a numpy uint32 array.
- `col` Values columns indices as a numpy uint32 array.
- `shape` Shape of the dense array as a tuple.

Convert in python using `scipy.sparse.csr_array((x["values"], (x["row"], x["col"])), shape=x["shape"])`.

### Open bigWig and bigBed files for writing

```python
writer = gwseq_io.open(path, mode, type, chr_sizes, genome, fields, items_per_slot, compression_level)

with gwseq_io.open("path/to/file.bigwig", "w", genome="mm10") as writer:
    ...
```

Parameters:
- `mode` Opening mode. Must be set to "w" (write).
- `type` Type of file to write. Either "bigwig" or "bigbed". "bigwig" by default.
- `chr_sizes` Map of chromosome IDs and their sizes. Every written coordinate is checked against it, and a written ID is resolved against its keys the way a read one is, so "1" and "chr1" reach the same entry. Inferred from what is written if omitted, a chromosome then ending where its last value or entry does. None by default.
- `genome` Genome ID to get the chromosome IDs and their sizes, as `get_chr_sizes` returns them. May not be set with `chr_sizes`. None by default.
- `fields` Entries keys and their types. Only for bigBed files. The first three must be the coordinates `chr`, `start`, `end` (or the aliases `chr_id`, `chrom`, `chromStart` and `chromEnd`). Anything else is refused when the file is written. Types may be "string", "int", "uint" or "float". {"chr": "string", "start": "uint", "end": "uint", "name": "string"} by default.
- `items_per_slot` Values or entries one block holds, and records one zoom block holds. Use -1 for recommended (1024 for bigWig, 512 for bigBed, as the UCSC writers use). -1 by default.
- `compression_level` zlib level for the data and zoom blocks, or 0 to leave them uncompressed. 6 by default.
- `parallel` Number of threads compressing blocks. Use -1 for recommended (one per core, capped at 12). -1 by default.

Attributes:
- `path` `type` `closed` Path being written, "bigwig" or "bigbed", and whether `close` has run.
- `chr_sizes` Chromosome sizes as they will be written, in the order the chromosomes were written.
- `section_count` `section_counts` Sections, or blocks of entries, written so far — in total and, for a bigWig, by encoding ("bedgraph", "varstep", "fixedstep"). Each section takes whichever of the three costs the fewest bytes. Counted when a block is placed in the file, not when it is handed over, so with `parallel > 1` a section still being compressed is not in the tally yet; the counts are complete once `close()` has run.
- `entry_count` `fields` Entries written so far, and the columns they are written with (bigBed only).
- `skipped_count` Values dropped for not being finite.
- `clipped_count` Values cut back to the end of a declared chromosome they hung over.

Notes:
- `close`, which is what leaving the `with` block runs, finishes the file and stops the compression threads. Nothing on disk is a bigWig or bigBed until it returns, and nothing of the writer's is still running once it has. A writer dropped without it is closed by the garbage collector instead, which is later and not up to you.
- Only chromosomes that were actually written go into the file, so `chr_sizes` and `genome` are a bounds check and a spelling of the names rather than a list of what the file will contain.
- A bigWig value that starts inside a declared chromosome and ends past it is written up to that end rather than refused, since a chromosome is rarely a whole number of bins long. One that starts at or past the end raises, as does any bigBed entry running past it.

### Write bigWig values

```python
writer.write_value(chr_id, start, end, value)
writer.write_values(chr_id, start, span, values)

writer.write_value("chr1", start=1000, end=1010, value=0.1)
writer.write_values("chr1", start=1000, span=10, values=[0.1, 0.3, 0.2, 0.1])
```

Parameters (write_value):
- `chr_id` `start` `end` Chromosome ID, start and end of value.
- `value` Location value.

Parameters (write_values):
- `chr_id` `start` Chromosome ID and start of first value.
- `span` Window in bp of each successive locations relative to their starts, so `values[n]` covers `[start + n * span, start + (n + 1) * span)`.
- `values` Locations values, as a list or a numpy array.

Notes:
- Values must be pooled by chromosome, added in order and without overlap.
- Each section is written in the narrowest of the three encodings that holds it: fixedStep while every value shares one span and one step, four bytes a value; variableStep once the starts turn irregular, eight; bedGraph once the spans differ too, twelve. A section opens fixedStep and only widens, and closes on `items_per_slot` values, a change of chromosome, or a value it costs more to widen for — the extra item size over what is buffered and over what is left of the call, discounted for how well a near-constant coordinate column deflates — than to open a header, R-tree leaf and cold zlib stream for, so a one-off irregularity is absorbed where the start of a differently shaped run splits. For better performances, hand successive locations of one span over in one `write_values` call: the run is fixedStep by construction and goes in as one memcpy, its starts never materialised. `section_counts` is the tally.
- A NaN or infinite value is not written. It leaves a gap, which is what a bigWig means by a base carrying no data, and a reader fills it with the `def_value` it was asked for. `skipped_count` counts them.
- A chromosome that received only non-finite values still enters the file's chromosome list, with a size of 1 where its size was not declared — the writer sizes an undeclared chromosome from what reached it, and nothing did. Declare `chr_sizes` if a chromosome has to keep its real length whatever lands on it.
- A value hanging over the end of a declared chromosome is written up to that end, which is what makes a span that does not divide a chromosome ordinary rather than an error. `clipped_count` counts them. A value starting at or past the end raises, and in a `write_values` run only the last value can hang over, so a run reaching whole values past the end raises too.

### Write bigBed entries

```python
writer.write_entry(chr_id, start, end, ...)

writer.write_entry("chr1", start=1000, end=1010, fields={"name": "read#1"})
```

Parameters:
- `chr_id` `start` `end` Chromosome ID, start and end of entry.
- `fields` Map of additional fields as specified in file `fields`. The first three declared fields are the coordinates and are written from `start` and `end`, so naming one here is an error. A declared field left out is written empty for a string and 0 for a number.

Notes:
- Entries must be pooled by chromosome and added in order of their start. Unlike bigWig values, they may overlap and nest freely.
- The summary statistics a bigBed carries, and its zoom levels, describe the depth of coverage its entries make, as the format asks: a base under three entries counts once towards the bases covered and three towards the sum.

### Convert bedGraph or WIG to bigWig

```python
gwseq_io.convert_to_bigwig(input_path, output_path, ...)

gwseq_io.convert_to_bigwig("track.bedgraph.gz", "track.bigwig", genome="mm10")
```

Parameters:
- `input_path` Path to input bedGraph or WIG file. May be gzipped.
- `output_path` Path to output file.
- `bin_size` Force a specified bin size in the output. A bin holds the base-weighted mean of what falls in it, so a 500 bp interval counts for five hundred times what a 1 bp one does, and a bin nothing covers is left as a gap. Takes bins as is by default.
- `chr_sizes` `genome` `items_per_slot` `compression_level` `parallel` Identical to `open` in write mode.
- `progress` Function called during conversion. Takes the bytes read and the total size of the input as parameters. Use default callback function if true. None by default.

Returns a map of `format` ("bedgraph" or "wig", as it was sniffed), `line_count`, `item_count`, `skipped_count`, `clipped_count` (values cut back to the end of their chromosome) and `chr_sizes` as written.

Notes:
- Which of the two formats the input is comes from its content, not from its name: the first line that is neither blank, a comment, nor a `track` or `browser` declaration decides. A `fixedStep` or `variableStep` line makes it a WIG, four columns of chromosome, start, end and value a bedGraph, and anything else is refused. The format is settled once, so a file holding both is refused as well.
- WIG coordinates are 1-based and bedGraph ones 0-based half-open. `step` and `span` both default to 1; a declaration with no `chrom`, or a `fixedStep` with no `start`, is an error rather than a guess.
- Values must be pooled by chromosome, in order and without overlap, as `write_value` asks — what `sort -k1,1 -k2,2n` gives. Input that is not raises, naming the line. Nothing is sorted or spooled, so a conversion of any size holds a megabyte of input and one open section.
- Nothing on disk is a bigWig until the call returns, exactly as for a writer.

### Convert BED to bigBed

```python
gwseq_io.convert_to_bigbed(input_path, output_path, ...)

gwseq_io.convert_to_bigbed("peaks.bed.gz", "peaks.bigbed", genome="mm10")
```

Parameters:
- `input_path` Path to input BED file. May be gzipped.
- `output_path` Path to output file.
- `chr_sizes` `genome` `items_per_slot` `compression_level` `progress` Identical to `convert_to_bigwig`.
- `fields` Entries keys and their types, as in `open` in write mode. Taken from the standard BED columns — `chrom`, `chromStart`, `chromEnd`, `name`, `score`, `strand`, `thickStart`, `thickEnd`, `itemRgb`, `blockCount`, `blockSizes`, `blockStarts`, then `field13` and up — for however many columns the first record has, by default.

Returns a map as `convert_to_bigwig` does, with `format` always "bed".

Notes:
- Lines are split on tabs, a BED being tab-delimited and its `name` column being allowed to hold spaces. Every record must carry the same number of columns as the first one, a bigBed storing one shape of record.
- A BED carries no column names of its own, so a file whose columns are named otherwise needs `fields` to keep them.
- Entries must be pooled by chromosome and in order of their start, but may overlap and nest freely, as `write_entry` allows.

### Convert SAM to BAM

Not implemented yet.

```python
gwseq_io.convert_to_bam(input_path, output_path)
```

Parameters:
- `input_path` Path to input SAM file. May be gzipped.
- `output_path` Path to output file.

### Get genome chromosome sizes

```python
gwseq_io.get_chr_sizes(genome, ...)
```

Parameters:
- `genome` Genome name (eg, "mm10").
- `full` Include unplaced chromosomes if true. Not by default.

Returns a map of chromosome IDs and their sizes, sorted by chromosome ID as a string, so `chr10` comes before `chr2`. Genomes that are not bundled, and any call with `full`, are fetched from `api.genome.ucsc.edu` and cached for the process lifetime.


## Dev notes

### Project layout

```
gwseq_io/
├── CMakeLists.txt          # CMake build (nanobind module)
├── pyproject.toml          # PEP 517 build config (scikit-build-core)
├── docs/                   # Files formats specifications
├── tests/                  # Regression suite (pytest) — see tests/README.md
└── src/
    ├── cpp/
    │   ├── binding/        # nanobind bindings, one module per format
    │   ├── genomes.cpp     # Built-in genome chromosome sizes
    │   ├── bbi/            # bigWig / bigBed reader, writer and text converters
    │   ├── hic/            # HiC reader
    │   ├── bam/            # BAM reader (header, BAI index, records)
    │   └── util/           # C++17 utility library (see util/README.md)
    └── gwseq_io/
        └── __init__.py     # Python package entry-point
```

Every module under `src/cpp` is a self-contained `.cpp` guarded by `#pragma once` and `#include`d by `binding/binding.cpp`, so only `binding/binding.cpp` is compiled — see [src/cpp/util/README.md](src/cpp/util/README.md).

`binding/` holds one module per format — [bbi.cpp](src/cpp/binding/bbi.cpp), [bam.cpp](src/cpp/binding/bam.cpp) and [hic.cpp](src/cpp/binding/hic.cpp), each registering its own types through a `bind_*(m)` function — plus [util.cpp](src/cpp/binding/util.cpp) for the conversions they share and [binding.cpp](src/cpp/binding/binding.cpp) for the module itself, `open()` and the other free functions.

### Build from source

| Dependency | Version | Notes |
|---|---|---|
| Python | ≥ 3.9 | with the development headers (`Python.h`) |
| C++ compiler | C++17 | clang / gcc / MSVC |
| CMake | ≥ 3.15 | pulled from PyPI by scikit-build-core if missing |
| Ninja | any | same, on non-MSVC platforms |
| git | any | needed to fetch zlib-ng, and curl or zlib when those have to be built |
| nanobind | ≥ 2.11 | installed automatically as a build requirement |
| scikit-build-core | ≥ 1.0 | same |
| curl | any | fetched and built from source if not found |
| zlib-ng | 2.2.2 | fetched and built from source; use zlib as fallback |

nanobind and scikit-build-core are resolved by pip from `pyproject.toml`, so they never need to be installed by hand, and CMake and Ninja are added the same way when the system does not already provide a suitable version. What has to come from the OS is the compiler toolchain, the Python headers, and — to avoid a from-source build of the dependencies — the curl and zlib development packages.

A curl built from source uses the platform's native TLS backend on Windows (Schannel) and macOS (Secure Transport), but OpenSSL on Linux, which is why the OpenSSL headers are listed in the Linux commands below.

#### Prerequisites — Windows

Install [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/) 2019 or newer and tick the **Desktop development with C++** workload — it brings MSVC, the Windows SDK, CMake and Ninja. Then install [Python](https://www.python.org/downloads/) (the official installer ships the headers and libs) and [Git for Windows](https://git-scm.com/download/win).

With winget:

```powershell
winget install Microsoft.VisualStudio.2022.BuildTools --override "--wait --passive --add Microsoft.VisualStudio.Workload.VCTools --includeRecommended"
winget install Python.Python.3.12
winget install Git.Git
```

Windows has no system curl or zlib to link against, so both are cloned and built on the first configure — expect a noticeably longer initial build. Run the build from a *Developer Command Prompt* (or *Developer PowerShell*) so that MSVC is on the path.

#### Prerequisites — macOS

The Command Line Tools provide clang, git, and the system curl and zlib:

```bash
xcode-select --install
```

That is enough on its own, since pip fetches CMake and Ninja. To use the system CMake and a Python other than the one shipped with macOS:

```bash
brew install cmake ninja python
```

#### Prerequisites — Linux

```bash
# Fedora / RHEL
sudo dnf install gcc-c++ cmake ninja-build git \
                 python3-devel libcurl-devel zlib-devel openssl-devel

# Debian / Ubuntu
sudo apt install build-essential cmake ninja-build git \
                 python3-dev python3-venv libcurl4-openssl-dev zlib1g-dev libssl-dev

# Arch
sudo pacman -S --needed base-devel cmake ninja git python curl zlib openssl
```

Arch ships headers with the runtime packages, so `curl`, `zlib` and `openssl` cover both.

#### Build

```bash
# 1. Create and activate a virtual environment (recommended)
python -m venv .env
source .env/bin/activate       # Windows: .env\Scripts\activate

# 2. Build and install the package in editable mode, with the dev extras
pip install -e ".[dev]"
```

To build a wheel instead of installing in place:

```bash
pip install build
python -m build --wheel
```

### Tests

```bash
python -m pytest              # the fast tier, ~45 s
python -m pytest --slow       # everything, ~5 min
```

Run from the repository root. The fixtures the suite reads are ~1.5 GB of real
bigWig, bigBed, BAM and HiC files that are not in the repository; `tests/data.json`
names them, and a check whose fixture is absent is skipped rather than failed, so
a plain checkout still runs everything that writes its own input. See
[tests/README.md](tests/README.md).
