Metadata-Version: 2.4
Name: nbase-converter
Version: 0.0.6
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Healthcare Industry
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Typing :: Typed
Requires-Dist: orjson>=3.12,<4
Requires-Dist: pandas>=3.0.3
Requires-Dist: polars>=1.41.2
Requires-Dist: pyarrow>=24.0.0
Requires-Dist: pyreadstat>=1.3.6,<2
License-File: licenses/nbase-converter-MIT.txt
License-File: licenses/python-third-party-notices.txt
License-File: licenses/sas7bdat-MIT.txt
License-File: licenses/sas_xport-MIT.txt
Summary: Fast Arrow-stream conversion for SAS7BDAT, SAS XPORT, and Parquet datasets
Keywords: sas,sas7bdat,xpt,parquet,arrow,clinical-research
Home-Page: https://bitbucket.org/nimort/nbase-converter
Author: Nimble Clinical Research LLC
Maintainer: Nimble Clinical Research LLC
License-Expression: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://bitbucket.org/nimort/nbase-converter/src/main/CHANGELOG.md
Project-URL: Issues, https://bitbucket.org/nimort/nbase-converter/issues
Project-URL: Repository, https://bitbucket.org/nimort/nbase-converter

# nbase-converter

`nbase-converter` is a native Rust/Arrow Python package for reading, writing, and converting
SAS7BDAT, SAS XPORT v5/v8, and Parquet datasets.

Version `0.0.6` is the next alpha release, not yet published. The production SAS7BDAT writer's
previous Windows SAS 9.4 validation is recorded in the repository release notes. Validation of
the expanded encoding support remains pending; stable `1.0.0` compatibility is not yet declared.

## Install

The command below applies after `0.0.6` is published.
Before publication, install the wheel built from this checkout for local validation.

```sh
python -m pip install nbase-converter==0.0.6
```

Python 3.11 through 3.14 is supported on 64-bit Linux, macOS, and Windows.

## Example

```python
from nbase_converter import convert_dataset, filter_field, read_dataset, write_dataset

result = read_dataset(
    "input.xpt",
    columns=["USUBJID", "AGE"],
    filter=filter_field("AGE") >= 18,
    output="pandas",
)

write_dataset(
    result.data,
    "output.parquet",
    target_format="parquet",
    overwrite=True,
)

convert_dataset(
    "source.parquet",
    "submission.xpt",
    target_format="xpt8",
    metadata={
        "dataset_label": "Adverse Events",
        "columns": {
            "ASTDT": {
                "label": "Analysis Start Date",
                "sas": {"format": "IS8601DA", "length": 8},
            }
        },
    },
)
```

The public dataset operations are:

- `read_dataset`: materialized, streaming, or metadata-only reads with projection, row bounds,
  encoding selection, and structured filters.
- `write_dataset`: Arrow-compatible writes to Parquet, XPT v5/v8, or SAS7BDAT.
- `convert_dataset`: bounded-memory full-file conversion with optional supplemental metadata.

For writes and conversions, explicit metadata overrides recognized input metadata while omitted
attributes remain preserved. Supported supplemental attributes are the dataset label and each
column's label, SAS format, informat, and storage length. For string attributes, `""` explicitly
clears an inherited value; whitespace-only labels remain literal labels.

## Text encoding

`write_dataset(..., encoding=None)` and `convert_dataset(..., encoding=None)` leave output
encoding unspecified. The native engine selects UTF-8 when the option is omitted or `None`.
XPT v5/v8 and SAS7BDAT also accept `Windows-1252`, `ISO-8859-1`, `ASCII`, `Shift-JIS`, and `Big5`.
Names are case-insensitive; common aliases such as `utf8`, `cp1252`, `latin1`, `us-ascii`, `sjis`,
`cp932`, and `big-5` are accepted. ISO-8859-1 and ASCII use their literal character sets.
Parquet output always uses UTF-8 and rejects other requested encodings. UTF-16 is unsupported
for these dataset targets. Unrepresentable characters and byte-width overflows fail without
replacing text, truncating values, or publishing a partial target.

`read_dataset(encoding=...)` selects input decoding. For conversion, input decoding is selected
separately with `source_encoding`; **existing conversion calls using `encoding` to decode the
source must migrate to `source_encoding`**. `encoding` now selects output encoding.

```python
convert_dataset("input.parquet", "output.xpt", target_format="xpt8")
convert_dataset("input.parquet", "legacy.xpt", target_format="xpt8", encoding="Windows-1252")
convert_dataset(
    "legacy.xpt", "output.sas7bdat", target_format="sas7bdat",
    source_encoding="Windows-1252", encoding=None,
)
```

Reports contain the resolved output encoding in `encodings.target`, including `UTF-8` when
`encoding=None`. `encodings.selected` remains the source decoding override. XPT files do not
reliably declare their encoding: recipients must select the matching reader or SAS session
encoding. The package's default XPT reader retains UTF-8 decoding with Windows-1252 fallback;
use an explicit input encoding for other legacy XPT encodings.

The R API uses `encoding=NULL` for output and `source_encoding=NULL` for conversion input.
The CLI uses `--encoding` for output and `--source-encoding` for input; omitted output encoding
is resolved by the engine to UTF-8.

Structured filters are created with `filter_field`, combined with `filter_all`, `filter_any`,
or `filter_not`, and evaluated by these leaf helpers:

- comparisons: `filter_equal`, `filter_not_equal`, `filter_less_than`,
  `filter_less_than_or_equal`, `filter_greater_than`, and `filter_greater_than_or_equal`;
- collections and text: `filter_in`, `filter_between`, `filter_contains`, `filter_starts_with`,
  and `filter_ends_with`;
- missing values: `filter_is_missing`, `filter_is_not_missing`, `filter_is_null`, and
  `filter_is_not_null`.

Python comparison operators are also available on `filter_field(...)`. Public result and metadata
contracts ship as `TypedDict` definitions, and the wheel is marked as a typed package.

The default cross-format type policy accepts Float64 and UTF-8 text. Non-null IEEE NaNs are
written as standard SAS numeric missing values and reported as coercions. Local filesystem paths
are the only supported storage interface in this release.

Source, issues, release notes, and the full contract are available from the
[Bitbucket repository](https://bitbucket.org/nimort/nbase-converter).

