Metadata-Version: 2.4
Name: loadfile
Version: 0.1.0
Summary: Load tabular data from any storage backend (GCS, S3, Azure, local) and any common format into a pandas DataFrame with a single call.
Project-URL: Homepage, https://github.com/VedantAndhale/loadfile
Project-URL: Repository, https://github.com/VedantAndhale/loadfile
Project-URL: Issues, https://github.com/VedantAndhale/loadfile/issues
Author: loadfile contributors
License: MIT
License-File: LICENSE
Keywords: azure,csv,data,fsspec,gcs,loader,pandas,parquet,s3
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Requires-Dist: fsspec>=2023.1.0
Requires-Dist: pandas>=1.5
Requires-Dist: pyarrow>=10.0
Provides-Extra: all
Requires-Dist: adlfs; extra == 'all'
Requires-Dist: gcsfs; extra == 'all'
Requires-Dist: openpyxl>=3.1; extra == 'all'
Requires-Dist: s3fs; extra == 'all'
Provides-Extra: azure
Requires-Dist: adlfs; extra == 'azure'
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: openpyxl>=3.1; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Provides-Extra: excel
Requires-Dist: openpyxl>=3.1; extra == 'excel'
Provides-Extra: gcs
Requires-Dist: gcsfs; extra == 'gcs'
Provides-Extra: s3
Requires-Dist: s3fs; extra == 's3'
Description-Content-Type: text/markdown

# loadfile

Load tabular data from **any storage backend** (Google Cloud Storage, AWS S3,
Azure, local disk, …) and **any common format** into a pandas DataFrame with a
single call.

```python
from loadfile import load

df = load("gs://my-bucket/data.parquet")
df = load("data/local.csv")
df = load("s3://bucket/export.json", storage_options={"anon": True})
```

The storage backend is chosen automatically from the URL prefix (powered by
[`fsspec`](https://filesystem-spec.readthedocs.io/)). The file format is detected
from the extension (or forced with `format=`).

> The function is also exported as `load_data` (an alias of `load`) for
> compatibility with the original helper.

## Install

```bash
pip install loadfile            # core: pandas + fsspec + pyarrow (local files)
pip install "loadfile[gcs]"     # Google Cloud Storage (gcsfs)
pip install "loadfile[s3]"      # AWS S3 (s3fs)
pip install "loadfile[azure]"   # Azure (adlfs)
pip install "loadfile[excel]"   # .xlsx support (openpyxl)
pip install "loadfile[all]"     # everything
```

## Supported formats

| Format            | Extensions                          |
|-------------------|-------------------------------------|
| Parquet           | `.parquet`                          |
| CSV               | `.csv` (and `.csv.gz`, `.csv.bz2`…) |
| JSON / JSON Lines | `.json`, `.jsonl`, `.ndjson`        |
| Excel             | `.xlsx`, `.xls`                     |
| Feather           | `.feather`                          |
| ZIP               | `.zip` (wrapping any of the above)  |

## Usage

### Format override and reader options

Any extra keyword arguments are forwarded to the underlying pandas reader:

```python
load("export.tsv", format="csv", sep="\t")
load("book.xlsx", sheet_name="Sheet2")
load("big.csv", usecols=["id", "value"])
```

### Credentials

Pass backend options via `storage_options` (forwarded to fsspec):

```python
load("gs://bucket/data.parquet", storage_options={"token": "/path/key.json"})
load("s3://bucket/data.csv", storage_options={"key": "...", "secret": "..."})
```

### ZIP archives

A `.zip` may contain any mix of supported formats. Each member is loaded with the
reader matching its own extension.

```python
# Single data file in the zip -> a DataFrame
df = load("archive.zip")

# Multiple data files -> a dict {member_name: DataFrame}
frames = load("archive.zip")               # {"a.csv": ..., "b.parquet": ...}

# Pick one member -> a DataFrame
df = load("archive.zip", filename="a.csv")

# Pick a subset -> a dict of just those members
frames = load("archive.zip", filename=["a.csv", "b.parquet"])
```

Non-data members (e.g. `readme.txt`) and nested zips are ignored. A zip with no
supported files raises `EmptyArchiveError`; a missing requested member raises
`MemberNotFoundError`.

## Development

```bash
pip install -e ".[dev,all]"
pytest
```

## License

MIT
