Metadata-Version: 2.4
Name: polars-corpus
Version: 0.2.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Text Processing
Classifier: Topic :: Text Processing :: General
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Typing :: Typed
Requires-Dist: polars>=1.36,<2.0
Requires-Dist: lark>=1.2
Requires-Dist: nltk>=3.9
Requires-Dist: anywidget>=0.9
Requires-Dist: sentence-transformers ; extra == 'embeddings'
Requires-Dist: model2vec ; extra == 'embeddings'
Requires-Dist: umap-learn ; extra == 'embeddings'
Requires-Dist: hdbscan ; extra == 'embeddings'
Requires-Dist: great-tables ; extra == 'examples'
Requires-Dist: scipy ; extra == 'examples'
Requires-Dist: matplotlib ; extra == 'examples'
Requires-Dist: seaborn ; extra == 'examples'
Requires-Dist: lxml ; extra == 'examples'
Requires-Dist: pyarrow>=14 ; extra == 'examples'
Requires-Dist: statsmodels ; extra == 'examples'
Requires-Dist: adjusttext ; extra == 'examples'
Provides-Extra: embeddings
Provides-Extra: examples
License-File: LICENSE.txt
Summary: Text processing plugins for polars
Author-email: Rob Malouf <rmalouf@sdsu.edu>
License-Expression: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Issues, https://github.com/rmalouf/polars_corpus/issues
Project-URL: Repository, https://github.com/rmalouf/polars_corpus

# polars-corpus

Corpus linguistics tools for Polars DataFrames.

> **⚠️ Early Stage**: This project is under active development. The API and query language **will** change as we refine the design. Feedback and contributions are welcome!

## Installation

```bash
pip install polars-corpus
```

## Usage

```python
import polars as pl
import polars_corpus as plc

# Load your corpus as a DataFrame with token and POS columns
corpus = pl.read_parquet("brown.parquet")

# Search for patterns
results = plc.search(corpus, "small _{SUBST}", pos_column="tag")

# Generate concordances
results.concordance("token", window=5)

# Or use the .corpus namespace
corpus.corpus.search("america").concordance("token", window=2)
```

## Query Syntax

Supports two query languages:
- **Simple** (BNCweb-style): `small _{SUBST}`, `( small | little ) _N*`
- **CQP**: Standard CQP syntax for corpus queries

## Features

- Pattern search and concordancing
- Frequency distributions and collocations
- Keywords and statistical measures
- Handles 100M+ word corpora on 16GB memory

## Examples

See the `examples/` directory for notebooks covering:
- Concordance generation
- Collocation analysis
- Frequency analysis
- Keyword extraction
- Stylometry

## Requirements

- Python 3.11+
- Polars 1.35+

## License

MIT

