Metadata-Version: 2.4
Name: scta-scikit
Version: 0.2.0
Summary: Texts, search, statistics and similarity for the Scholastic Commentaries and Texts Archive (SCTA), with a downloadable corpus and models.
Author-email: "Jeffrey C. Witt" <jeffreycwitt@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/scta/scta-scikit
Project-URL: Documentation, https://github.com/scta/scta-scikit/blob/main/readme_cli.md
Project-URL: Corpus, https://huggingface.co/datasets/scta/scta-corpus
Project-URL: SCTA, https://scta.info
Keywords: latin,medieval,scholastic,corpus,nlp,scta
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=2.0
Requires-Dist: numpy>=1.26
Requires-Dist: scipy>=1.11
Requires-Dist: scikit-learn>=1.3
Requires-Dist: gensim>=4.3
Requires-Dist: joblib>=1.3
Requires-Dist: requests>=2.31
Requires-Dist: SPARQLWrapper>=2.0
Requires-Dist: huggingface_hub>=0.24
Provides-Extra: build
Requires-Dist: stanza>=1.10; extra == "build"
Requires-Dist: torch>=2.4; extra == "build"
Requires-Dist: cltk>=2.0; extra == "build"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: PyYAML>=6; extra == "dev"
Dynamic: license-file

# scta-scikit

Texts, search, statistics and similarity for the
[Scholastic Commentaries and Texts Archive](https://scta.info) (SCTA): the
medieval Latin commentaries, questions and treatises the SCTA publishes, as
documents you can load at any level of their structure (whole work, book,
question, paragraph, ...), search, compare and model.

## Install

```bash
pip install scta-scikit
```

Python 3.12 or newer. (Maintainers who lemmatize the corpus: `pip install "scta-scikit[build]"`.)

## Get the data

The corpus and models are published separately from the package, on Hugging Face:
[scta/scta-corpus](https://huggingface.co/datasets/scta/scta-corpus),
[scta/scta-doc2vec](https://huggingface.co/scta/scta-doc2vec) and
[scta/scta-para-tfidf](https://huggingface.co/scta/scta-para-tfidf). Download them once:

```python
import scta_scikit as sc

sc.download()                                   # the latest corpus (all texts, with lemmatized versions)
sc.download("2026.10")                          # a given version
sc.download(texts=["augustinedetrinitate"])     # only some texts
sc.download(doc2vec=True)                       # with the Doc2Vec models and the corpus they were trained on
sc.download(para_tfidf=True)                    # with the paragraph TF-IDF (for `scta match`)
```

or from the shell: `scta download`, `scta download 2026.10 --doc2vec`, `scta download --para-tfidf`, `scta download --texts augustinedetrinitate`.

The download holds the texts and their structure; the package builds the
document groupings, the question-title TF-IDF and the citation similarities
from them on your machine (about a minute for everything, or on first use).
Models come with the corpus version they were built from, unless you ask for
another. The data goes to `~/scta-data` unless you choose another folder:

```python
sc.set_data_dir("/path/to/data", save=True)    # remembered for future sessions
```

(`scta config --set-data-dir PATH`, or the `SCTA_DATA` environment variable.)

## Use

```python
import scta_scikit as sc

# documents at a level of a text's structure (here: books of De trinitate)
docs, texts, idx2label, label2idx = sc.load_docs("augustinedetrinitate", level=2)

# one document's text by its SCTA resource id
sc.get_document_text("adt-l1")

# keyword in context
corpus = sc.load_docs_by_toplevel(structure="structureBlock", toplevel="augustinedetrinitate")
sc.display_kwic(sc.kwic_search(corpus, "trinitas"))
```

From the shell (`scta --help`):

```bash
scta get adt-l1                       # text of a resource
scta search trinitas --toplevel augustinedetrinitate
scta stats --toplevel augustinedetrinitate --level 2
scta sim adt-l1                       # most similar documents (Doc2Vec)
scta sim adt-l1 --citations           # by shared citations
scta match "omnis doctrina vel rerum est vel signorum"   # paragraphs quoting or borrowing a passage
```

See [readme_cli.md](https://github.com/scta/scta-scikit/blob/main/readme_cli.md) for every command.

## License

The code is MIT licensed. The SCTA texts, and the corpus and models published
from them, are published under the
[Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
license.

## For maintainers

Building the corpus from the SCTA servers, lemmatizing it, training the models,
running all of it on the cluster, and publishing a release are described in
[readme_cli.md](https://github.com/scta/scta-scikit/blob/main/readme_cli.md) (`scta update`, `scta model build`, `scta publish`)
and [CLAUDE.md](https://github.com/scta/scta-scikit/blob/main/CLAUDE.md).
