Metadata-Version: 2.4
Name: sci-etl-cli
Version: 0.4.0
Summary: Command-line tool for running sci-etl-core extraction pipelines from a YAML file.
License-Expression: MIT
Project-URL: Homepage, https://github.com/xueromll/sci-etl-cli
Project-URL: Documentation, https://xueromll.github.io/sci-etl-core/latest/cli/
Project-URL: Repository, https://github.com/xueromll/sci-etl-cli
Project-URL: Issues, https://github.com/xueromll/sci-etl-cli/issues
Keywords: etl,cli,arxiv,llm,text-mining,scientific-literature
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Text Processing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: sci-etl-core[arxiv,async,config,html,llm,pdf,xml]<0.8,>=0.7.0
Requires-Dist: click>=8.2
Requires-Dist: rich>=13.7
Requires-Dist: pydantic>=2
Requires-Dist: python-dotenv
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Provides-Extra: lint
Requires-Dist: ruff; extra == "lint"
Requires-Dist: mypy; extra == "lint"
Requires-Dist: types-PyYAML; extra == "lint"
Provides-Extra: docs
Requires-Dist: mkdocs<2,>=1.6; extra == "docs"
Requires-Dist: mkdocs-material<10,>=9.7; extra == "docs"
Requires-Dist: mkdocs-click<1,>=0.9; extra == "docs"
Dynamic: license-file

# sci-etl-cli

`sci-etl` runs [sci-etl-core](https://github.com/xueromll/sci-etl-core)
extraction pipelines from a single YAML file, with no wiring code. It searches
arXiv, PubMed, Semantic Scholar, or OpenAlex, asks an LLM which papers are
relevant, extracts structured entities from their full text, and writes them
to a CSV, one row per entity tagged with the paper it came from. Runs resume where the last one stopped, and `forget` hands
chosen papers back for another pass.

**Documentation: https://xueromll.github.io/sci-etl-core/latest/cli/**

```text
$ sci-etl init hot-jupiters
$ sci-etl validate hot-jupiters/config.yaml
$ sci-etl search hot-jupiters/config.yaml --limit 5
$ sci-etl run hot-jupiters/config.yaml --limit 20
$ sci-etl status hot-jupiters/config.yaml
```

## Installation

Python 3.11 or newer is required.

```bash
pip install sci-etl-cli
```

This installs the `sci-etl` command along with
`sci-etl-core[config,async,arxiv,html,llm,pdf,xml]`, `click`, and `rich`, so
every source works out of the box. `pipx install sci-etl-cli` keeps it in an environment of
its own.

## Quick start

1. `sci-etl init hot-jupiters` writes a working example project.
2. Copy `.env.example` to `.env` beside the config and set `LLM_API_KEY`.
3. `sci-etl validate hot-jupiters/config.yaml` checks the config, prompts, key,
   and packages.
4. `sci-etl search hot-jupiters/config.yaml` previews matching papers without
   calling the LLM.
5. `sci-etl run hot-jupiters/config.yaml` exports entities to `out/planets.csv`.
   Run it again to continue where it stopped.

## Documentation

| Topic | Where |
|-------|-------|
| A first project, step by step | [Quick start](https://xueromll.github.io/sci-etl-core/latest/cli/quick-start/) |
| Every command and option, stopping, resuming, token usage | [Commands](https://xueromll.github.io/sci-etl-core/latest/cli/commands/) |
| Every config key and its default | [Config file](https://xueromll.github.io/sci-etl-core/latest/cli/config-file/) |
| Domain rules written in Python | [Plug-ins](https://xueromll.github.io/sci-etl-core/latest/cli/plugins/) |
| What each exit code means | [Exit codes](https://xueromll.github.io/sci-etl-core/latest/cli/exit-codes/) |

## Development

```bash
git clone https://github.com/xueromll/sci-etl-cli.git
cd sci-etl-cli
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -e ".[dev,lint]"
pytest --cov=sci_etl_cli --cov-report=term-missing
ruff check .
mypy
```

To work against a local sci-etl-core checkout, install it first with
`pip install -e "../sci-etl-core[config,async,arxiv,html,llm,pdf,xml]"`.

The suite runs offline: every source is served by an `httpx.MockTransport` and
the LLM by a scripted client, so `run` is exercised end to end against real CSV
and state files. Coverage must stay at 100%.

### Documentation

The pages under `docs/` are published as the CLI section of the
[sci-etl-core site](https://xueromll.github.io/sci-etl-core/), which reads this
repository's `docs/` folder and the `nav` in `mkdocs.yml`. Preview them on
their own with:

```bash
pip install -e ".[docs]"
mkdocs serve
```

Link to sci-etl-core pages with absolute URLs, since the pages are also built
on their own. `mkdocs build --strict` runs in CI. Publishing a release asks
sci-etl-core to rebuild the site when the `DOCS_DISPATCH_TOKEN` secret is set;
otherwise run sci-etl-core's Docs workflow by hand.

## License

Released under the MIT License. See [LICENSE](LICENSE).
