Metadata-Version: 2.4
Name: databusclient
Version: 1.0.0
Summary: A simple client for submitting, downloading, and deleting data on the DBpedia Databus
License: Apache-2.0 License
Author: DBpedia Association
Requires-Python: >=3.11,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: SPARQLWrapper (>=2.0.0,<3.0.0)
Requires-Dist: click (>=8.0.4,<9.0.0)
Requires-Dist: pyyaml (>=6.0.3,<7.0.0)
Requires-Dist: rdflib (>=7.2.1,<8.0.0)
Requires-Dist: requests (>=2.28.1,<3.0.0)
Requires-Dist: tqdm (>=4.42.1,<5.0.0)
Description-Content-Type: text/markdown

# Databus Python Client

Command-line and Python client for downloading and deploying datasets on DBpedia Databus.

## Table of Contents

- [Quickstart](#quickstart)
  - [Python](#python)
  - [Docker](#docker)
- [DBpedia](#dbpedia)
  - [Registration (Access Token)](#registration-access-token)
  - [DBpedia Knowledge Graphs](#dbpedia-knowledge-graphs)
    - [Download Live Fusion KG Dump (BUSL 1.1, registration needed)](#download-live-fusion-kg-dump-busl-11-registration-needed)
    - [Download Enriched Knowledge Graphs (BUSL 1.1, registration needed)](#download-enriched-knowledge-graphs-busl-11-registration-needed)
    - [Download DBpedia Wikipedia Knowledge Graphs (CC-BY-SA, no registration needed)](#download-dbpedia-wikipedia-knowledge-graphs-cc-by-sa-no-registration-needed)
    - [Download DBpedia Wikidata Knowledge Graphs (CC-BY-SA, no registration needed)](#download-dbpedia-wikidata-knowledge-graphs-cc-by-sa-no-registration-needed)
- [CLI Usage](#cli-usage)
  - [Download](#cli-download)
  - [Deploy](#cli-deploy)
  - [Delete](#cli-delete)
  - [Manifest](#cli-manifest)
  - [Workflow](#cli-workflow)
- [Module Usage](#module-usage)
- [Development & Contributing](#development--contributing)
  - [Linting](#linting)
  - [Testing](#testing)

## Quickstart

The client supports two main workflows: downloading datasets from the Databus and deploying datasets to the Databus. Below you can choose how to run it (Python or Docker), then follow the sections on [DBpedia downloads](#dbpedia-knowledge-graphs), [CLI usage](#cli-usage), or [module usage](#module-usage).

You can use either **Python** or **Docker**. Both methods support all client features. The Docker image is available at [dbpedia/databus-python-client](https://hub.docker.com/r/dbpedia/databus-python-client).

### Python

Requirements: [Python 3.11+](https://www.python.org/downloads/) and [pip](https://pip.pypa.io/en/stable/installation/).

Before using the client, install it via pip:

```bash
python3 -m pip install databusclient
```

Note: this repository prepares version `1.0.0`. If you previously installed `databusclient` via `pip` and observe different CLI behavior, upgrade to the latest release:

```bash
python3 -m pip install --upgrade databusclient==1.0.0
```

You can then use the client in the command line:

```bash
databusclient --help
databusclient [delete|deploy|download|manifest|workflow] --help
```

### Docker

Requirements: [Docker](https://docs.docker.com/get-docker/).

```bash
docker run --rm -v $(pwd):/data dbpedia/databus-python-client --help
```

The same Docker invocation pattern can be used for the commands documented in the [download](doc/cli-usage.md#cli-download), [deploy](doc/cli-usage.md#cli-deploy), [delete](doc/cli-usage.md#cli-delete), [manifest](doc/cli-usage.md#cli-manifest), and [workflow](doc/cli-usage.md#cli-workflow) sections.

## DBpedia

Commands to download the [DBpedia Knowledge Graphs](#dbpedia-knowledge-graphs) generated by Live Fusion. DBpedia Live Fusion publishes two kinds of KGs:

1. Open Core Knowledge Graphs under CC-BY-SA license, open with copyleft/share-alike, no registration needed.
2. Industry Knowledge Graphs under BUSL 1.1 license, unrestricted for research and experimentation, commercial license for productive use, free [registration](#registration-access-token) needed.

### Registration (Access Token)

To download BUSL 1.1 licensed datasets, you need to register and get an access token.

1. If you do not have a DBpedia Account yet (Forum/Databus), please register at [https://account.dbpedia.org](https://account.dbpedia.org).
2. Log in at [https://account.dbpedia.org](https://account.dbpedia.org) and create your token.
3. Save the token to a file, e.g. `vault-token.dat`.

### DBpedia Knowledge Graphs

#### Download Live Fusion KG Dump (BUSL 1.1, registration needed)

High-frequency, conflict-resolved knowledge graph that merges Live Wikipedia and Wikidata signals into a single, queryable dump for enterprise consumption. [More information](https://databus.dbpedia.org/dbpedia-enterprise/live-fusion-kg-dump).

```bash
databusclient download https://databus.dbpedia.org/dbpedia-enterprise/live-fusion-kg-dump --vault-token vault-token.dat
```

#### Download Enriched Knowledge Graphs (BUSL 1.1, registration needed)

**DBpedia Wikipedia Extraction Enriched**

DBpedia-based enrichment of structured Wikipedia extractions, currently EN DBpedia only. [More information](https://databus.dbpedia.org/dbpedia-enterprise/dbpedia-wikipedia-kg-enriched-dump).

```bash
databusclient download https://databus.dbpedia.org/dbpedia-enterprise/dbpedia-wikipedia-kg-enriched-dump --vault-token vault-token.dat
```

#### Download DBpedia Wikipedia Knowledge Graphs (CC-BY-SA, no registration needed)

Original extraction of structured Wikipedia data before enrichment. [More information](https://databus.dbpedia.org/dbpedia/dbpedia-wikipedia-kg-dump).

```bash
databusclient download https://databus.dbpedia.org/dbpedia/dbpedia-wikipedia-kg-dump
```

#### Download DBpedia Wikidata Knowledge Graphs (CC-BY-SA, no registration needed)

Original extraction of structured Wikidata data before enrichment. [More information](https://databus.dbpedia.org/dbpedia/dbpedia-wikidata-kg-dump).

```bash
databusclient download https://databus.dbpedia.org/dbpedia/dbpedia-wikidata-kg-dump
```

## CLI Usage

The command-line interface provides commands for downloading, deploying, and deleting datasets, as well as recording manifests and running declarative workflows. Detailed command documentation, options, examples, manifest operations, and workflow syntax are available in the [download](doc/cli-usage.md#cli-download), [deploy](doc/cli-usage.md#cli-deploy), [delete](doc/cli-usage.md#cli-delete), [manifest](doc/cli-usage.md#cli-manifest), and [workflow](doc/cli-usage.md#cli-workflow) sections.

<a id="cli-download"></a>
### Download

The `download` command retrieves Databus files, versions, artifacts, groups, collections, or SPARQL query results. It supports authentication, checksum validation, compression conversion, and RDF or tabular format conversion. See the [download docs](doc/cli-usage.md#cli-download).

<a id="cli-deploy"></a>
### Deploy

The `deploy` command publishes datasets using distribution arguments, metadata JSON files, or WebDAV/Nextcloud uploads. See the [deploy docs](doc/cli-usage.md#cli-deploy).

<a id="cli-delete"></a>
### Delete

The `delete` command removes Databus versions, artifacts, groups, or collections and provides dry-run and confirmation safeguards. See the [delete docs](doc/cli-usage.md#cli-delete).

<a id="cli-manifest"></a>
### Manifest

The manifest options record operation parameters, file outcomes, checksums, byte sizes, and execution summaries in JSON-LD. Manifests can also be replayed or summarized. See the [manifest docs](doc/cli-usage.md#cli-manifest).

<a id="cli-workflow"></a>
### Workflow

The `workflow` command runs declarative download, deploy, and delete pipelines from YAML files, with step chaining and per-step error handling. See the [workflow docs](doc/cli-usage.md#cli-workflow).

## Module Usage

The Python API exposes helpers for creating distributions and datasets and for deploying them programmatically. See the [module usage docs](doc/module-usage.md).

## Development & Contributing

Install development dependencies yourself or via [Poetry](https://python-poetry.org/):

```bash
poetry install --with dev
```

### Linting

The used linter is [Ruff](https://ruff.rs/). Ruff is configured in `pyproject.toml` and is enforced in CI (`.github/workflows/ruff.yml`).

For development, you can run linting locally with `ruff check .` and optionally auto-format with `ruff format .`.

To ensure compatibility with the `pyproject.toml` configured dependencies, run Ruff via Poetry:

```bash
# To check for linting issues:
poetry run ruff check .

# To auto-format code:
poetry run ruff format .
```

### Testing

When developing new features please make sure to add appropriate tests and ensure that all tests pass. Tests are under `tests/` and use [pytest](https://docs.pytest.org/en/7.4.x/) as test framework.

When fixing bugs or refactoring existing code, please make sure to add tests that cover the affected functionality. The current test coverage is very low, so any additional tests are highly appreciated.

To run tests locally, use:

```bash
pytest tests/
```

Or to ensure compatibility with the `pyproject.toml` configured dependencies, run pytest via Poetry:

```bash
poetry run pytest tests/
```

