Metadata-Version: 2.4
Name: xsd2rdf
Version: 2.0.0
Summary: Generates OWL, SHACL shapes and SKOS concepts from XML Schema (XSD) files
License-Expression: EUPL-1.2
License-File: LICENSE
Author: Mathias Vanden Auweele
Author-email: mathias@matdata.eu
Requires-Python: >=3.12
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: File Formats
Requires-Dist: pyoxigraph (>=0.4)
Requires-Dist: pyshacl (>0.30.0)
Requires-Dist: rdflib (>=7.0)
Requires-Dist: xmlschema (>=4.0)
Project-URL: Changelog, https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/xsd2rdf/-/blob/main/CHANGELOG.md
Project-URL: Repository, https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/xsd2rdf
Description-Content-Type: text/markdown

# XSD2RDF

[![License](https://img.shields.io/badge/license-EUPL_1.2-blue)](http://data.europa.eu/eli/dec_impl/2017/863/oj)

A tool to convert XML Schema (XSD) files into various RDF formats (SHACL, OWL, SKOS) with integrated validation capabilities.

## Overview

XSD2RDF allows you to convert XML Schema definitions into:

- **SHACL** (Shapes Constraint Language) for RDF data validation
- **OWL** (Web Ontology Language) for ontology representation
- **SKOS** (Simple Knowledge Organization System) for concept schemes and taxonomies

## Features

- Convert XSD to SHACL, OWL, and SKOS based on integrated principles
- SHACL shape constraints are linked to SKOS concept schemes when applicable
- Handle complex XSD structures (choices, unions, complex types, enumerations, etc.)
- SHACL shapes are validated according to SHACL-SHACL

This repository also includes a validation script to check RDF data against the generated SHACL shapes and SKOS concepts.

## Current usage

This tool is currently used by [ERA](https://www.era.europa.eu/) for the purpose of generating the [telematics ontology](https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/era-telematics-ontology-pipeline) in preparation to be merged with [ERA ontology](https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/era-ontology). 

## Installation

### From PyPI

```bash
pip install xsd2rdf
```

### From Source

```bash
git clone https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/xsd2rdf.git
cd xsd2rdf
python -m pip install poetry
poetry install
```

## Basic Usage

Convert an XSD file to all RDF formats (SHACL, OWL, SKOS):

```bash
xsd2rdf convert path/to/schema.xsd
```

This generates the following files:

- `schema.xsd.shape.ttl` (SHACL shapes)
- `schema.xsd.owl.ttl` (OWL ontology)
- `schema.xsd.skos.<Name>.ttl` (SKOS concept schemes, one file per enumeration)

### From Python

```python
from xsd2rdf import convert

result = convert("path/to/schema.xsd", namespaced_concepts=True)
result.shacl, result.owl  # the graphs, nothing is written yet
result.skos               # one graph per concept scheme
result.write("out")       # writes the same files as the command line
```

The settings are the fields of `xsd2rdf.Options`; pass them as keywords or as
`convert(path, Options(...))`. Progress and warnings go to the `xsd2rdf` logger.

## Command Line Parameters

`xsd2rdf convert PATH...` converts XSD files, and the `*.xsd` files of any folder
given. Without a subcommand the arguments are those of `convert`, so
`xsd2rdf -x schema.xsd` still works. The 1.x spellings of the flags
(`--XSD_FILE`, `-ssv`, `--skipShaclValidation`, `-pkqi` and the like) are gone
in 2.0; use the names below.

- `-x, --xsd-file`: XSD file to be converted
- `-f, --folder`: Folder containing non-related XSD files to be converted
- `-o, --output-dir`: Output directory for generated files (default: same as XSD file)
- `-a, --abbreviations-file`: File containing custom abbreviations, one per line
- `-d, --debug`: Enable debug output
- `-q, --quiet`: Only print warnings and errors
- `-n, --namespaced-concepts`: Use namespaced IRIs for SKOS concepts
- `-u, --namespace`: Override the XSD targetNamespace for generated SHACL, OWL and SKOS resources
- `-p, --has-prefix-for-properties`: Add `has` as a prefix for generated property IRIs
- `-s, --skip-shacl-validation`: Skip the check of the generated shapes against SHACL-SHACL (offline, with the copy that pyshacl ships)
- `--date YYYY-MM-DD`: Date written as `dct:created`, `dct:issued` and `dct:modified`. Without it the day of `SOURCE_DATE_EPOCH` is used if that variable is set, otherwise today.
- `--strict`: Stop with an error when the generated shapes are not well-formed SHACL
- `-k, --property-kind-qualified-iris`: Append `DP` or `OP` to every property IRI according to its OWL property kind
- `--no-functional-properties`: Do not declare `owl:FunctionalProperty` for properties whose shapes all have `sh:maxCount 1`. The SHACL output is unchanged.
- `--simple-type-subproperties`: Declare a property for each named simpleType and make the properties of the elements typed by it `rdfs:subPropertyOf` that property, as 1.x did. The SHACL output is unchanged.
- `--no-pruning`: Keep every component of included and imported schemas. By default only the components the main schema uses (directly or indirectly) are converted; use this option to convert a library schema on its own.
- `--source-namespaces`: Mint the IRIs of classes, properties and concept schemes from an imported schema in that schema's `targetNamespace`. By default everything goes into the main schema's namespace.
- `--namespace-map XSD_NS=IRI_BASE`: Mint the IRIs of components from the XSD namespace `XSD_NS` under `IRI_BASE` instead. Repeat the option for several namespaces.

Either -x or -f must be specified, but not both. If both are specified, -x takes precedence.

### abbreviations-file

Provide a custom abbreviations file to preserve domain-specific acronyms when labels are generated from camelCase or PascalCase names.

```bash
xsd2rdf convert path/to/schema.xsd -a path/to/abbreviations.txt
```

Effect on output:

- Labels and human-readable names keep the listed abbreviations in uppercase.
- Generated SHACL, OWL, and SKOS resources use the same IRIs as before, but the textual labels become more accurate.

### debug

Enable verbose logging while converting schemas.

```bash
xsd2rdf convert path/to/schema.xsd -d
```

Effect on output:

- Prints additional conversion details to the console.
- Does not change the generated SHACL, OWL, or SKOS files.

### namespaced-concepts

Generate SKOS concept IRIs in a hierarchical structure instead of a flat one.

```bash
xsd2rdf convert path/to/schema.xsd --namespaced-concepts
```

Effect on output:

- SKOS concepts become `targetnamespace/concepts/conceptschemename/conceptname` instead of `targetnamespace/concepts/conceptschemename_conceptname`.
- The generated SKOS files still represent the same concepts, but the IRIs are easier to group by concept scheme.

### namespace

Override the XSD target namespace used for generated RDF resources.

```bash
xsd2rdf convert path/to/schema.xsd --namespace http://data.europa.eu/949/
```

Effect on output:

- SHACL, OWL, and SKOS IRIs are emitted under the provided namespace instead of the schema target namespace.
- This is useful when generated resources must align with an existing vocabulary namespace.

### has-prefix-for-properties

Add `has` as a prefix to generated property IRIs.

```bash
xsd2rdf convert path/to/schema.xsd --has-prefix-for-properties
```

Effect on output:

- Property IRIs are prefixed with `has` where applicable.
- The change applies to generated SHACL paths and OWL property IRIs, making property names more explicit.

### skip-shacl-validation

Skip the SHACL-SHACL validation step during generation.

```bash
xsd2rdf convert path/to/schema.xsd -s
```

Effect on output:

- The SHACL validation phase is not run.
- Generation completes faster, but invalid SHACL output is not checked before files are written.

### property-kind-qualified-iris

Append a property-kind suffix to every generated property IRI so datatype and object properties remain distinct.

```bash
xsd2rdf convert path/to/schema.xsd \
  --property-kind-qualified-iris
```

Effect on output:

- Datatype properties receive the `DP` suffix and object properties receive the `OP` suffix.
- SHACL paths and OWL property IRIs no longer collide when the same local name is used for different property kinds.

This is a global opt-in mode. It changes all generated property IRIs, not only
IRIs that would otherwise be used as both datatype and object properties. The
default mode remains unchanged for backwards compatibility.

For example, consider two complex types that both use an `author` element in 
the same xsd file, but with a different structure. A `Book` has an author name 
(simple string), while an `Article` references an author as a complete person object:

```xml
<xsd:complexType name="Book">
  <xsd:sequence>
    <xsd:element name="title" type="xsd:string"/>
    <xsd:element name="author" type="xsd:string"/>
  </xsd:sequence>
</xsd:complexType>

<xsd:complexType name="Article">
  <xsd:sequence>
    <xsd:element name="title" type="xsd:string"/>
    <xsd:element name="author" type="PersonType"/>
  </xsd:sequence>
</xsd:complexType>

<xsd:complexType name="PersonType">
  <xsd:sequence>
    <xsd:element name="firstName" type="xsd:string"/>
    <xsd:element name="lastName" type="xsd:string"/>
  </xsd:sequence>
</xsd:complexType>
```

With property-kind qualification enabled, their SHACL paths and OWL property
IRIs are distinct:

```turtle
sh:path ex:authorDP .
ex:authorDP a owl:DatatypeProperty .

sh:path ex:authorOP .
ex:authorOP a owl:ObjectProperty .
```

## Tests

```bash
poetry install
poetry run pytest
```

The mapping rules are covered by the fixture suite in [`tests/fixtures`](tests/fixtures/README.md):
one folder per rule with a minimal XSD, the golden SHACL/OWL/SKOS output and
instance data that must (not) conform to the generated shapes. After an
intended change of the output, regenerate the golden files with
`poetry run pytest tests/test_fixtures.py --update-golden` and review the diff.
The test job runs on every merge request, together with a lint job:
`ruff check src tests docs` and `mypy` must pass (both are configured in `pyproject.toml`).

The fixtures are also published as a documentation site on GitLab Pages, built
by the `pages` job on `main`: one page per case with the XSD, the golden output,
the instance data and UML-like class diagrams of the XSD source, the SHACL shapes
and the OWL ontology. To build it locally (Mermaid then loads from a CDN):

```bash
poetry run python docs/build_site.py public
```

The real TAF and TAP schemas (see [`tests/corpus/corpus.toml`](tests/corpus/corpus.toml))
are converted by a separate `corpus` job, from a pipeline schedule on `main`. It
converts each schema twice, fails when a conversion fails or the two runs differ,
and publishes a report with run times and counts (and what changed since the
previous run) as job artifact. When neither the converter nor the corpus
repositories changed since the previous run, the job is skipped and the previous
report is kept. Locally:

```bash
poetry run python tests/corpus/run_corpus.py --corpus-dir corpus --out corpus-report
```

## Release new version CI/CD

The CI/CD script will automatically push a new version of the library to PyPi when a new Gitlab tag is created. As the tag name, use semantic versioning with 3 digits, examples: `v1.2.3`. 

The tag sets the version in `pyproject.toml` for the build. Before tagging, move the changes under a dated heading in [CHANGELOG.md](CHANGELOG.md).

## Validating data

Check RDF data against the shapes of a converted schema:

```bash
xsd2rdf convert path/to/schema.xsd -o out
xsd2rdf validate path/to/schema.xsd data.ttl -o out
```

`validate` reads `out/schema.xsd.shape.ttl` and adds the SKOS concept schemes
(`out/schema.xsd.skos.*.ttl`) to the data, so concept-valued properties can be
checked. Several data files may be given, in any RDF format rdflib reads. The
exit code is 0 when the data conforms, 1 when it does not and 2 when a file is
missing. `--inference rdfs` applies RDFS inference to the data first.

## Wiki Pages

Some [wiki pages](https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/xsd2rdf/-/wikis/home) explain the inner workings of the convertor and the list of 
mapping rules implemented within the tool.

## License

[EUPL 1.2](http://data.europa.eu/eli/dec_impl/2017/863/oj)

