Metadata-Version: 2.4
Name: jsonschema2rdf
Version: 1.0.0
Summary: Generates OWL, SHACL shapes and SKOS concepts from JSON Schema, with the mapping model of xsd2rdf
License-Expression: EUPL-1.2
License-File: LICENSE
Author: Mathias Vanden Auweele
Author-email: mathias@matdata.eu
Requires-Python: >=3.12
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: File Formats
Requires-Dist: pyyaml (>=6.0)
Requires-Dist: xsd2rdf (>=2.1.0,<3)
Project-URL: Changelog, https://gitlab.com/era-europa-eu/private/OPDU/SOT/era-ontology/telematics/jsonschema2rdf/-/blob/main/CHANGELOG.md
Project-URL: Homepage, https://gitlab.com/era-europa-eu/private/OPDU/SOT/era-ontology/telematics/jsonschema2rdf
Project-URL: Repository, https://gitlab.com/era-europa-eu/private/OPDU/SOT/era-ontology/telematics/jsonschema2rdf
Description-Content-Type: text/markdown

# jsonschema2rdf

[![License](https://img.shields.io/badge/license-EUPL_1.2-blue)](http://data.europa.eu/eli/dec_impl/2017/863/oj)

Converts JSON Schema into SHACL shapes, an OWL ontology and SKOS concept
schemes, with the mapping model of [xsd2rdf](https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/xsd2rdf).

jsonschema2rdf reads a JSON Schema and fills the SHACL shape model of xsd2rdf
(`xsd2rdf.shapes`) as xsd2rdf fills it for an XSD: the same shape IRIs, labels,
constraints and SKOS schemes. Everything after that is xsd2rdf's own code: the
OWL ontology is read from the shapes (`xsd2rdf.model`), the `-k` naming policy
qualifies the property IRIs, the SKOS writer makes one file per concept scheme,
the shapes are checked against SHACL-SHACL and the result has the file names of
xsd2rdf. So a JSON Schema and an XSD that describe the same thing give the same
kind of output, and the options mean the same.

openapi2rdf builds on this package for the schemas of an OpenAPI document.

## Installation

Python 3.12 or later.

```bash
pip install jsonschema2rdf
```

From source:

```bash
python -m pip install poetry
poetry install
```

jsonschema2rdf depends on `xsd2rdf>=2.1.0,<3` and PyYAML.

## Usage

```bash
jsonschema2rdf convert path/to/schema.json -o out
```

writes, as xsd2rdf does:

- `schema.json.shape.ttl`: the SHACL shapes
- `schema.json.owl.ttl`: the OWL ontology
- `schema.json.skos.<Name>.ttl`: one SKOS concept scheme per enum
- `schema.json.provenance.ttl`: with `--source-provenance` only

A folder converts each of its `*.json` files on its own. A schema may also be
written in YAML (`.yaml`, `.yml`).

```bash
jsonschema2rdf validate path/to/schema.json data.ttl -o out
```

validates RDF data against the converted shapes, with the concept schemes added
to the data (this is `xsd2rdf validate`; exit code 0 conforms, 1 does not, 2 a
file is missing).

From Python:

```python
from jsonschema2rdf import convert

result = convert("schema.json", use_namespace="http://data.europa.eu/949/", property_kind_qualified_iris=True)
result.shacl, result.owl, result.skos   # the graphs (xsd2rdf.Result), nothing is written yet
result.write("out")
```

The settings are the fields of `jsonschema2rdf.Options`. Progress and warnings
go to the `jsonschema2rdf` and `xsd2rdf` loggers.

## Options

The flags are those of xsd2rdf where the setting is the same.

| Flag | Option | Effect |
|---|---|---|
| `-o, --output-dir` | | Output folder (default: the folder of the schema). |
| `-x, --schema-file`, `-f, --folder` | | A schema file, or a folder whose `*.json` files are converted one by one (positional paths work too). |
| `-u, --namespace` | `use_namespace` | Namespace of the classes, properties and concept schemes; the shapes go in `<namespace>shapes/`. Default: the schema's absolute `http(s)` `$id` (draft 4: `id`) with a `/` at the end, else `http://example.com/`. |
| `-k, --property-kind-qualified-iris` | `property_kind_qualified_iris` | Append `DP` or `OP` to every property IRI according to its OWL property kind (the naming policy of xsd2rdf). |
| `--functional-properties` | `functional_properties` | Declare `owl:FunctionalProperty` for properties whose shapes all have `sh:maxCount 1`. **Off by default** (xsd2rdf: on): it is a global axiom drawn from a local cardinality. `--no-functional-properties` is accepted too. |
| `-n, --namespaced-concepts` | `namespaced_concepts` | Concept IRIs `<ns>concepts/<Scheme>/<value>` instead of `<ns>concepts/<Scheme>_<value>`. |
| `-p, --has-prefix-for-properties` | `has_prefix_for_properties` | Property IRIs start with `has`. |
| `-a, --abbreviations-file` | `abbreviations_file` | Words that labels keep in capitals, one per line. |
| `--source-provenance` | `source_provenance` | Also write `<schema>.provenance.ttl` (see [Provenance](#provenance)). |
| `--no-pruning` | `prune` | Also convert the definitions of referenced documents that the schema does not use. |
| `--date YYYY-MM-DD` | `date` | `dct:created`, `dct:issued` and `dct:modified` (default: `SOURCE_DATE_EPOCH`, else today). |
| `-s, --skip-shacl-validation` | `skip_shacl_validation` | Skip the SHACL-SHACL check of the shapes. |
| `--strict` | `strict` | An ill-formed shapes graph is an error. |
| `-d, --debug`, `-q, --quiet` | `debug` | More or less output. |

xsd2rdf's `--simple-type-subproperties`, `--source-namespaces` and
`--namespace-map` have no JSON Schema counterpart.

## Mapping rules

Each rule has a fixture in [`tests/fixtures`](tests/fixtures/README.md) with
its golden output and instance data that must (not) conform. `<ns>` is the
namespace, `<shapes>` is `<ns>shapes/`. Names become IRIs as in xsd2rdf: a
class name starts with a capital, a property name with a lower-case letter
(`-p` puts `has` in front), characters that do not belong in an IRI become
`_`. Labels (`sh:name`, `rdfs:label`) are made from the names by xsd2rdf's
label function (`CarDescriptionType` → "Type of car description"), or are the
schema's `title`.

### Documents and names

| JSON Schema | Result |
|---|---|
| The schema file | Its `definitions` / `$defs` are converted, used or not, and its root schema if it is an object schema. |
| `$ref` | Resolved against the file it is written in (`other.json#/definitions/X`); a URI that is the `$id` of a loaded document resolves to that document. Remote documents are not fetched, plain-name fragments (`$anchor`) are not supported. |
| A definition of another document | Converted when the schema uses it, directly or not (`--no-pruning`: always). |
| A named schema | `#/definitions/<Name>`, `#/$defs/<Name>`, or a document root (named after its `title`, else the file). The same name from two documents: the second gets `_2`. |
| A root that is only a `$ref` | No class of its own: the referenced definition is the class. |
| A definition that is only a `$ref` (an alias) | No class: what refers to it refers to the target. |

### Object schemas

| JSON Schema | SHACL | OWL |
|---|---|---|
| Named object schema (`type: object`, `properties`, or an `allOf` with an object part) | `<shapes>NodeShape/<Name>` with `sh:targetClass <ns><Name>`, `sh:name`, `sh:description` (from `description`) | `owl:Class` with `rdfs:label`, `rdfs:comment` |
| Property `p` of `<Name>` | `<shapes>PropertyShape/<Name>/<p>` with `sh:path <ns><p>`, linked by `sh:property` | `owl:DatatypeProperty` or `owl:ObjectProperty` (from the shapes), `rdfs:domain` the class(es), `rdfs:range` from the shapes |
| `required` | `sh:minCount 1` | |
| A property that is not an array | `sh:maxCount 1` | `owl:FunctionalProperty` only with `--functional-properties` |
| Property whose value is a `$ref` to a named object schema | `sh:nodeKind sh:IRI`, `sh:node <shapes>NodeShape/<Target>` | object property, range the target class |
| Property whose value is an inline object schema | `sh:nodeKind sh:IRI`, `sh:class <ns><P>`, `sh:node <shapes>NodeShape/<Name>/<p>`, a node shape for the inline schema (as a local element with an anonymous complexType in xsd2rdf) | class `<P>` named after the property |
| `allOf: [ {$ref: Base}, {properties...} ]` | `sh:node <shapes>NodeShape/<Base>` on the node shape; the inline parts add properties and `required` | `rdfs:subClassOf <Base>` |
| `allOf` with one member around a `$ref` | the `$ref` | |
| `description` next to a `$ref` (or around it in a one-member `allOf`) | `sh:description` of the property shape | `rdfs:comment` of the property |

Two node shapes that would target the same class (an inline object named like
another class) are scoped as in xsd2rdf: a node shape that a property shape
reaches with `sh:node` loses its `sh:targetClass`.

### Arrays

| JSON Schema | SHACL |
|---|---|
| `type: array`, `items` | the property has the values of `items`; no `sh:maxCount` |
| `maxItems` | `sh:maxCount` |
| `minItems` of a required array | `sh:minCount` (at least 1) |
| `minItems` of an optional array | nothing: an empty and a missing array are the same in RDF |
| A named array of inline objects or enum values | the items take the definition's name (class or scheme) |
| `uniqueItems` | nothing (RDF values are a set) |

### Scalars

| JSON Schema | SHACL / OWL |
|---|---|
| `type` `string`, `integer`, `number`, `boolean` | `sh:datatype` `xsd:string`, `xsd:integer`, `xsd:decimal`, `xsd:boolean`; datatype property with that range |
| `format` `date-time`, `date`, `time`, `duration` | `xsd:dateTime`, `xsd:date`, `xsd:time`, `xsd:duration` |
| `format` `uri`, `uri-reference`, `iri`, `iri-reference`, `url` | `xsd:anyURI` |
| `format` `int32`, `int64`, `float`, `double`, `byte` (OpenAPI) | `xsd:int`, `xsd:long`, `xsd:float`, `xsd:double`, `xsd:base64Binary` |
| `contentEncoding: base64` | `xsd:base64Binary` |
| other formats (`email`, `uuid`, ...) | the datatype of the type |
| `minLength`, `maxLength` | `sh:minLength`, `sh:maxLength` |
| `pattern` | `sh:pattern`, as it is (a JSON Schema pattern is not anchored, like a SPARQL regex) |
| `minimum`, `maximum` | `sh:minInclusive`, `sh:maxInclusive`, typed with the datatype |
| `exclusiveMinimum`, `exclusiveMaximum` (number) | `sh:minExclusive`, `sh:maxExclusive` |
| `exclusiveMinimum: true` (draft 4) | `minimum` becomes `sh:minExclusive` (likewise for the maximum) |
| `const` | `sh:in` with the one value |
| `default` | `sh:defaultValue`, typed with the datatype |
| `$ref` to a named scalar schema | the datatype and facets of the definition, the property's own keywords winning; no shape of its own (a value space, as a named simpleType in xsd2rdf) |
| `type: [string, integer]` | `sh:or ( [sh:datatype xsd:string] [sh:datatype xsd:integer] )`; range the union of the datatypes |
| `type: [..., "null"]`, `null` in `enum` | nullable: a required property gets no `sh:minCount` |

### Enums

| JSON Schema | SKOS | SHACL / OWL |
|---|---|---|
| Named schema with `enum` | `skos:ConceptScheme <ns>concepts/<Name>/<Name>`, `skos:definition` from `description` | |
| Inline `enum` on property `p` of class `<C>` | the scheme `<C><P>` (`Wagon` + `status` → `WagonStatus`) | |
| Each value | `skos:Concept <ns>concepts/<Name>_<value>` (`-n`: `<ns>concepts/<Name>/<value>`) with `skos:notation` (`xsd:token`), `skos:prefLabel` made from the value, `skos:inScheme`, `skos:topConceptOf` | |
| A property with an enum value | | `sh:nodeKind sh:IRI`, `sh:class skos:Concept` and the SPARQL constraint of xsd2rdf that the concept is `skos:inScheme` the scheme; object property with range `skos:Concept` |
| `default` of an enum | | `sh:defaultValue` the concept |
| `null` in an enum | no concept | nullable |

The scheme files are `<schema>.skos.<Name>.ttl`. Two different enums that would
get the same scheme name: the second gets `_2`.

### oneOf and anyOf

| JSON Schema | SHACL |
|---|---|
| `oneOf` / `anyOf` of scalar schemas | `sh:or` of their datatypes and facets |
| `oneOf` of `$ref`s to named object schemas | `sh:nodeKind sh:IRI`, `sh:xone ( [sh:node A] [sh:node B] )` |
| `anyOf` of named object schemas | `sh:nodeKind sh:IRI` only (warning): `sh:or` would make it a datatype property in the xsd2rdf model |
| other mixes | nothing (warning) |

### Not mapped

The shape model of xsd2rdf has no term for these, so they are reported (one
warning per keyword, with the number of schemas and the first place) and left
out: `additionalProperties` (`false` would be `sh:closed`), `patternProperties`,
`propertyNames`, `dependencies` / `dependentSchemas` / `dependentRequired`,
`minProperties` / `maxProperties`, `unevaluatedProperties`, `not`,
`if` / `then` / `else`, `oneOf` / `anyOf` next to `properties`, `multipleOf`,
`items` as a list (tuples). `examples`, `readOnly`, `writeOnly`, `deprecated`
and `$comment` are ignored.

### The ontology

`<ns>` is an `owl:Ontology` with `rdfs:label`
`Ontology generated from <title or file name> JSON Schema`, `dct:title` and `dct:description` (the schema's `title`
and `description`, else that label) and `dct:created`, `dct:issued`,
`dct:modified`.

## Provenance

With `--source-provenance`, `<schema>.provenance.ttl` links each node and
property shape, class, property and concept scheme to the schema it was made
from with `dct:source`: the file (relative to the main schema) and a JSON
Pointer, as a URI reference typed `xsd:anyURI`, for example
`types.json#/definitions/Ticket/properties/route`. A class or property gets one
value per schema it was made from. Each property shape also gets the JSON name
of its property, `jsonschema2rdf:jsonName` (the property IRI is lower-camel and
may carry `has`, `DP` or `OP`, so it cannot be turned back into the name).
The term is defined in [`vocabulary/jsonschema2rdf.ttl`](vocabulary/jsonschema2rdf.ttl),
in the namespace `https://w3id.org/jsonschema2rdf#`.

## Building on jsonschema2rdf

`jsonschema2rdf.JsonSchemaToRDF` is a subclass of `xsd2rdf.XSDtoRDF`. The
parts that differ between JSON Schema dialects are methods that a subclass can
override: `named_type_name` and `entry_points` (where the named schemas are),
`is_nullable`, `enum_values` and `enum_entries`, `discriminator_property`,
`ontology_metadata` and `read`. `jsonschema2rdf.cli` is the command line,
parameterised by a `Program`. openapi2rdf is built this way.

## Tests

```bash
poetry install
poetry run pytest
```

- `tests/test_fixtures.py`: the [fixture suite](tests/fixtures/README.md), one
  folder per mapping rule (golden output, SHACL-SHACL, instance data, the
  same bytes in two runs). After an intended output change:
  `poetry run pytest tests/test_fixtures.py --update-golden`, then review the
  diff. Regenerate the catalogue with `python tests/fixture_support.py`.
- `tests/test_golden.py`: the real B14 schema (UIC IRS 90918-4) of the ERA
  telematics ontology, see [tests/golden](tests/golden/README.md).

`ruff check src tests` and `mypy` are configured in `pyproject.toml`.

The fixtures are also published as a documentation site on GitLab Pages,
built by the `pages` job on `main`: one page per case with the JSON Schema source,
the golden output, the instance data and UML-like class diagrams of the source,
the SHACL shapes and the OWL ontology, plus the mapping rules above and a
summary of the B14 golden test. The site is at
[https://era-europa-eu.gitlab.io/private/OPDU/SOT/era-ontology/telematics/jsonschema2rdf/](https://era-europa-eu.gitlab.io/private/OPDU/SOT/era-ontology/telematics/jsonschema2rdf/)
(Deploy → Pages in GitLab shows the address if the project uses a unique Pages
domain; the site is visible to whoever can see the project). To build it
locally (Mermaid then loads from a CDN):

```bash
poetry run python docs/build_site.py public
```

## Releasing

Releases are published to [PyPI](https://pypi.org/project/jsonschema2rdf/) from GitLab CI.

1. Update `CHANGELOG.md`: move the changes to a new `[X.Y.Z] - YYYY-MM-DD`
   section and merge it to `main`.
2. In GitLab, create a release (Deploy → Releases → New release) with a new
   tag `vX.Y.Z` created from `main`. The release notes can go in its
   description.
3. The tag pipeline runs lint and the tests, sets the package version from the
   tag (`v1.2.3` gives `1.2.3`), builds the wheel and the sdist, and uploads
   them to PyPI with trusted publishing: no PyPI password or token is stored in
   GitLab. Only tags of the form `vX.Y.Z` (or `X.Y.Z`) build and publish.

The PyPI project has a GitLab trusted publisher with these settings (before
the first release, when the project does not exist on PyPI yet, add it as a
pending publisher under Your account → Publishing):

| Setting | Value |
|---|---|
| Namespace | `era-europa-eu/private/OPDU/SOT/era-ontology/telematics` |
| Project name | `jsonschema2rdf` |
| Top-level pipeline file path | `.gitlab-ci.yml` |
| Environment name | (empty) |

## License

[EUPL 1.2](http://data.europa.eu/eli/dec_impl/2017/863/oj)

