Metadata-Version: 2.4
Name: pyshifty
Version: 0.2.7
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Scientific/Engineering
Requires-Dist: rdflib>=6.0 ; extra == 'rdflib'
Provides-Extra: rdflib
Summary: Python bindings for the shifty SHACL engine
Keywords: shacl,rdf,validation,knowledge-graph,semantic-web
Home-Page: https://github.com/gtfierro/shifty
Author-email: Gabe Fierro <gtfierro@mines.edu>
License-Expression: BSD-3-Clause
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://github.com/gtfierro/shifty#readme
Project-URL: Homepage, https://github.com/gtfierro/shifty
Project-URL: Repository, https://github.com/gtfierro/shifty

# shifty

A formalism-first SHACL validation and SHACL-AF inference engine written in Rust, grounded in the algebraic treatment of *Common Foundations for SHACL, ShEx, and PG-Schema* (arXiv:2502.01295). Available as a command-line tool and as Python bindings (`pyshifty`).

## Features

- **Full SHACL Core validation** — node and property shapes, all standard constraint components
- **SHACL-AF inference** — forward-chaining `sh:rule` evaluation (Triple Rules, SPARQL Construct Rules) to a fixed point, with stratification analysis for recursive rulesets
- **Algebraic IR** — shapes are lowered to a path algebra (π) and shape grammar (φ) before evaluation; the same IR drives both validation and inference
- **Native SPARQL execution** — a subset of `sh:sparql` constraints and SPARQL Construct rules runs directly over an indexed dataset without a full SPARQL engine, with automatic fallback to Spareval for unsupported constructs
- **Multi-layer pipeline** — parsing → algebraic lowering → normalization/CSE → physical planning → execution; each layer is independently inspectable
- **pyshifty-compatible Python API** — `validate()` returns `(conforms, report_graph, results_text)` matching pyshifty's interface

## Installation

### CLI

```sh
cargo install --path crates/shifty-cli
```

Or build from source:

```sh
cargo build --release -p shifty-cli
# binary at target/release/shifty
```

### Python

```sh
pip install pyshifty
```

The package installs as `pyshifty` but is imported as `shifty`:

```python
import shifty
```

To build from source (requires Rust and [maturin](https://github.com/PyO3/maturin)):

```sh
cd python
pip install maturin
maturin develop
```

## CLI usage

### Validate

```sh
shifty validate --shapes shapes.ttl --data data.ttl
```

```
conforms: false
violations: 1
  <http://example.org/bob>  [target: ∃ rdf:type .⊤]
      - (ex:name) 123 → expected datatype xsd:string
```

Emit a W3C `sh:ValidationReport` in Turtle:

```sh
shifty validate --shapes shapes.ttl --data data.ttl --report
```

JSON output:

```sh
shifty validate --shapes shapes.ttl --data data.ttl --format json
```

Graph mode controls which triples are visible to path traversal and SPARQL evaluation:

```sh
# default: focus nodes from data; paths/SPARQL use data ∪ shapes
shifty validate --shapes shapes.ttl --data data.ttl --graph-mode union

# focus nodes and evaluation use data only
shifty validate --shapes shapes.ttl --data data.ttl --graph-mode data

# focus nodes and evaluation both use data ∪ shapes
shifty validate --shapes shapes.ttl --data data.ttl --graph-mode union-all

# validate only selected named shapes as top-level entry points
shifty validate \
  --shapes shapes.ttl \
  --data data.ttl \
  --shape-name http://example.org/PersonShape
```

`--shape-name` is repeatable and has the alias `--entry-shape`. Selected
shapes are used only as entry points; referenced helper shapes are still
evaluated normally.

### Infer

Run SHACL-AF rules to a fixed point, then print the derived triples:

```sh
shifty infer --shapes rules.ttl --data data.ttl
```

```
inferred 3 triple(s):
  <http://example.org/r1> <http://example.org/area> "6"^^<http://www.w3.org/2001/XMLSchema#integer>
  ...
```

### Inspect

Inspect how a shapes graph looks at each stage of the pipeline:

```sh
# Raw triples after parsing
shifty inspect --stage rdf shapes.ttl

# Lowered algebraic IR (φ/π notation)
shifty inspect --stage algebra shapes.ttl

# After normalization and common-subexpression elimination
shifty inspect --stage normalized shapes.ttl

# Stratification analysis (recursion detection)
shifty inspect --stage strata shapes.ttl

# Physical plan: focus sources + cost-ordered shape checks
shifty inspect --stage plan shapes.ttl

# SPARQL constraint capability: which queries run native vs. Spareval
shifty inspect --stage capability shapes.ttl
```

All stages support `--format text` (default), `--format json`; the `algebra` and `normalized` stages also support `--format dot` for Graphviz output.

Shapes files and data files may be local paths or HTTP/HTTPS URLs. Both `--shapes` and `--data` are repeatable to merge multiple files.

## Python usage

```python
import shifty
```

### Validate (pyshifty-compatible)

```python
shapes = """
@prefix sh:  <http://www.w3.org/ns/shifty#> .
@prefix ex:  <http://example.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

ex:PersonShape a sh:NodeShape ;
    sh:targetClass ex:Person ;
    sh:property [
        sh:path ex:name ;
        sh:minCount 1 ;
        sh:datatype xsd:string ;
    ] ;
    sh:property [
        sh:path ex:age ;
        sh:maxCount 1 ;
        sh:datatype xsd:integer ;
    ] .
"""

data = """
@prefix ex: <http://example.org/> .

ex:Alice a ex:Person ; ex:name "Alice" ; ex:age 30 .
ex:Bob   a ex:Person .
"""

conforms, report_graph, results_text = shifty.validate(data, shapes)
# conforms → False
# report_graph → rdflib.Graph with sh:ValidationReport
# results_text → human-readable summary
```

Graph inputs can be a string, `bytes`, `pathlib.Path`, or `rdflib.Graph`. If `shacl_graph` is omitted or passed as `None`, shapes are expected to be embedded in the data graph. Do not pass an empty `rdflib.Graph()` for embedded shapes; that is treated as an explicit empty shapes graph.

> **Where shapes are read from.** Pass a *single* graph (omit `shacl_graph` or pass `None`) and shifty reads both the shape definitions and the data from that one graph. Pass a *separate* shapes graph and the schema is compiled **only** from it — SHACL vocabulary that happens to sit in the data graph is ignored, never turned into constraints. This keeps validation predictable and matches SHACL's separation of the shapes graph from the data graph. To validate against shapes that live in the data graph, union that graph into the `shacl_graph` argument yourself (it accepts a list, unioned before evaluation); shifty will not read shapes from the data side automatically.

Any data or shapes argument also accepts a **list (or tuple)** of the above;
the members are unioned (merged at the RDF triple level, the same way the CLI's
repeatable `--shapes` / `--data` merge) before being passed to the engine. A
single input keeps its native fast path.

```python
# Union two shapes files and two data files before validating.
shifty.validate(["data1.ttl", "data2.ttl"], ["shapes1.ttl", "shapes2.ttl"])

# Works everywhere graphs are accepted:
validator = shifty.PreparedValidator(["shapes1.ttl", "shapes2.ttl"])
validator.validate([rdflib.Graph(), extra_data])
```

To validate a shapes graph against itself, pass it once. The embedded path
parses and plans one graph without constructing separate data and shapes
graphs:

```python
result = shifty.validate_algebra("shapes.ttl", infer=False)
conforms, report_graph, results_text = shifty.validate("shapes.ttl", infer=False)
```

`pathlib.Path` inputs are parsed directly in Rust. `rdflib.Graph` inputs are
transferred as N-Triples.

### Reuse prepared shapes

For multiple data graphs using the same shapes, cache parsing, normalization,
and planning with `PreparedValidator`:

```python
validator = shifty.PreparedValidator(shapes)

result = validator.validate_algebra(data, infer=False)
conforms, report_graph, results_text = validator.validate(data)
```

### Validate with structured result

`validate_algebra` returns an `AlgebraResult` with typed `Violation` objects instead of an RDF report graph:

```python
result = shifty.validate_algebra(data, shapes)
print(result.conforms)        # False
print(result.results_text)    # human-readable summary (built and cached on first access)
for v in result.violations:
    print(v.focus_node)       # IRI of the failing focus node
    print(v.shape_name)       # IRI of the violated shape, or None
    for r in v.reasons:
        print(r.message)          # engine-generated failure description (always set)
        print(r.author_message)   # the shape's sh:message if it declared one, else None
        print(r.path)             # path that was checked, if applicable
        print(r.value)            # the offending value node

        # Prefer the author's message when present, fall back to the generated one:
        print(r.author_message or r.message)
```

Set `infer=False` when validation should not first run embedded SHACL-AF rules
to a fixed point.

### Infer

Run SHACL-AF rules to a fixed point:

```python
rules = """
@prefix sh: <http://www.w3.org/ns/shifty#> .
@prefix ex: <http://example.org/> .

ex:RectangleShape a sh:NodeShape ;
    sh:targetClass ex:Rectangle ;
    sh:rule [
        a sh:TripleRule ;
        sh:subject sh:this ;
        sh:predicate ex:area ;
        sh:object [ sh:path ex:width ] ;
    ] .
"""

data = """
@prefix ex: <http://example.org/> .
ex:r1 a ex:Rectangle ; ex:width 3 ; ex:height 2 .
"""

result = shifty.infer(data, rules)
print(result.inferred_count)    # number of newly derived triples
g = result.graph()              # rdflib.Graph with original + inferred data
```

If rules are embedded in the data graph, omit the second argument or pass
`None`:

```python
result = shifty.infer(combined_data_and_rules)
result = shifty.infer(combined_data_and_rules, None)
```

Passing `rdflib.Graph()` as the second argument means “run with an explicit
empty rules graph,” so no embedded rules will be parsed.

### graph_mode

`validate()` and `validate_algebra()` accept a `graph_mode` keyword argument:

```python
shifty.validate(data, shapes, graph_mode="union")      # default
shifty.validate(data, shapes, graph_mode="data")
shifty.validate(data, shapes, graph_mode="union-all")
```

When `shacl_graph` is omitted, all three modes are equivalent because focus
discovery and evaluation use the same embedded graph. `infer()` does not accept
`graph_mode`.

### shape_names

`validate()` and `validate_algebra()` accept `shape_names=[...]` to validate
only the named SHACL shapes in that list as top-level entry points:

```python
result = shifty.validate_algebra(
    data,
    shapes,
    shape_names=["http://example.org/PersonShape"],
)
```

Only target-bearing statements owned by those named shapes are used as entry
points. Any helper shapes referenced from them through `sh:node`, `sh:property`,
qualified value shapes, boolean shape expressions, and similar dependencies are
still evaluated normally. Shape names may be passed as bare IRIs or wrapped in
angle brackets.

### File inputs

```python
import pathlib

conforms, report, text = shifty.validate(
    pathlib.Path("data.ttl"),
    pathlib.Path("shapes.ttl"),
)
```

## Property witnesses (conformance bindings)

`validate`/`validate_algebra` report violations. `PreparedValidator.witnesses()`
is their inverse: for every focus node that *conforms* to a target/profile node
shape, it returns the values each `sh:property` shape's `sh:path` resolved to.
Useful when a SHACL profile doubles as an extraction schema — e.g. disambiguating
several same-typed sensors on a piece of equipment via `sh:qualifiedValueShape`.

(This is a different concept from the `RepairSession` "witnesses" below, which
records why a focus node *fails* a statement for repair synthesis. This one only
reports observed, passing bindings — no repair suggestions.)

```python
shapes = """
@prefix sh:  <http://www.w3.org/ns/shacl#> .
@prefix zea: <http://example.org/zea#> .
@prefix ex:  <http://example.org/> .

ex:VavProfile a sh:NodeShape ;
    sh:targetClass ex:Vav ;
    sh:property [
        zea:role ex:OutsideAirTempRole ;
        sh:path ex:hasPoint ;
        sh:qualifiedValueShape [ sh:hasValue ex:oat ] ;
        sh:qualifiedMinCount 1 ;
        sh:qualifiedMaxCount 1 ;
    ] ;
    sh:property [
        zea:role ex:ReturnAirTempRole ;
        sh:path ex:hasPoint ;
        sh:qualifiedValueShape [ sh:hasValue ex:rat ] ;
        sh:qualifiedMinCount 1 ;
        sh:qualifiedMaxCount 1 ;
    ] .
ex:OutsideAirTempRole zea:roleName "outsideAirTemp" .
ex:ReturnAirTempRole zea:roleName "returnAirTemp" .
"""
data = """
@prefix ex: <http://example.org/> .
ex:vav1 a ex:Vav ; ex:hasPoint ex:oat, ex:rat, ex:sat, ex:mat .
"""

validator = shifty.PreparedValidator(shapes)
for w in validator.witnesses(data, key_path="zea:role/zea:roleName"):
    print(w.focus, w.key, w.values)
# <http://example.org/vav1> outsideAirTemp ['<http://example.org/oat>']
# <http://example.org/vav1> returnAirTemp  ['<http://example.org/rat>']
```

`key_path` is a SPARQL 1.1 property path expression (sequence `/`, alternation
`|`, inverse `^`, and the Kleene forms `*`/`+`/`?` are all supported), evaluated
from each `sh:property` shape's own node, over the shapes graph, to produce a
stable key. The key here isn't a direct annotation on the property shape — it
lives one hop further away, through an intermediate role-descriptor node — which
is exactly what a bare predicate lookup couldn't reach but a path can. A direct
annotation (`zea:roleName "outsideAirTemp"` right on the `sh:property` shape)
would just be `key_path="zea:roleName"`; a descriptor that points *at* the
property shape instead of the other way around would use an inverse hop,
`key_path="^zea:describes/zea:roleName"`. Prefixes resolve against the shapes
document's declared `@prefix`es. Property shapes where the path resolves to no
value fall back to their own IRI/blank-node id as `.key`. `.values` entries are
rendered in full (`<iri>`, `"lit"`, `"lit"@lang`, `"lit"^^<datatype>`) so IRI and
literal bindings stay distinguishable.

## Witnesses (symbolic repair)

`RepairSession` exposes the *witnessing* layer: for each statement it reports why
a focus node fails (a `FocusWitness`) or why it holds (a `FocusSatisfaction`),
the structured input to repair synthesis. The session is immutable; it computes
and gates but decides nothing.

```python
shapes = """
@prefix sh:  <http://www.w3.org/ns/shacl#> .
@prefix ex:  <http://example.org/> .

ex:PersonShape a sh:NodeShape ;
    sh:targetClass ex:Person ;
    sh:property [ sh:path ex:name ; sh:minCount 1 ] .
"""
data = """
@prefix ex: <http://example.org/> .
ex:carol a ex:Person ; ex:name "Carol" .   # passes ex:PersonShape
ex:dan   a ex:Person .                      # fails: no ex:name
"""

session = shifty.RepairSession(shapes, data, infer=False)
```

### The whole horizon

`witnesses()` returns one `FocusWitness` per `(focus node, failed statement)`
across the entire schema. Empty ⟺ the graph conforms.

```python
for w in session.witnesses():
    print(w.focus)        # '<http://example.org/dan>'
    print(w.statement)    # 0 — index into the schema's statements
    print(w.target)       # 'class(<http://example.org/Person>)' — rendered selector
```

### Structured access (strings *and* objects)

Everything that has a readable string also has a structured, inspectable form, so
you can branch and process externally instead of parsing text. `w.target` is the
rendered selector; `w.selector` is the same thing decomposed:

```python
sel = w.selector
print(sel.kind)      # TargetKind.Class — an enumerated discriminant
print(sel.value)     # '<http://example.org/Person>' — N-Triples, round-trips
print(sel.render)    # 'class(<http://example.org/Person>)' == w.target
print(str(sel))      # same rendered string

if sel.kind == shifty.TargetKind.Class:
    ...              # dispatch on the kind, not on a substring
```

`kind` fields are real enums, not bare strings — so the valid set is discoverable
at runtime and usable in `match`/comparisons:

```python
shifty.TargetKind   # Class | SubjectsOf | ObjectsOf | Node | Path | Sparql
shifty.WitnessKind  # Atom | Relational | Closed | CountLow | CountHigh | Not | Opaque
shifty.SatKind      # Atom | Match | Not | Blocked | Coinductive
shifty.ChoiceKind   # Any | Repeat
```

### Scope to one shape

`witnesses_for(shape_iri)` narrows the horizon to the statements that target a
single shape, matched against the schema's shape IRIs (angle brackets optional).
It raises `ValueError` if no shape is named `shape_iri`.

```python
for w in session.witnesses_for("http://example.org/PersonShape"):
    # flat bag of failing leaves (AND/OR structure dropped)
    for a in w.summary():       # a is a WitnessAtom
        print(a.kind, a.path, a.detail)   # WitnessKind.CountLow <…/name> have 0, need 1
        if a.kind == shifty.WitnessKind.CountLow:
            ...

    print(w.explain())          # indented witness tree:
                                # CountLow along <…/name>: have 0, need 1

    tree = w.repair_tree()      # synthesize the repair space for this violation
    print(tree.is_blocked)      # False — a data repair exists in scope
```

### Passing nodes and the values that satisfied them

`satisfactions_for(shape_iri)` is the dual: one `FocusSatisfaction` per *passing*
focus node for that shape. Each records why the node conforms, including the
values matched along every checked path — the satisfaction-side mirror of
`witnesses_for`.

```python
for fs in session.satisfactions_for("http://example.org/PersonShape"):
    print(fs.focus)             # '<http://example.org/carol>'
    print(fs.statement)         # 0
    print(fs.target)            # same rendered selector as the witness side
    print(fs.selector.kind)     # TargetKind.Class — same structured selector too

    for a in fs.summary():      # a is a SatAtom
        # one Match leaf per value that satisfied a checked path
        if a.kind == shifty.SatKind.Match:
            print(a.path, a.value)        # <…/name> "Carol"

    print(fs.explain())         # CountHeld: 1 match(es)
```

`witnesses_for` and `satisfactions_for` partition the targeted focus nodes:
every node that fails appears in one, every node that holds in the other. For
`closed`, relational (`sh:equals`/`sh:lessThan`/…), and opaque-SPARQL
constraints a satisfaction leaf is reported as `SatKind.Blocked` — the node
holds, but no enumerable value set is exposed.

## Crate structure

| crate | role |
|---|---|
| `shifty-algebra` | path algebra π, shape grammar φ, schema arena, rendering |
| `shifty-parse` | Turtle/RDF → algebraic IR lowering |
| `shifty-opt` | normalization, stratification, physical planning, native SPARQL lowering |
| `shifty-engine` | validation + AF inference execution, SPARQL executor |
| `shifty-cli` | `shifty` binary |
| `pyshifty` (python/) | PyO3 bindings, published as `pyshifty` on PyPI |

## Design docs

The `docs/` directory contains the full design:

- [`docs/00-formalism.md`](docs/00-formalism.md) — path algebra π, shape grammar φ, selectors, reference semantics
- [`docs/01-gap-analysis.md`](docs/01-gap-analysis.md) — W3C SHACL/SHACL-AF coverage and known gaps
- [`docs/02-roadmap.md`](docs/02-roadmap.md) — layered build plan (Layer 0 → 7)

## License

BSD-3-Clause

