Metadata-Version: 2.5
Name: lokf
Version: 0.9.0
Summary: Linked Open Knowledge Format (LOKF) — a semantic profile of Google's Open Knowledge Format, defined once in LinkML
Project-URL: Documentation, https://lokf.nolan-nichols.com/
Project-URL: Repository, https://github.com/nicholsn/lokf
Project-URL: Issues, https://github.com/nicholsn/lokf/issues
Author: Nolan Nichols
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: dcat,json-ld,knowledge-graph,linkml,mcp,okf,prov,rdf,sparql
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Markup
Requires-Python: >=3.10
Requires-Dist: curies>=0.14
Requires-Dist: linkml-runtime>=1.12.0
Requires-Dist: mcp>=2.0
Requires-Dist: pyoxigraph>=0.5.4
Requires-Dist: pyyaml>=6.0
Requires-Dist: rdflib>=7.0
Requires-Dist: ruamel-yaml>=0.18
Requires-Dist: tomli>=2.0; python_version < '3.11'
Requires-Dist: typer>=0.15
Provides-Extra: build
Requires-Dist: linkml>=1.12.0; extra == 'build'
Requires-Dist: markdown-it-py>=3; extra == 'build'
Provides-Extra: shacl
Requires-Dist: linkml>=1.12.0; extra == 'shacl'
Requires-Dist: markdown-it-py>=3; extra == 'shacl'
Requires-Dist: pyshacl<1,>=0.30; extra == 'shacl'
Provides-Extra: tables
Requires-Dist: pandas>=2.0; extra == 'tables'
Requires-Dist: pyarrow>=15; extra == 'tables'
Description-Content-Type: text/markdown

# Linked Open Knowledge Format (LOKF)

A **semantic profile of Google's Open Knowledge Format (OKF) v0.2**. LOKF keeps OKF's
markdown-plus-frontmatter authoring model but binds every concept, field, and
relationship to **schema.org, W3C DCAT, and W3C PROV-O**, so a bundle of markdown
files is *also* valid **JSON-LD** that expands losslessly to **RDF**. The format is
defined once in **LinkML**; the JSON-LD context, JSON Schema, SHACL shapes, and OWL
ontology are all generated from that single source.

> One sentence: *write OKF markdown, get a queryable knowledge graph for free.*

**Documentation:** <https://lokf.nolan-nichols.com/>

## Why

OKF is deliberately minimal — the only required field is `type`, links are
untyped, and there's no shared vocabulary. LOKF adds six things while keeping
OKF's ergonomics:

1. **Shared meaning** - types and fields map to public ontology terms.
2. **Typed relationships** - `dependsOn`, `derivedFrom`, `isPartOf`, … each pinned to an RDF predicate, instead of one untyped markdown link.
3. **A real graph** - the same bundle is queryable with SPARQL, validatable with SHACL, and reason-able with OWL.
4. **Queryable trust** - OKF v0.2's provenance, trust, and lifecycle frontmatter (`sources`, `usage_window`, `generated`, `verified`, `status`, `stale_after`) and the `AttestedComputation` type, each bound to PROV-O / schema.org, so "where did this come from and should I trust it" is a SPARQL query rather than YAML an agent has to interpret. See [SPEC §5.4](./SPEC.md).
5. **Domain schemas** - a project's own types and keys, declared in a LinkML schema that `imports: [lokf]`. The same validator checks them (`lokf validate --schema`), `--check-refs` resolves their relations, and `convert`, `query` and `serve --schema` project them under their own IRIs. The core vocabulary stays small; a domain extends it without forking it. See [SPEC §6.2](./SPEC.md).
6. **Documentation modes** - an optional `genre` facet (and `Tutorial`/`Explanation` types) aligns each concept's prose to the four [Diátaxis](https://diataxis.fr/) modes, on an axis orthogonal to `type`.

It stays **bidirectionally compatible**: every LOKF bundle is a valid OKF bundle, and every OKF bundle is valid LOKF with default mappings — with or without its domain schema.

## Files

| Path | What it is | Hand-edited? |
|------|------------|:---:|
| `lokf.yaml` | **The LinkML schema — the single source of truth.** | ✅ edit this |
| `SPEC.md` | The human-readable specification. | ✅ |
| `lokf.context.jsonld` | Generated JSON-LD `@context` (+ `type`→`@type`, `id`→`@id` aliases). Attach to concepts to get Linked Data. | ⚙️ generated |
| `lokf.schema.json` | Generated JSON Schema — validates frontmatter / bundles. | ⚙️ generated |
| `lokf.shacl.ttl` | Generated SHACL shapes — validates the RDF graph. | ⚙️ generated |
| `lokf.owl.ttl` | Generated OWL ontology — reasoning & alignment. | ⚙️ generated |
| `lokf.sql` | Generated relational schema — `CREATE TABLE` DDL (FKs for typed relations). | ⚙️ generated |
| `src/lokf/datamodel.py` | Generated Python bindings — `from lokf.datamodel import Metric`. | ⚙️ generated |
| `examples/acme-knowledge/` | A conformant 8-concept reference bundle. | ✅ |
| `examples/*.nt` | RDF triples produced from the example frontmatter. | ⚙️ generated |

## Anatomy of a LOKF concept

`metrics/weekly-active-users.md` — ordinary OKF markdown; every key has a defined
RDF meaning:

<!-- --8<-- [start:concept-glance] -->
```markdown title="metrics/weekly-active-users.md"
---
type: Metric                                    # -> rdf:type lokf:Metric
id: https://acme.example/knowledge/metrics/weekly-active-users   # -> @id (subject)
title: Weekly Active Users                       # -> schema:name
unit: users                                      # -> schema:unitText
generated:                                       # -> prov:wasGeneratedBy
  by: human:jsmith@acme                          #    prov:wasAssociatedWith
  at: 2026-06-30T12:00:00Z                       #    prov:endedAtTime
status: stable                                   # -> schema:creativeWorkStatus
derivedFrom: [ .../tables/user-events ]          # -> prov:wasDerivedFrom
dependsOn:   [ .../glossary/active-user ]        # -> dcterms:requires
measures:    [ .../glossary/active-user ]        # -> lokf:measures
---
# Definition
Distinct users with a qualifying event in a trailing 7-day window.
```
<!-- --8<-- [end:concept-glance] -->

Attach `lokf.context.jsonld` and this expands to RDF triples using
`schema:`, `prov:`, `dcterms:`, and `lokf:` predicates — no separate file.

## Regenerate the artifacts

Everything downstream of `lokf.yaml` is generated. One command reproduces every
artifact, re-assembles the reference bundle, re-validates it, and re-emits the RDF:

```bash
uv sync
just build      # == uv run lokf-build
```

The individual generators produce raw LinkML artifacts. Use `lokf-build` to
apply the authoring-context and SHACL adjustments and synchronize packaged copies:

<!-- --8<-- [start:gen-commands] -->
```bash
# JSON-LD context (aliased type->@type, id->@id for authoring)
uv run gen-jsonld-context lokf.yaml > lokf.context.base.jsonld
uv run gen-json-schema     lokf.yaml > lokf.schema.json
uv run lokf-build  # includes SHACL adapted to the authoring RDF projection
uv run gen-owl             lokf.yaml > lokf.owl.ttl
```
<!-- --8<-- [end:gen-commands] -->

The published `lokf.context.jsonld` is the generated context with two standard
JSON-LD keyword aliases applied so unmodified OKF frontmatter is valid Linked Data:

```python
import json
c = json.load(open("lokf.context.base.jsonld"))
c["@context"]["type"] = "@type"   # OKF's required field designates the RDF class
c["@context"]["id"]   = "@id"     # the concept IRI is the RDF subject
json.dump(c, open("lokf.context.jsonld", "w"), indent=2)
```

## Validate a bundle

<!-- --8<-- [start:validate-bundle] -->
```bash
# `just build` assembles examples/acme-knowledge.bundle.json from the markdown, then:
uv run linkml-validate -s lokf.yaml -C KnowledgeBundle examples/acme-knowledge.bundle.json
# -> No issues found

# Or validate a single concept against its class
uv run linkml-validate -s lokf.yaml -C Metric metric.json
```
<!-- --8<-- [end:validate-bundle] -->

To also validate the projected graph, install `lokf[shacl]` and run:

```bash
lokf validate knowledge --shacl
lokf validate knowledge --shacl --schema domain.yaml
```

The command checks nested values, local relation ranges, and RDF node kinds.
It recognizes Concept inheritance and mixins, and leaves external referenced
IRIs unverified without fetching them. Explicit concept IDs remain local even
outside the bundle namespace. Domain shapes are generated from the selected
schema. The shipped shapes use distinct shape identifiers for classes sharing
an RDF class, and validate the authoring projection (`type` is `rdf:type`,
optional enums are literals, and provenance objects can be untyped).

For direct Python graph checks, `lokf.shacl.validate_graph(graph, shapes,
base_iri, concept_iris)` applies the same local/external policy and embedded
class hierarchy. The unmodified artifact assumes referenced classes are known;
a standalone SHACL engine must supply that hierarchy as ontology data and
choose its own external-reference policy. JSON Schema retains closed-field and
per-field cardinality checks where multiple authoring fields share one predicate.

Validation is closed-world (SPEC §9): a project-specific type or key needs a domain schema that `imports: [lokf]` (SPEC §6.2), passed via `lokf validate --schema`. Pass the same file to `lokf convert`, `query` or `serve` with `--schema` to project its types and keys under their own IRIs. A published LinkML vocabulary becomes such a schema's import with `lokf adapt VOCAB.yaml`; `examples/biolink/` does this for biolink-model.

Schema validation sees one concept at a time, so two files declaring one `id` (a sync client's conflict copy is the usual way) pass and then merge into one subject in the graph; `--check-ids` catches that. It cannot tell a live relation target from a fabricated one either - both are valid strings. `--check-refs` adds that pass:

```bash
uv run lokf validate examples/acme-knowledge --check-refs
# -> OK — 8 concepts ... All in-namespace relation targets resolve.
```

It checks every typed-relation target naming the bundle's **own** namespace —
a relative ref, or an IRI under `base_iri`. External resources are left alone,
since slots like `definedBy` and `source` are defined as taking one.

## Markdown → RDF in one command

The `lokf` CLI projects a concept — or a whole bundle directory — straight to
RDF. No context wiring, no glue code:

<!-- --8<-- [start:quickstart-rdf] -->
```bash
# a single concept -> Turtle on stdout
uv run lokf convert examples/acme-knowledge/metrics/weekly-active-users.md --format ttl

# the same thing behind the just recipe
just gen-rdf-turtle examples/acme-knowledge/metrics/weekly-active-users.md
```

`--format` also takes `nt`, `jsonld`, `xml`, `n3`, and `trig`; `--output FILE`
writes to disk instead of stdout. Point `convert` at the bundle directory
(`examples/acme-knowledge`) to project all eight concepts at once.

Prefer to stay in Python? `lokf.rdf.serialize` is the same projection the CLI
calls:

```python
from lokf import rdf

# a concept file, or the bundle directory — either resolves IRIs correctly
print(rdf.serialize("examples/acme-knowledge/metrics/weekly-active-users.md", "ttl"))
```
<!-- --8<-- [end:quickstart-rdf] -->

This is the whole thesis: OKF authoring in, RDF knowledge graph out.

## Two projections: a graph *and* tables

Because LOKF is defined once in LinkML, a well-modeled bundle projects **two**
ways — the *literal* knowledge graph and the *abstract* one (well-modeled,
linked data you can put in tables):

- **A graph** — markdown → JSON-LD → RDF, queryable with SPARQL, validatable
  with SHACL, reason-able with OWL (shown above).
- **Tables** — the same schema generates typed **Python bindings**
  (`src/lokf/datamodel.py`) and a **relational schema** (`lokf.sql` — one table
  per type, foreign keys for the typed relations). So the same well-modeled data
  is either a queryable graph *or* a set of linked tables: DataFrames to analyze,
  SQL to persist, a lakehouse to scale.

```python
from lokf.datamodel import Metric

Metric(id="https://acme.example/knowledge/metrics/wau",
       type="Metric", title="Weekly Active Users", unit="users")
```

`lokf tables` projects a whole bundle to linked tables — one per type plus a
`relations` edge table (needs `lokf[tables]`):

```bash
lokf tables examples/acme-knowledge --format parquet  --output build/tables
lokf tables examples/acme-knowledge --format bigquery --location gs://bucket/lokf
```

## Start your own

`lokf new my-kb` scaffolds a complete knowledge-base repo — a starter bundle, a
full Astro site with concept pages and the interactive **graph browser**
(`/graph`, plus `graph.jsonld`), a GitHub Pages workflow, a `justfile`, and the
bundled agent skills — so you (or an AI agent) can go from an idea to a
published, queryable knowledge graph:

```bash
lokf new my-kb --title "My Knowledge Base"
cd my-kb && just setup && just dev
```

See the [scaffold docs](https://lokf.nolan-nichols.com/toolkit/scaffold/).

## Status

LOKF v0.2 is a **draft profile** and is **not affiliated with or endorsed by
Google**. The format version tracks OKF's `<major>.<minor>` scheme; the LinkML
schema and the `lokf` package carry their own version (currently 0.9.0) so the
toolkit can ship fixes without implying a format change — see
[SPEC §12](./SPEC.md). "Open Knowledge Format" / "OKF" refer to the format published by Google
Cloud (`github.com/GoogleCloudPlatform/knowledge-catalog`); LOKF extends it under
its open terms.

## AI assistance

LOKF is developed openly and with substantial AI assistance (Claude) — including
the toolkit, the LinkML schema, the documentation, and this README. Nolan
Nichols reviews and takes full responsibility for everything committed. Per the
project's [AI Covenant](AI_COVENANT.md), AI is a tool, not a co-author: it is
never credited in commit messages, and every contribution is owned and defended
by a human. See [CONTRIBUTING.md](CONTRIBUTING.md) to get involved.

## License

The **software** — the `lokf` toolkit in `src/`, the tests, and the build
tooling — is licensed under the Apache License 2.0. See [`LICENSE`](./LICENSE)
and [`NOTICE`](./NOTICE).

The **specification and vocabulary** — [`SPEC.md`](./SPEC.md), the LinkML schema
[`lokf.yaml`](./lokf.yaml), and the artifacts generated from it
(`lokf.context.jsonld`, `lokf.schema.json`, `lokf.shacl.ttl`, `lokf.owl.ttl`,
`lokf.sql`) — are licensed under
[CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/).

`src/lokf/datamodel.py` is generated too, but it is Python that ships *in* the
package, so it is Apache-2.0 like the rest of `src/`. Its header comment — and
`lokf.owl.ttl`'s `dcterms:license` — transcribe `lokf.yaml`'s own `license:`
key, so wherever CC-BY-4.0 appears inside a generated file it is the schema's
license, not the package's. The published package declares `Apache-2.0`, which
is the license of the code you install.



The installed package includes `lokf/data/lokf.schema.json` and
`lokf/data/lokf.shacl.ttl`, synchronized by `lokf-build`. Python hosts can read
these with `importlib.resources.files("lokf").joinpath("data", filename)`;
non-Python clients can use the same versioned artifacts from the repository.
