Metadata-Version: 2.5
Name: xbrlkit
Version: 0.3.0
Summary: Work with XBRL filings above Arelle: one parse, portable models
Project-URL: Repository, https://github.com/RoboFinSystems/xbrlkit
Project-URL: Homepage, https://github.com/RoboFinSystems/xbrlkit#readme
Project-URL: Bug Tracker, https://github.com/RoboFinSystems/xbrlkit/issues
Author: RFS LLC
License: MIT
License-File: LICENSE
Keywords: arelle,edgar,financial,holon,jsonld,ladybugdb,oim,parquet,property-graph,rdf,sec,tavi,xbrl
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Framework :: Pydantic
Classifier: Framework :: Pydantic :: 2
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Office/Business :: Financial
Classifier: Topic :: Office/Business :: Financial :: Accounting
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Utilities
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: arelle-release
Requires-Dist: httpx>=0.28
Requires-Dist: pydantic>=2.11
Requires-Dist: pyshacl
Requires-Dist: python-dotenv>=1.0
Requires-Dist: rdflib
Requires-Dist: regex
Requires-Dist: requests>=2.31
Provides-Extra: dev
Requires-Dist: basedpyright>=1.21; extra == 'dev'
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: ladybug>=0.18; extra == 'dev'
Requires-Dist: pyarrow>=17; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.26; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: ruff>=0.12; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: lpg
Requires-Dist: ladybug>=0.18; extra == 'lpg'
Requires-Dist: pyarrow>=17; extra == 'lpg'
Description-Content-Type: text/markdown

# xbrlkit

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Work with XBRL filings above [Arelle](https://arelle.org): fetch a filing, parse
it **once** into a neutral typed model, and project that model into whichever
portable representation you need.

```
EDGAR ──▶ Arelle ──▶ XbrlModel ──┬──▶ holon.jsonld   (RDF / JSON-LD)
                                 ├──▶ Tavi           (compiled model)
                                 ├──▶ xBRL-JSON      (OIM)
                                 └──▶ …
```

Arelle stays the parser — nobody should reimplement DTS resolution. What it does
not give you is anything ergonomic to *hold*: `ModelXbrl` is a large mutable
object graph tied to a controller you have to close. `XbrlModel` is the answer to
that — stateless, single-filing, lossless, and the waist every projection hangs
off.

**The one architectural rule:** everything goes through `XbrlModel`. A feature
that reaches into Arelle's `ModelXbrl` directly is bypassing the waist, and that
is the change that turns a kit into a junk drawer.

## Projections

| Target | Status | Notes |
| --- | --- | --- |
| **holon** (`.holon.jsonld`) | shipped | RDF/JSON-LD, renders in the [Holon Viewer](https://holon.robosystems.ai/) |
| **Tavi** (`.tavi.json`) | shipped | [Project Tavi](https://www.xbrl.org/Specification/tavi/PWD-2026-09-01/tavi-PWD-2026-09-01.html) compiled model, PWD-2026-09-01 |
| **OIM** (`.oim.json`) | shipped | xBRL-JSON, checked fact-for-fact against Arelle's own writer |
| **property graph** (`.lbug`, parquet) | shipped | the [RoboSystems](https://robosystems.ai) `sec` graph's tables, ids and DDL, as one LadybugDB file per filing; row-identical to the platform's own processor on a 26-filing corpus |

The OIM projection is the one with a **released reference implementation** to
check against: Arelle's `saveLoadableOIM` writes the same document from the
same filing. A second writer is redundant as a feature — its value is that
every difference is a fidelity bug in the parse or the model, and those same
bugs are otherwise silent in the holon output, which has nothing to check it.
Current parity is every fact on 3M FY2024 (3,150) and Boeing FY2024 (2,688),
and all but one on Microsoft FY2024 (1,855 of 1,856); footnotes are the one
construct the model does not carry.

Tavi is a **public working draft** and its name is explicitly a working title,
so treat that projection as tracking a moving target. It has been diffed,
object class by object class, against the compiled model Arelle's unreleased
`XbrlModel` plugin ([Arelle PR #2418](https://github.com/Arelle/Arelle/pull/2418))
writes for 3M FY2024; the two agree on every fact outside that plugin's own
defects and on every cube. Where the draft left a choice open, the choice and
its reason are recorded in `SPEC_AMBIGUITIES` and carried in the
`.tavi.gaps.json` sidecar `--format tavi` writes alongside the document — the
sidecar also records what the filing carries that the model has nowhere to put,
and that file is the point of the projection, not a by-product of it.

## Property graph

`xbrlkit build --format lpg` (with the `lpg` extra: `pip install "xbrlkit[lpg]"`)
writes the filing as a single-file [LadybugDB](https://github.com/LadybugDB/ladybug)
database with the tables the RoboSystems `sec` graph is built from — the same
node labels, relationship types, columns and ids, declared once in
`xbrlkit.schema` — so Cypher written against the shared graph runs on the file
and a fact in either is the same row. What the platform adds after projection
is not in the file: text blocks stay inline in `Fact.value`, and the enrichment
columns and tables (`canonical_concept`, `canonical_type`, `FactSet`,
`Classification`) are empty. The projection is checked row for row against the
platform's own processor on the Filing Ladder's 26-filing corpus; the two
explained differences are association ids (random on the platform, derived
from the arc here) and exact duplicate arcs inside Arelle's aggregate
`XBRL-dimensions` network, which the derived ids collapse.

```python
from xbrlkit.serialize import to_graph_tables, write_parquet, build_lbug

tables = to_graph_tables(model)          # node and relationship rows, schema order
write_parquet(tables, Path("out/mmm"))   # nodes/*.parquet, relationships/*.parquet
build_lbug(tables, Path("out/mmm.lbug")) # CREATE TABLE … + COPY FROM, one file
```

## Text

`xbrlkit.text` reads the filing's primary HTML document — no Arelle, no
network — and returns its text as sections:

| Parser | Sections | Notes |
| --- | --- | --- |
| `iXBRLParser` | every inline-XBRL text block (notes, policies, tables), with the XBRL element names it contains | `ix:continuation` chains resolved; nested continuations and nested text blocks included |
| `NarrativeExtractor` | the 10-K / 10-Q Items — Business, Risk Factors, Cybersecurity, Properties, MD&A, Market Risk | table-of-contents rows and cross-references rejected; a 10-Q's Part I and Part II Items kept apart |

Both render HTML tables as markdown pipe tables and split a long section into
balanced parts at paragraph boundaries (`part`, `part_count`, and a `label`
like `"MD&A (2/6)"`) instead of truncating it. Measured on a 26-filing corpus
of 2024–2025 10-Ks and 10-Qs: every text block's full text is carried, where a
map of outermost continuations alone lost 15–29% of the note text on nine of
the filings, and every target Item starts at its body heading.

```python
from xbrlkit.text import iXBRLParser, NarrativeExtractor

html = open("mmm-20241231.htm").read()
for s in iXBRLParser().parse(html):
  print(s.section_id, s.label, s.word_count, s.xbrl_elements[:3])
for s in NarrativeExtractor().extract(html, form_type="10-K"):
  print(s.section_id, s.label, s.word_count)
```

## Install

### As a package

```bash
pip install xbrlkit
```

Exposes the `xbrlkit` CLI (`xbrlkit build …`, `xbrlkit fetch …`, `xbrlkit query …`)
and the library — use this to consume it from another project. Set your SEC
User-Agent via the environment (see [SEC User-Agent](#sec-user-agent)).

### From source (development)

```bash
# Install the toolchain
brew install uv just

# Install dependencies and provision .env from the template
just install
```

`just install` creates `.env` from `.env.example` on first run — then set your
SEC User-Agent in it.

## SEC User-Agent

SEC EDGAR requires a descriptive `User-Agent` on every request, or it throttles
you (empty responses / HTTP 429). `just install` already created your `.env` —
set your details there:

```bash
# .env
SEC_GOV_USER_AGENT="Your Name your@email.com"
```

`.env` is loaded automatically by every command. Outside the `just` workflow,
`export SEC_GOV_USER_AGENT="Your Name your@email.com"` or pass `--user-agent`.

## Usage

```bash
# Build a holon.jsonld from a specific filing (-> ./output/)
xbrlkit build --cik 320193 --accno 0000320193-23-000106

# Fetch the latest filing for a ticker (-> ./output/)
xbrlkit fetch --ticker NVDA

# Query consolidated facts in a built holon (in-memory SPARQL)
xbrlkit query --in output/0000320193-23-000106.holon.jsonld --element us-gaap:Assets
```

From a source checkout, `just` wraps the same CLI as a shorthand:
`just build 320193 0000320193-23-000106` and `just fetch NVDA`.

## View & explore

Built holons render in the **RoboSystems Holon Viewer** — a browser-based reader
that renders the financial statements and lets you ask questions of the report
with AI:

- **Hosted:** <https://holon.robosystems.ai/> — open a `holon.jsonld` and explore
  the statements, notes, and dimensional facts, or chat with the report.
- **Source:** <https://github.com/RoboFinSystems/robosystems-holon-viewer> — run
  it locally or self-host.

The viewer reads a holon entirely client-side, so a single `holon.jsonld` is a
complete, portable, self-describing report.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

MIT © 2026 RFS LLC
