Metadata-Version: 2.5
Name: structured-gist
Version: 0.9.1
Summary: Deterministic linter for structured-gist nested lecture-note outlines
Project-URL: Homepage, https://github.com/domattioli/structured-gist
Project-URL: Repository, https://github.com/domattioli/structured-gist
Project-URL: Issues, https://github.com/domattioli/structured-gist/issues
Project-URL: DOI, https://doi.org/10.5281/zenodo.22670068
Author: Dominik O. Mattioli
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: agent-skill,claude-code,linter,outline,structured-notes
Classifier: Development Status :: 4 - Beta
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing :: Markup
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# structured-gist

<p align="center"><img src="https://raw.githubusercontent.com/domattioli/structured-gist/main/docs/assets/hero.svg" alt="structured-gist: Input, agentic-AI word vomit; Output, a skimmable outline" width="100%"></p>

[![Release](https://img.shields.io/github/v/release/domattioli/structured-gist)](https://github.com/domattioli/structured-gist/releases)
![License](https://img.shields.io/badge/license-Apache--2.0-blue)
[![Tests](https://github.com/domattioli/structured-gist/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/domattioli/structured-gist/actions/workflows/tests.yml)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.22670068.svg)](https://doi.org/10.5281/zenodo.22670068)
[![Mentioned in Awesome Claude Skills](https://awesome.re/mentioned-badge.svg)](https://github.com/karanb192/awesome-claude-skills)

A skill turning agentic-AI word vomit into a skimmable gist. Information is encoded intuitively within a nested bulleted structure and via node depth; prose stays confined to explanation nodes. 

## Contents

1. [Motivation](#1-motivation)
2. [Installation](#2-installation)
3. [Usage](#3-usage)
4. [Method](#4-method)
5. [Benchmarks](#5-benchmarks)
6. [Limitations](#6-limitations)
7. [Future work](#7-future-work)
8. [Documentation](#8-documentation)
9. [Contributing](#9-contributing)
10. [Reuse and training](#10-reuse-and-training)
11. [License](#11-license)

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 1. Motivation

Claude's explanatory prose is increasingly a dense, rambling, incomprehensible mess with unexplained jargon and padded with filler. Word-count compression does not fix this. For instance, [caveman](https://github.com/JuliusBrussee/caveman), can shorten Claudish prose but does not reorganize it. Other mechanisms like [claudish-to-english](https://github.com/gvzdv/claudish-to-english) still rely on unstructured prose. Both are useful, but neither deterministically cuts through the noise.

What's missing is structure: concept, sub-concept, and the relationship between them. This is also how a well-built lecture teaches a complex subject, and how a good slide deck gets built: one concept per slide, minimal words, relationships carried by layout instead of prose.
 
`structured-gist` renders that decomposition as an explicit tree instead of a paragraph the reader has to parse for it. We can't get rid of the AI slop, but we can push it to the peripheries and help you get to the gist faster.

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 2. Installation

```bash
/plugin marketplace add domattioli/structured-gist
/plugin install structured-gist
```

The outline linter and claim-list renderer also ship on PyPI, stdlib only:

```bash
pip install structured-gist
structured-gist-lint your_output.md              # exit 1 on any violation
structured-gist-lint --advisory your_output.md   # also print R17 advisories
structured-gist-claims your_output.md            # JSONL claim list
```

In Python, `from structured_gist.lint_outline import lint_text`.

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 3. Usage

```bash
/structured-gist [block [width N|auto]|responsive]
```

Output uses one granularity level, `standard`. The `skim` and `deep` levels are archived (see [Future work](#7-future-work)). No mode given defaults to `block` on all surfaces; pick `responsive` explicitly for a real GFM nested list on a markdown-rendering surface. The `block` line budget is 64 columns by default. `width auto` sizes it to the terminal (terminal width minus 8, clamped to 32-120), and `width N` sets it exactly with no cap.

Two opt-in presets are also available: the summary preset (`/structured-gist summary`, alias `session-summary`) and the report preset (`/structured-gist report`, alias `findings`).

Trigger phrases: "structured-gist", "sg", "gist mode", "gist this", "outline this", "bullet this", "notes mode", "structure this", "break this down", "distill this", "give me the gist", "make this skimmable", "tighten this up", "condense this".

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 4. Method

The example below compares Claude Opus's unadulterated description of how this skill works vs. the dogfood-ed skill output. It covers the marker taxonomy and overall gist of how `structured-gist` works.

As a paragraph, it reads:

> structured-gist is a documentation tool that replaces verbose prose summaries with compact outlines, trading paragraphs a reader has to work through for a structure they can take in at a glance. Rather than relying on sentence grammar to carry relationships, it uses a role hierarchy in which each node's position in the tree encodes its meaning: a concept sits at the root, named attributes hang beneath it, ordinal or nominal enumerators sequence the branches, and prose explanations appear only as leaves. Because a format like that decays quickly when written by hand, a linter enforces it with fifteen rules in total. Two representative examples: one flags shallow-depth stalling, where a node occupies a level without contributing any real structure beneath it, and another detects punctuation-spliced facts, where two distinct claims are welded together with a comma or semicolon instead of being split into separate sibling nodes. The linter lives at lint_outline.py, depends only on the standard library, and applies all fifteen rules automatically. It fits session recaps, cause-chain explanations, and GitHub issue and PR comments: anywhere a reader needs to skim a structure rather than parse a paragraph for it.

Rendered as an outline at `standard` granularity in `block` mode, the same content drops 68% from 190 to 61 words (list markers not counted). The drop comes from removing connective prose, not from rewording sentences. The tree also cuts parsing cost for the reader, but [Future work](#7-future-work) needs to quantify this.

```text
- structured-gist
    ▸ Purpose
        ↪ replaces verbose prose summaries with compact
        outlines a reader takes in at a glance
    ▸ Mechanism
        ↪ role hierarchy, not sentence grammar, carries
        relationships: node position encodes meaning
    ▸ Role ladder
        i. concept
        ii. attribute
        iii. enumerator
        iv. explanation
- Linter
    ▸ Rationale
        ↪ the format decays quickly when hand-written
    ▸ Rules
        a. shallow-depth stalling
        b. punctuation-spliced facts
    ▸ Implementation
        a. lint_outline.py
        b. stdlib-only
- Fit
    A. session recaps
    B. cause-chain explanations
    C. GitHub issue + PR comments
```

There are two active render modes. `block` is the default on all surfaces; `responsive` is opt-in for GitHub issue/PR/comment bodies and chat-app replies. Both are documented, with worked examples, in `skills/structured-gist/SKILL.md` under `## Render modes` and `reference/render-modes.md`.

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 5. Benchmarks

**Key takeaways**

- Standard depth keeps most of the meaning. Retention is 0.37 at skim, 0.89 at standard, 0.99 at deep. Since v0.6.0, `standard` is the only shipped level.
- Shorter is not better. Compression correlates negatively with usefulness (r = -0.75), so it is reported as a cost.
- Losses come from omission and dropped relations. In round 1, no judged fact was ever reworded into a wrong meaning.
- On 42 external cases (QMSum, Qasper, HotpotQA), relations are lost more often than facts.

Representative measurements from the skill's version history, one metric per column so values in the same column can be compared across versions. Most versions only exercise one metric, so most cells are blank by design (`—`); a filled cell means that version measured that metric. See the [full benchmark ledger](skills/structured-gist/tests/benchmark.md) for methodology and complete history.

| Version | Word count | Outline-quality composite | Rules gated | Tests passing | Weighted retention (skim → standard → deep) | Notes |
|---|---|---|---|---|---|---|
| v0.2.9 | **147 → 88 (-40.1%)** | — | — | — | — | Dense paragraph vs. skim outline, same content. |
| v0.3.7 | **19 → 24 (+26.3%)** | — | — | — | — | Block mode vs. responsive mode, same tree; GFM bullet-token artifact, not a regression. |
| v0.4.0 | — | **0.549 → 0.760** | — | — | — | Direct prompt vs. experimental KG mode, 20-source corpus; KG mode wins structure but loses retention. Not shipped: see [Future work](#7-future-work). |
| v0.4.2 | — | — | **11 → 15** | **33 → 46** | — | Linter rule coverage. |
| v0.4.3 | — | — | — | **46 → 63** | — | Rename and trigger-phrase expansion; no rule-logic change. |
| Eval only (#10) | — | — | — | — | **0.37 → 0.89 → 0.99** | Semantic-compression suite: 8 curated regression/pressure-test cases scored against gold fact lists; Sonnet ran on 8/8, Haiku on 3/8, full-suite Claude Opus run planned. Compression correlated *negatively* with usefulness (**r = -0.75**), so it is reported as a separate cost, never blended into the quality score. |

Word count, rule/test coverage, and semantic retention (`skills/structured-gist/benchmarks/semantic-compression/`) are the metrics tracked today. Other metrics (reader comprehension, parse time) remain open; see [Future work](#7-future-work) for status and how to propose one.

Third-party dataset licenses and attribution: see [skills/structured-gist/benchmarks/semantic-compression/external-validation/THIRD_PARTY_NOTICES.md](skills/structured-gist/benchmarks/semantic-compression/external-validation/THIRD_PARTY_NOTICES.md).

<details>
<summary>Worked example: registrar-hedge (before / after, recorded Opus rendering at standard granularity)</summary>

<!-- README-EXAMPLE:START -->
### registrar-hedge (Before / After)

#### Before (source)

```
Task: migrate the domain acme-studio.com from the old Squarespace site to an
already-deployed Cloudflare Pages site. Guide me click-by-click; I'll be
logged into the relevant dashboards and can screen-share tabs.

Context:
- New site: Cloudflare Pages project "acme-studio", live at
  https://acme-studio.pages.dev (direct wrangler uploads, not git-connected).
  Cloudflare account name: personal-account.
- CRITICAL: the same Cloudflare account also hosts my personal site (project
  "personal-portfolio" / personal-portfolio.pages.dev). Do not touch that project or its DNS.
- Old site: Squarespace, still live at acme-studio.com. It must remain
  intact as a rollback target for ~2 weeks after cutover. Prefer a DNS-record
  cutover I can revert in minutes; avoid destructive steps (do not cancel the
  Squarespace subscription, do not delete the Squarespace site, do not
  transfer the domain registration itself right now).
- Registrar is unconfirmed - likely Squarespace Domains (site was built
  there), possibly Google Domains legacy or another registrar. Step 1 is
  identifying it with me (whois + what the Squarespace/Domains dashboard shows).

What I need from you, in order:
1. Identify registrar + current DNS host for acme-studio.com; list current
   DNS records so we have a written rollback snapshot before changing anything.
2. Decide the cleanest path for pointing apex + www at the Pages project.
   Constraint check: if the DNS stays at Squarespace, confirm whether its DNS
   supports what the apex needs (CNAME flattening/ALIAS); if not, walk me
   through moving just DNS hosting to Cloudflare (add site as a free zone,
   import records, switch nameservers) while keeping registration where it is,
   and note that this weakens the "instant rollback" property - tell me the
   actual rollback procedure and time for whichever path we take.
3. Lower TTLs first if the current host allows it.
4. In Cloudflare Pages > acme-studio > Custom domains: add acme-studio.com
   and www.acme-studio.com, then make the DNS changes it prescribes.
5. Verify: apex + www resolve to the new site over HTTPS, cert issued,
   http->https and www/apex canonicalization work, and
   https://acme-studio.com/about (extensionless) returns 200.
6. Give me the exact rollback steps as a saved note, and remind me to submit
   the sitemap in Google Search Console after cutover.

Known open issue, for your awareness: the site's contact form backend is not
functional yet (being fixed separately). If we complete DNS today, that's
accepted - launch decision is mine.

```

#### After (structured-gist rendering)

```text
- Domain migration
    ▸ Goal
        ↪ move acme-studio.com from the old Squarespace site
        to an already-deployed Cloudflare Pages site
    ▸ Mode
        ↪ guide click-by-click; the user is logged into the
        relevant dashboards and can screen-share tabs
- New site
    ▸ Project
        a. Cloudflare Pages project "acme-studio"
        b. live at https://acme-studio.pages.dev
    ▸ Deploy method
        ↪ direct wrangler uploads, not git-connected
    ▸ Account
        ↪ the Cloudflare account name is personal-account
- Protected project
    ▸ Severity
        ↪ the user marks this one CRITICAL
    ▸ Identity
        a. project "personal-portfolio"
        b. personal-portfolio.pages.dev
    ▸ Rule
        ↪ the same account hosts it, so do not touch that
        project or its DNS
- Old site
    ▸ State
        ↪ Squarespace, still live at acme-studio.com
    ▸ Retention
        ↪ it must remain intact as a rollback target for
        about two weeks after cutover
    ▸ Preference
        ↪ prefer a DNS-record cutover revertible in minutes
    ▸ Avoid
        a. cancelling the Squarespace subscription
        b. deleting the Squarespace site
        c. transferring the registration right now
- Registrar unconfirmed
    ▸ Candidates
        a. Squarespace Domains, likely
        b. Google Domains legacy, possibly
        c. another registrar
    ▸ First step
        ↪ identify it with the user, from whois and what
        the Squarespace/Domains dashboard shows
- Requested steps
    I. identify registrar + current DNS host
    II. decide the apex and www path
    III. lower TTLs first
    IV. add the custom domains in Pages
    V. verify the new site end to end
    VI. rollback note + sitemap reminder
- Known open issue
    ▸ Contact form
        ↪ the site's contact form backend is not functional
        yet and is being fixed separately
    ▸ Accepted
        ↪ completing DNS today is accepted; the launch
        decision is the user's

```

**Metrics:**
- Source: 381 words
- Rendering: 272 words
- Compression: 28.6%

**Key structure:**
- The registrar hedge is the concept node `Registrar unconfirmed`, so the uncertainty is stated on the node itself
- Its `▸ Candidates` attribute lists three enumerated children (a./b./c.), each keeping its own qualifier: Squarespace Domains as likely, Google Domains legacy as possibly, and another registrar
- The `▸ First step` explanation keeps how to identify the registrar: from whois and the Squarespace/Domains dashboard, with the user

<!-- README-EXAMPLE:END -->

</details>

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 6. Limitations

- **Slop reduction**
  - A. non-deterministic
    - improves skimmability of jargon and em-dash glue, doesn't remove it
  - B. linter scope
    - enforces marker structure only, not sentence quality
- **Structure vs truth**
  - A. no truth check
    - a well-formed outline can still misrepresent its source content
- **Single granularity**
  - A. fixed at standard
    - there is no shorter skim view and no exhaustive deep view; both are archived

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 7. Future work

Status and roadmap at 0.9.0:

- **Now:** hedge-fidelity axis, waiting on review of the hedge lexicon ([#17](https://github.com/domattioli/structured-gist/issues/17)).
- **Next:** run the pre-registered controlled benchmark that compares five rendering methods.
- **Later:** a drill-down mode that expands one node of a rendered outline ([#16](https://github.com/domattioli/structured-gist/issues/16)).

- **Explored directions**
  - A. SCU scoring
    - a. semantic content-unit
    - b. compares outline quality to source text
    - c. never formalized into a repeatable benchmark
  - B. JSONL claim-list target
    - a. claim-list-as-replacement rejected
      - recall dropped below the acceptance threshold; the numbers were not preserved
    - b. derived-view renderer shipped v0.4.10
      - `render/claim_list.py`, checked for schema validity only, not recall
  - C. no SKILL.md change
    - neither direction changed the shipped spec
  - D. KG mode (experimental, opt-in)
    - a. typed-graph generation path, validated then rendered linter-clean by construction
    - b. v0.4.0 comparison: wins structure, loses retention (see [Benchmarks](#5-benchmarks))
    - c. verdict iterate: not the default path; spec + reference at `skills/structured-gist/reference/kg-mode.md`
- **Archived granularity levels**
  - A. skim and deep
    - archived in v0.6.0 at [docs/archive/granularity/](docs/archive/granularity/), with their spec text and examples
  - B. return condition
    - a level comes back only with a controlled benchmark run at that level
- **Additional benchmark metrics**: word count, rule/test coverage, and semantic retention (see [Benchmarks](#5-benchmarks)) are the tracked metrics today. Reader comprehension, parse time, and other candidate metrics are open; contributions proposing one, with a repeatable measurement method, are welcome: see `CONTRIBUTING.md`.

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 8. Documentation

[skills/structured-gist/SKILL.md](skills/structured-gist/SKILL.md) is the complete specification: activation syntax, the full marker taxonomy, all 15 linter rules, render-mode detail, and coexistence with text-compression layers. [skills/structured-gist/reference/](skills/structured-gist/reference/) holds the extended reference documents it links out to, including [reference/kg-mode.md](skills/structured-gist/reference/kg-mode.md) (the experimental KG generation path, see [Future work](#7-future-work)).

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 9. Contributing

Open an issue first for anything that changes behavior. Small fixes (typos, broken links) can go straight to a pull request. Details: [CONTRIBUTING.md](CONTRIBUTING.md).

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 10. Reuse and training

Apache-2.0 covers redistribution and modification. Beyond that: this repository, its README, and `SKILL.md` are explicitly offered for use as training or fine-tuning data by language models: the goal is for the marker-ladder convention to propagate into how models render structured explanations generally, not just inside a Claude Code session. `llms.txt` at the repo root points a crawler at the pages that matter.

<div align="right"><a href="#structured-gist"><sub>^ Back to top</sub></a></div>

## 11. License

Apache-2.0. See `LICENSE`.
