Metadata-Version: 2.4
Name: fiona-assistant
Version: 1.2.0
Summary: Tools for managing the LLM wiki.
Author: nvaldeziii
License-Expression: GPL-3.0-only
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pymarkdownlnt>=0.9.0
Requires-Dist: ruff>=0.16.8
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: GitPython>=3.1.0
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: markdownify>=1.1.0
Requires-Dist: pypdf>=5.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pylint; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# Fiona-Assistant

> Your LLM agent writes the wiki. Fiona keeps it clean.

Fiona-Assistant is a command-line toolkit for running an **llm-wiki**: a
folder of interlinked markdown that an AI agent builds and maintains for
you. You bring sources (web pages, PDFs, notes) and questions. The agent
compiles them into wiki pages. Fiona does every deterministic part of the
job: it scaffolds the wiki, ingests and archives sources, lints markdown,
frontmatter, Python, and JSON, checks links, draws a link graph, versions
everything in git, and tells the agent exactly what to fix next.

The result: the agent spends its tokens on reading and writing, not on
listing files, chasing broken links, or reformatting tables.

```bash
python -m pip install fiona-assistant
fiona-assistant wiki init my-wiki
cd my-wiki
fiona-assistant wiki ingest https://example.com/article.html
fiona-assistant housekeeping
```

---

## Table of Contents

- [Why Fiona](#why-fiona)
- [Features](#features)
- [Quick Start](#quick-start)
- [How It Works](#how-it-works)
- [Command Reference](#command-reference)
- [Output Built for Agents](#output-built-for-agents)
- [Roadmap](#roadmap)
- [Development](#development)
- [Documentation](#documentation)
- [License](#license)

---

## Why Fiona

An LLM agent can maintain a knowledge base, but left alone it wastes
context on chores and slowly breaks things: dead links, missing
frontmatter, wrapped table rows, orphan pages, overwritten sources.

Fiona fixes that split of work:

| The agent does | Fiona does |
| --- | --- |
| Reads sources and writes synthesized pages | Fetches, archives, and converts sources to markdown |
| Decides what links to what | Finds dead links, dead wikilinks, and orphan pages |
| Refines raw pages | Lints markdown, frontmatter, Python, and JSON |
| Answers your questions from the wiki | Commits, snapshots, and restores wiki history |
| Merges template updates | Detects template updates and never overwrites your files |

> Use scripts for deterministic tasks. Reserve AI tokens for creative and
> analytical work.

---

## Features

### Wiki in one command

`fiona-assistant wiki init` creates a ready-to-use llm-wiki: directory
layout, settings, agent rules (`AGENTS.md`), page schema, index, log, and
bundled agent skills. It also creates a git repository and makes the
first commit, with `personal/` kept out of version control.

### Source ingestion

`fiona-assistant wiki ingest <url-or-file>` turns a source into a wiki page
in one step:

- Downloads the URL or reads the local file
- Archives an untouched copy under `raw/` (duplicates are detected by
  SHA-256, so the same bytes are never stored twice)
- Converts it to markdown (HTML, PDF, plain text, and markdown ship today)
- Writes a page under `wiki/` with full provenance in its frontmatter and
  `status: raw`, so the agent knows it still needs refining
- Appends the action to `wiki/log.md`

`wiki ingest --pending` lists every page still waiting to be refined.

### One-call maintenance

`fiona-assistant housekeeping` runs the whole maintenance routine and
prints a to-do list for the agent:

- **check-update**: is the packaged template newer than this wiki?
- **merge**: which `*-update.md` files are waiting to be merged?
- **ingest**: which pages are still `status: raw`?
- **sanity**: which lint checks fail, and where?
- **graph**: regenerates `wiki_graph.html` and lists orphan pages

It only reads, except for writing the graph. Every finding comes with a
concrete "next step" the agent can act on. Add `--json` for machine use.

### Quality checks

`fiona-assistant wiki sanity` runs five checks in parallel and gives one
pass/fail verdict:

| Check | What it catches |
| --- | --- |
| `markdownlint` | Markdown style, via PyMarkdown |
| `frontmatter_lint` | Missing or malformed YAML frontmatter (`name`, `version`, and more) |
| `python-ruff` | Python issues in wiki scripts, via Ruff |
| `mdlinter` | Dead links, dead `[[wikilinks]]`, and table rows split across lines |
| `jsonlint` | JSON syntax, indentation, trailing commas, duplicate keys |

Each linter is also available on its own, and `mdoctor fix` and
`json-lint fix` repair what they can automatically.

### Interactive link graph

`fiona-assistant wiki graph` writes a single self-contained HTML file
that shows how your pages link to each other. Nodes are scored by how
many different pages point at them, so hub pages stand out. It
understands markdown links and `[[wikilinks]]`, shows external URLs, and
flags broken links and orphans. `--format json` exports the same graph
for other tools.

### Safe template updates

As Fiona improves, so does its wiki template. `wiki check-update` shows
what changed, and `wiki update` brings your wiki up to date without ever
overwriting a file you own. When both sides changed, Fiona writes
`<name>-update.md` next to your file and `housekeeping` tells the agent
to merge it.

### Git history built in

`fiona-assistant git` wraps the operations an agent needs, with
timestamped commit messages:

- `save` commits all current edits
- `snapshot <name>` commits and tags a restore point
- `restore <name>` returns to a snapshot without losing history
- `undo`, `clean`, and `status` for everyday recovery

### Guardrails

- `personal/` and `raw/` are read-only for every automated tool. Ingest
  may add new files to `raw/`; it never changes an existing one.
- Commands that need a decision prompt on a terminal and accept
  `--yes`, `--force`, or `--non-interactive` for unattended runs.
- Respects `.gitignore` when scanning.

### Table of contents generation

`fiona-assistant librarian toc` builds `table_of_contents.md` for a folder
tree. `librarian toc file <path>` keeps an in-file TOC under the H1 up to
date, with optional section numbering (`--sectionized`).

---

## Quick Start

### 1. Install

Requires Python 3.11 or newer.

```bash
python -m pip install fiona-assistant
```

### 2. Create a wiki

```bash
fiona-assistant wiki init my-wiki
cd my-wiki
```

### 3. Point your agent at it

Open the folder in your AI coding agent (Cursor, Claude Code, or any
agent that reads `AGENTS.md`). The bundled `AGENTS.md` loads the Fiona
skill, which teaches the agent the ingest, refine, and housekeeping
workflow and when to call each command.

### 4. Feed it sources

```bash
fiona-assistant wiki ingest https://example.com/post.html
fiona-assistant wiki ingest ./notes/meeting.pdf
```

Then ask the agent to refine the new pages.

### 5. Keep it healthy

```bash
fiona-assistant housekeeping
fiona-assistant git save --description "ingest meeting notes"
```

---

## How It Works

A wiki created by Fiona has this layout:

```text
my-wiki/
├── AGENTS.md              # Entry point for the agent; loads the Fiona skill
├── fiona-settings.json    # Marks the wiki root; tool settings
├── ingest/                # Drop zone for files waiting to be parsed
├── raw/                   # Immutable archive of original sources
│   ├── articles/
│   ├── papers/
│   ├── transcripts/
│   ├── assets/
│   └── other/
├── wiki/                  # Pages the agent writes and maintains
│   ├── schema.md          # Page types, tags, conventions
│   ├── index.md           # Content catalog
│   ├── log.md             # Append-only action log
│   ├── concepts/
│   ├── entities/
│   ├── comparisons/
│   ├── queries/
│   └── sources/
├── personal/              # Your own notes (read-only for agents, untracked)
└── utils/                 # Agent skills, personas, and helper scripts
```

The cycle is:

- **Ingest**: `wiki ingest` archives a source in `raw/` and writes a raw
  page in `wiki/`.
- **Refine**: the agent rewrites the page, links it to related pages, and
  sets `status: refined`.
- **Maintain**: `housekeeping` reports what needs attention; the agent
  fixes it.
- **Save**: `git save` records the change.

The full page schema lives in the template's
[schema](./src/fiona/data/llm_wiki/wiki/schema.md).

---

## Command Reference

Run `fiona-assistant --help` or `fiona-assistant <command> --help` for all
flags. Commands that take an optional directory default to the current
directory.

### Wiki lifecycle

| Command | What it does |
| --- | --- |
| `wiki init [dir]` | Create a new llm-wiki with git history |
| `wiki ingest <source>` | Fetch, archive, and convert a URL or file into a raw page |
| `wiki ingest --pending` | List pages still marked `status: raw` |
| `wiki sanity [dir]` | Run every quality check; exit 1 on failure |
| `wiki graph [dir]` | Write `wiki_graph.html` (or JSON with `--format json`) |
| `wiki check-update [dir]` | Show what a template update would change |
| `wiki update [dir]` | Apply a template update (`--dry-run`, `--yes`) |
| `housekeeping [dir]` | Run the full maintenance routine and list next steps |

### Linters

| Command | What it does |
| --- | --- |
| `mdoctor chklint [dir]` | Markdown lint via PyMarkdown |
| `mdoctor fix [dir]` | Auto-fix markdown lint violations |
| `fmlint chklint [dir]` | Frontmatter lint |
| `python-ruff chklint [dir]` | Python lint via Ruff |
| `json-lint chklint [dir]` | JSON lint |
| `json-lint fix [dir]` | Auto-fix JSON indentation and trailing commas |

### Files and TOC

| Command | What it does |
| --- | --- |
| `librarian list [dir]` | List files in a directory |
| `librarian toc [dir]` | Write `table_of_contents.md` (`--recursive-toc`, `--exclude REGEX`) |
| `librarian toc file <path>` | Write an in-file TOC below the H1 (`--sectionized`) |
| `librarian toc file '*'` | Update in-file TOCs for every markdown file in the wiki |

### Git

| Command | What it does |
| --- | --- |
| `git save --description TEXT` | Commit all edits |
| `git snapshot <name> --description TEXT` | Commit and tag a restore point |
| `git restore <name>` | Restore a snapshot as a new commit |
| `git undo` | Drop the last commit |
| `git clean` | Discard uncommitted edits |
| `git status` | Show whether the tree is clean |

Every command above is run as `fiona-assistant <command>`. The
`librarian` and `mdoctor` tools are also installed as standalone
commands.

---

## Output Built for Agents

Every report uses the same shape: a header, the target, rule
definitions, findings per file with line and column, and a summary. An
agent can parse it without guessing, and a human can skim it.

```text
Markdown Lint Violations
TARGET: wiki

- DEFINITIONS:
  - MDL001: Dead Link, Relative markdown link does not resolve to an existing file
  - MDL002: Dead Wiki Link, Wiki-internal link does not resolve to any md page in the wiki

- FILE: wiki/concepts/retrieval.md
  - MDL001 line 12: ../sources/old-paper.md does not exist
  - MDL002 line 30: [[embeddings]] does not resolve to any page

- SUMMARY:
  - Total files with violations: 1
  - Total violations: 2
```

`wiki sanity`, `housekeeping`, `wiki ingest`, `wiki check-update`, and
`wiki update` also accept `--json` for structured output.

---

## Roadmap

Planned work, tracked with requirements and acceptance criteria in
[SPEC](./SPEC.md) and prioritized in [todo](./todo.md):

| Feature | What it will do |
| --- | --- |
| Wiki root discovery | Run any command from a subfolder; Fiona finds the nearest `fiona-settings.json` |
| Backup | One command writes a compressed, restorable archive of the wiki outside the tree |
| `setup` command | Interactive review and editing of `fiona-settings.json`, with validation |
| Smarter graph | Show only important nodes until you expand a parent; hide gitignored pages |
| Code blocks in markdown | Lint Python and shell snippets inside fenced code blocks |
| Shellcheck | Lint shell scripts as part of wiki sanity |
| HTML and JavaScript lint | Validate HTML and JS files in the wiki |
| `python-ruff fix` | Auto-fix Python lint findings |
| More converters | Ingest more source formats beyond HTML, PDF, text, and markdown |
| Version tags for scripts | Track versions of Python and shell files like markdown pages |

---

## Development

### Set up

Clone the repository and load the environment with the bootstrap script.

Linux and macOS:

```bash
git clone <repository-url>
cd fiona-assistant
source bootstrap --env fiona_env -c
```

Windows (PowerShell):

```powershell
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
. .\bootstrap.ps1 --env fiona_env -c
```

The default install is editable (`pip install -e ".[dev]"`). On Windows
the environment lives in `.venvs\fiona_env`.

### Other install modes

| Goal | Bash | PowerShell |
| --- | --- | --- |
| Install this checkout as a normal package | `source bootstrap --local` | `. .\bootstrap.ps1 --local` |
| Same as `pip install fiona-assistant`, no venv | `source bootstrap --local --no-env` | `powershell -ExecutionPolicy Bypass -File .\bootstrap.ps1 --local --no-env` |

`--local --no-env` runs `python -m pip install --user .`. If
`fiona-assistant` is not found afterwards, add the user scripts
directory printed by bootstrap to `PATH`.

### Test and check

```bash
# Unit tests with coverage
python scripts/run_tests.py

# Full sanity: tests, coverage gate, markdown, Ruff, frontmatter, JSON
python scripts/run_sanity.py --non-interactive
```

Contributions follow the spec-first, test-first lifecycle in
[AGENTS](./AGENTS.md).

### Project structure

```text
fiona-assistant/
├── src/fiona/             # The installable package
│   ├── __main__.py        # fiona-assistant CLI
│   ├── librarian/         # File listing and TOC generation
│   ├── mdoctor/           # Markdown lint and fix
│   ├── customlinter/      # Frontmatter, markdown link, Python, JSON linters
│   ├── python_ruff/       # Ruff wrapper
│   ├── gittool/           # Git operations
│   ├── sanity_runner/     # Shared sanity engine
│   ├── wiki/              # init, update, sanity, graph, ingest, housekeeping
│   └── data/llm_wiki/     # Packaged wiki template
├── tests/                 # Unit tests (not installed)
├── scripts/               # Test and sanity runners, dev settings
├── bootstrap              # Environment setup (bash)
├── bootstrap.ps1          # Environment setup (PowerShell)
├── SPEC.md                # Requirements and acceptance criteria
└── pyproject.toml
```

---

## Documentation

- [Project Specification](./SPEC.md): every command, setting, and rule,
  with acceptance criteria
- [LLM Wiki Agent Rules](./src/fiona/data/llm_wiki/AGENTS.md): what a
  deployed wiki tells its agent
- [Fiona skill](./src/fiona/data/llm_wiki/utils/skills/fiona/SKILL.md):
  the workflow the agent follows inside a wiki
- [Development Agent Rules](./AGENTS.md): how to contribute to this
  repository
- [Roadmap](./todo.md): planned and in-progress work

---

## License

GPL-3.0-only
