Metadata-Version: 2.4
Name: sigil-obsidian
Version: 1.9.7
Summary: Schema enforcement and CRUD integrity layer for Obsidian vault folders
Author-email: Gert Schepens <gert@dgtl.be>
License: MIT License
        
        Copyright (c) 2026 Gert Schepens
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Repository, https://gitlab.com/DigitalGert/sigil
Project-URL: Issues, https://gitlab.com/DigitalGert/sigil/-/issues
Keywords: obsidian,schema,validation,agents,cli,markdown
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Text Processing :: Markup :: Markdown
Classifier: Topic :: Utilities
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer<1,>=0.12
Requires-Dist: rich<15,>=13
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: httpx<1,>=0.27
Requires-Dist: packaging>=20
Provides-Extra: dev
Requires-Dist: pytest<9,>=8.0; extra == "dev"
Requires-Dist: pytest-cov<6,>=5.0; extra == "dev"
Requires-Dist: ruff<1,>=0.4; extra == "dev"
Dynamic: license-file

# Sigil

> Schema enforcement and CRUD integrity layer for Obsidian vault folders — built for agents and humans.

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://gitlab.com/DigitalGert/sigil/-/blob/main/LICENSE)
[![PyPI](https://img.shields.io/pypi/v/sigil-obsidian)](https://pypi.org/project/sigil-obsidian/)
[![Python](https://img.shields.io/pypi/pyversions/sigil-obsidian)](https://pypi.org/project/sigil-obsidian/)

![Sigil](https://gitlab.com/DigitalGert/sigil/-/raw/main/logo_original.png)

---

*Sigil came to life when I saw the notification that the backend I was using for structured data was going to stop working. I realized that that old backend had never really worked decently at all. Half of the time it worked well, the rest of the time, all sorts of chaos happened. It annoyed me at every contact point, and ended up more of a nuisance than a help. Yet, it did fix an important point for me — I need structured data, where the agents working with it can't just forget about the structure and make a big inconsistent mess.*

*I am, like we all are, a big fan of Obsidian vaults, and on a friction level, those really work best for me. It's accessible, readable, and it has that exciting Bases view, that just looks like it'll do magic. Obsidian is spread over all my devices though, so the ample plugins weren't exactly an easy answer either.*

*And the answer is ultimately simple — I just need schema enforcement; an integrity layer for when the agents are writing to my Obsidian vault. (And if I make a mess, they can fix it too.) And I like building things 🙂 So I wrote a briefing, and half a day of agents running, we now have Sigil ready for all of us to escape the dread of having to use anything other than Obsidian. Point your agents here, and enjoy the structure! (And join us in dev and debugging, this is very much an alive project!)*

— **Gert Schepens**

---

```
$ sigil check projects/

✓ projects/ — 12 files checked

  ✗ acme-proposal.md
    · Status: missing (required)
    · Created: invalid format — expected ISO 8601, got "June 3"

  ✗ old-contact.md
    · Title: missing (required)

2 files with violations · 10 files clean
```

## Features

- **Schema-enforced CRUD** — `create`, `read`, `update`, `delete` all validate against the folder's schema. Constraints are enforced at write time — invalid input is rejected before anything is written to disk.
- **13 field types** — `string`, `text`, `integer`, `float`, `boolean`, `date`, `datetime`, `enum`, `url`, `email`, `list`, `tags`, `reference`
- **`sigil check`** — full folder audit with violation reports, proposed fixes, and `--fix` for auto-repair
- **`sigil move`** — inter-folder workflow pipeline; destination schema is the sole gate
- **`sigil migrate`** — evolve a schema and rewrite existing files in bulk: add fields, rename values, change required/optional
- **`sigil import`** — bulk-create from a JSON array or CSV file, with per-record constraint enforcement
- **`auto` defaults** — `"auto": "today"` or `"now"` fills date/datetime fields at create time without prompting
- **Final states** — mark enum values as terminal; `sigil read --final` and `sigil stats` surface them
- **Filename templates** — `"{Date:%y%j}_{Title.slug}"` generates consistent, sanitized filenames — including inline date format specs
- **`sigil stats`** — field completion counts, enum distributions, active/final breakdown
- **`sigil sources`** — multi-vault registry with `--health`, `--discover`, and `--prune`
- **`sigil logs`** — scan debug logs for command usage, error patterns, and agent vs human usage split
- **Obsidian Bases** — auto-generates a scoped `.base` view file on `sigil init`
- **Agent-ready JSON** — every command has `--json`; first daily call includes `_sigil` hints for command discovery

---

## Install

```bash
pip install sigil-obsidian
```

Requires Python 3.11+.

---

## Quick start

```bash
sigil init /path/to/vault/projects   # guided schema setup
sigil check projects/                # audit all files
sigil check projects/ --fix          # auto-repair violations
sigil schema projects/               # inspect the schema
sigil create projects/               # create a validated file
sigil read projects/ --json          # structured output for agents
```

---

## Commands

### Discovery and audit

```bash
sigil sources                        # list all known vaults and managed folders
sigil sources --json                 # machine-readable output
sigil sources --health               # add violation count per folder
sigil sources --discover             # find Sigil folders not yet registered
sigil sources --prune                # remove missing paths from config

sigil check <folder>                 # audit all files, list violations
sigil check <folder> --fix           # auto-apply non-destructive fixes
sigil check <folder> --json          # machine-readable output
sigil check <file>                   # validate a single file

sigil schema <folder>                # show required fields, types, options
sigil schema <folder> --json
sigil stats <folder>                 # counts, distributions, finals breakdown
sigil stats <folder> --json
sigil logs                           # scan debug logs — usage, errors, json/human split
```

### CRUD

```bash
sigil create <folder>                              # interactive field prompts
sigil create <folder> --data '{"Title": "..."}'    # pass fields directly
sigil create <folder> --schema invoices            # pick named schema (multi-schema folders)

sigil read <folder>                                # list all records
sigil read <folder> --final                        # only records in a final state
sigil read <folder> --json                         # machine-readable
sigil read <folder> <file>                         # single file

sigil update <folder> <file> --data '{"Status": "Done"}'
sigil update <folder> <file> --data '{}' --schema invoices   # pick named schema
sigil delete <folder> <file>
```

All write operations (`create`, `update`, `import`) enforce the full schema at the point of write — constraints, patterns, min/max, conditional rules, and uniqueness. Invalid data is rejected before anything touches disk.

### Bulk operations

```bash
sigil import <folder> records.json          # JSON array of objects
sigil import <folder> records.csv           # CSV with header row
sigil import <folder> records.json --skip-errors   # continue on invalid rows
```

### Workflows

```bash
sigil move <folder> <file>            # move file to linked workflow folder
```

Blocked if the file doesn't satisfy the destination schema. No state list needed — the destination schema is the sole gate.

### Schema evolution

```bash
sigil migrate add-field <folder> <name> --type <type> --default <value>
sigil migrate rename-value <folder> <field> <old> <new>
sigil migrate set-required <folder> <field>
sigil migrate set-optional <folder> <field>
```

Updates the schema and rewrites all existing files in bulk. `rename-value` also updates `finals` if the renamed value was terminal.

### Obsidian

```bash
sigil base <folder>                   # regenerate .base file (run after moving a folder)
sigil install-skill [<dir>]           # copy skill.md to your agent skills directory
sigil install-cron [<dir>]            # copy cron.md (scheduled integrity check)
```

---

## Schema file

Each managed folder has a `sigil.schema.md` file. The schema is a JSON block inside a Markdown file, so it's human-readable in Obsidian and machine-parseable by Sigil:

```markdown
---
sigil: https://gitlab.com/DigitalGert/sigil
sigil_version: 1.9.7
sigil_install: pip install sigil-obsidian
sigil_skill: https://gitlab.com/DigitalGert/sigil/-/raw/main/skill.md
---

This folder is managed by Sigil ...

\```json
{
  "filename": "{Date}_{Title.slug}",
  "ignore": ["README.md", "*.template.md"],
  "fields": {
    "Title":   { "type": "string",   "required": true,  "obsidian_property": true,
                 "purpose": "Short name for the record" },
    "Status":  { "type": "enum",     "required": true,  "obsidian_property": true,
                 "values": ["Idea", "Drafting", "Published", "Shelved"],
                 "finals": ["Published", "Shelved"],
                 "default": "Idea" },
    "Date":    { "type": "date",     "required": true,  "obsidian_property": true,
                 "auto": "today" },
    "Tags":    { "type": "tags",     "required": false, "obsidian_property": true },
    "Content": { "type": "text",     "required": false }
  },
  "workflow": {
    "links": "communications/published",
    "description": "When a draft reaches Status: Published, move it here with sigil move."
  }
}
\```
```

### Field options

| Option | Description |
|--------|-------------|
| `type` | Field type — see table below |
| `required` | `true` → violation if absent |
| `obsidian_property` | `true` → YAML frontmatter; `false` → `## FieldName` body section |
| `values` | Enum values array (required for `enum` type) |
| `finals` | Subset of `values` that represent a terminal/done state |
| `default` | Static default written at create time and by `check --fix` when absent |
| `auto` | `"today"` or `"now"` — auto-fills a date/datetime field at create time |
| `purpose` | Human/agent-readable description, shown by `sigil schema` |
| `pattern` | Regex the value must fully match — `string`, `text`, `url`, `email`, `reference` |
| `minLength` / `maxLength` | Character length bounds — string types |
| `min` / `max` | Numeric bounds (`integer`, `float`) or date bounds ISO string (`date`, `datetime`) |
| `minItems` / `maxItems` | Item count bounds — `list`, `tags` |
| `contains` | Values that must all be present — `list`, `tags` |
| `unique` | Value must be unique across all records in the folder |
| `warn` | Soft requirement — reported as `⚠ warning` instead of `✗ error` |

### Field types

| Type | Description |
|------|-------------|
| `string` | Single-line text |
| `text` | Multiline text — stored as `## FieldName` body section unless `obsidian_property: true` |
| `integer` | Whole number |
| `float` | Decimal number |
| `boolean` | `true` / `false` |
| `date` | ISO 8601 date — `YYYY-MM-DD` |
| `datetime` | ISO 8601 datetime — `YYYY-MM-DDThh:mm:ss` |
| `enum` | Fixed set of values — requires a `values` array |
| `url` | Valid URL (`http://` or `https://`) |
| `email` | Valid email address |
| `list` | Array of strings |
| `tags` | Obsidian-style tags — array |
| `reference` | Obsidian `[[wikilink]]` to another note |

### Filename templates

The `filename` key controls how new files are named:

```json
"filename": "{Date}_{Title.slug}"
```

| Token | Result |
|-------|--------|
| `{FieldName}` | Field value, filesystem-sanitized (strips `/\:*?"<>|`, preserves spaces and unicode) |
| `{FieldName.slug}` | Lowercase-hyphenated slug: `"My Great Post"` → `my-great-post` |
| `{DateField}` | Date fields render as `YYYY-MM-DD` by default, or per `date_format` in the field definition |
| `{DateField:strftime_format}` | Inline date format override — e.g. `{Date:%y%j}` renders as `26085` (two-digit year + day-of-year) |

Inline format specs let you derive a filename token from an existing date field without storing a separate derived field:

```json
"filename": "{Date:%y%j}_{Title}"
```

`Date: 2026-03-26` → filename prefix `26085_`. The format is applied only to the filename — the stored `Date` value is unchanged.

- Stems are truncated to 252 characters before `.md` is appended (filesystem limit)
- If a required token has no value, the rename is skipped rather than producing a broken name

### Ignore patterns

```json
"ignore": ["README.md", "*.template.md"]
```

Files matching any pattern are skipped in `sigil check`, `sigil read`, and `sigil stats`.

### Workflow

```json
"workflow": {
  "links": "communications/published",
  "description": "Approved drafts move here. Platform must be set."
}
```

Declares a pipeline to another folder. `sigil move <folder> <file>` validates the file against the destination schema — if it passes, the file moves; if not, violations are listed and nothing changes. No state list is maintained in the source schema — the destination schema is the sole gate.

### Cross-field constraints

```json
"cross_field": [
  {"field_a": "Start", "operator": "<", "field_b": "End", "message": "Start must be before End"}
]
```

Compares two field values using `==`, `!=`, `<`, `>`, `<=`, `>=`. Dates and numbers are coerced for comparison. Skipped if either field is absent. `message` is optional — a default is generated from the rule.

### Conditional validation

```json
"conditional": [
  {
    "when": {"field": "Status", "operator": "==", "value": "active"},
    "then": {"field": "Deadline", "required": true}
  }
]
```

When the `when` condition is met, the `then` clause overrides the named field's constraints for that record (`required`, `min`, `max`, `minLength`, `maxLength`, `pattern`, etc.). When the condition is not met, the field's validation is skipped entirely. Use this to express "Deadline is optional for drafts, required when active."

### Write-time constraint enforcement

All constraints defined in the schema are enforced at `sigil create`, `sigil update`, and `sigil import` — not only at `sigil check`. Writes that would produce invalid files are rejected before anything touches disk:

- Required field missing → error, write blocked
- Pattern mismatch, min/max violation, length bounds → error, write blocked
- Conditional rule triggered (e.g. PaidDate required when Paid is true) → error if not satisfied
- Unique constraint violated → error listing the conflicting file
- `warn: true` fields → violation printed as warning, write proceeds

---

## Multi-schema folders

> **Note:** one schema per folder is the primary and recommended pattern. Multi-schema is supported for cases where a single folder genuinely holds distinct record types that cannot be separated. Don't reach for it by default.

A single `sigil.schema.md` can contain multiple schemas, each with an optional `match` selector. Sigil applies the matching schema(s) per file during check, and you pick a schema by name at create/update time.

```json
{
  "schemas": [
    {
      "name": "invoices",
      "match": {"tags": ["invoice"]},
      "filename": "{Code}_{Title}",
      "fields": {
        "Code":   {"type": "string", "required": true, "obsidian_property": true,
                   "pattern": "INV-\\d{4}", "unique": true},
        "Title":  {"type": "string", "required": true, "obsidian_property": true},
        "Amount": {"type": "float",  "required": true, "obsidian_property": true, "min": 0}
      }
    },
    {
      "name": "notes",
      "match": {"tags": ["note"]},
      "fields": {
        "Title": {"type": "string", "required": true, "obsidian_property": true}
      }
    }
  ]
}
```

### Match selectors

Each schema can have an optional `match` object with `tags`, `file`, or both:

| Selector | Example | Behaviour |
|----------|---------|-----------|
| `tags` | `{"tags": ["invoice"]}` | Matches files whose frontmatter `tags` list contains any of the listed tags |
| `file` | `{"file": "INV-*.md"}` | Matches files whose name matches the glob |
| Both | `{"tags": ["invoice"], "file": "INV-*.md"}` | OR semantics — either match fires |
| Absent | _(no `match` key)_ | Applies to all files (default/fallback schema) |

- Tag matching is case-insensitive and checks `tags:` / `Tags:` / `TAGS:` frontmatter keys
- A file with no matching schema is silently skipped during `sigil check`
- In multi-schema mode, `unique` is scoped per schema (not folder-wide)
- Conditional rules and cross-field constraints are per-schema

### Creating and updating in multi-schema folders

```bash
sigil create <folder> --schema invoices    # use the "invoices" schema
sigil update <folder> <file> --schema notes
```

If all schemas have matchers and `--schema` is omitted, Sigil exits with an error listing available schema names. If one schema has no `match` (matcherless), it acts as the default and `--schema` can be omitted.

### Reading multi-schema folders

`sigil read` without `--schema` runs in **raw mode** — it reads every file in the folder, shows all their actual frontmatter fields as dynamic columns, and lets `--where` filter on any field name regardless of which schema it belongs to:

```bash
sigil read <folder>                          # all files, all fields, dynamic columns
sigil read <folder> --where Title=Foo        # filter across all schemas by any field
sigil read <folder> --where Status=Done --json
```

Each JSON record includes `_schema: "invoices"` (or whichever schema matched) so consumers can tell file types apart.

Use `--schema` to scope to one schema's typed view:

```bash
sigil read <folder> --schema invoices        # only invoice files, invoice columns
sigil read <folder> --schema invoices --where Code=INV-0001
```

### Viewing the schema definition

```bash
sigil schema <folder>        # shows each schema with its match header
sigil schema <folder> --json # {"schemas": [...]} with full field definitions
```

### Bulk import in multi-schema folders

`sigil import` always uses the primary (matcherless) schema. If your folder has only named schemas (all with matchers), import is not supported — create files individually with `sigil create --schema <name>` instead.

---

## `check --fix` auto-repairs

`sigil check --fix` applies non-destructive fixes without human input:

| Violation | Fix applied |
|-----------|-------------|
| Body field found in YAML frontmatter | Moves to `## FieldName` body section |
| Enum value with one fuzzy match | Corrects automatically |
| Filename doesn't match template | Renames the file |
| `sigil_schema` breadcrumb missing or stale | Adds / updates it |
| Required field absent — has a `default` | Inserts the default value |
| Multiple fuzzy enum matches | Lists candidates — human or agent decides |

---

## Breadcrumbs

Every file created or updated by Sigil gets `sigil_schema` in its YAML frontmatter, pointing to the schema file that governs it:

```yaml
---
sigil_schema: communications/drafts/sigil.schema.md
Title: Why I Switched to Obsidian
Status: Drafting
Date: 2026-06-21
---
```

This lets agents identify Sigil-managed files cold and locate the schema without a config lookup. If the breadcrumb is missing or stale, `sigil check --fix` corrects it.

---

## Agent integration

Sigil is a general-purpose CLI — it works from any agent that can run shell commands. No platform lock-in.

### skill.md

Sigil ships a compact command reference (`skill.md`) that any agent can load as context:

```bash
sigil install-skill                             # install to default skills directory
sigil install-skill ~/.openclaw/skills/         # custom path
sigil install-skill                             # omit path to preview location first
```

Or install manually:

```bash
curl -o ~/.claude/skills/sigil.md \
  https://gitlab.com/DigitalGert/sigil/-/raw/main/skill.md
```

### `_sigil` hints in JSON output

The first `--json` call of the day for each command type includes a `_sigil` key with ready-to-run commands for every available operation on that folder:

```json
{
  "folder": "/path/to/drafts",
  "total_violations": 0,
  "_sigil": {
    "hints": {
      "create": "sigil create \"/path/to/drafts\" --data '{\"Title\": \"...\", \"Status\": \"Idea\"}'",
      "read":   "sigil read \"/path/to/drafts\" --json",
      "move":   "sigil move \"/path/to/drafts\" <file>  # → communications/published",
      "import": "sigil import \"/path/to/drafts\" records.json  # or records.csv",
      "logs":   "sigil logs  # available when debug logging is enabled in ~/.sigil.config"
    },
    "workflow_next": "communications/published",
    "workflow_description": "When a draft reaches Status: Published..."
  }
}
```

Hints are suppressed on repeat calls the same day (tracked in `~/.sigil.config` under `hints_shown`). When a folder has a `workflow` declared, `move`, `workflow_next`, and `workflow_description` are always included. Check `"_sigil" in response` before accessing.

### Isolated config for agent environments

Point `SIGIL_CONFIG` to a path inside your skills directory to keep Sigil's state portable and separate from your personal config:

```bash
export SIGIL_CONFIG=~/.claude/skills/sigil.config
```

### cron.md — scheduled integrity checks

Sigil ships a cron skill file for scheduled vault integrity checks:

```bash
sigil install-cron ~/.claude/crons/
```

---

## Config

Global config lives at `~/.sigil.config` (YAML). Created on first `sigil init`.

```yaml
vaults:
  - path: /home/user/Documents/Dara
    name: Dara
managed_folders:
  - path: /home/user/Documents/Dara/communications/drafts
    schema: custom
    initialized: '2026-06-21'
hints_shown:
  check: '2026-06-21'
  read:  '2026-06-21'
```

Override the config path:

```bash
export SIGIL_CONFIG=/path/to/config     # environment variable
sigil --config /path/to/config ...      # per-command flag
```

### Debug logging

```yaml
debug: true
log_path: ~/sigil-logs/
```

When enabled, Sigil writes dated log files (`sigil-YYYY-MM-DD.log`) to `log_path`. `sigil logs` scans them and reports command usage, error counts, and the JSON vs human output-mode split — useful for understanding how agents are using the tool.

---

## Exit codes

| Code | Meaning |
|------|---------|
| 0 | Clean — no violations |
| 1 | Violations found |
| 2 | Tool error |

---

## Built-in schema templates

`sigil init <folder>` offers four templates:

| Template | Fields |
|----------|--------|
| `task` | Title, Status (Open/In Progress/Done/Cancelled), Priority, Created, Tags, Description |
| `contact` | Name, Email, Phone, Relationship, Category, Tags, Notes |
| `note` | Title, Date, Tags, Updated, Content |
| `project` | Title, Status, Start, End, Tags, Description |

Pass `--template task` to skip the interactive prompt.

---

## Contributing

This is an active project. Bug reports, schema template ideas, and pull requests are welcome.

- **Issues:** [gitlab.com/DigitalGert/sigil/-/issues](https://gitlab.com/DigitalGert/sigil/-/issues)
- **Source:** [gitlab.com/DigitalGert/sigil](https://gitlab.com/DigitalGert/sigil)
- **PyPI:** [pypi.org/project/sigil-obsidian](https://pypi.org/project/sigil-obsidian/)

### Development setup

```bash
git clone https://gitlab.com/DigitalGert/sigil.git
cd sigil
pip install -e ".[dev]"
python -m pytest tests/
python -m ruff check src/
```

The test suite has 415 tests covering all flows, all 13 field types, schema parsing, CRUD edge cases, workflow moves, migrations, imports, health checks, the hint system, multi-schema matching, raw-mode read, constraint enforcement at write time, and CLI-layer error paths.

### Versioning

Sigil follows [SemVer](https://semver.org/). Breaking changes (removed commands, changed JSON keys) increment the minor version. The `sigil_version` field in each schema file records the Sigil version that last wrote it — `sigil check` warns when the installed version lags behind.

---

## License

MIT — see [LICENSE](LICENSE).
