Metadata-Version: 2.4
Name: zotcli
Version: 0.2.2
Summary: A CLI for browsing, exporting, and adding items to a Zotero library
Author-email: MohamedNumair <mo7amednumair@gmail.com>
License: MIT
Requires-Python: >=3.10
Requires-Dist: click>=8.1
Requires-Dist: platformdirs>=4.0
Requires-Dist: pydantic>=2.0
Requires-Dist: python-dateutil>=2.9
Requires-Dist: rich>=13.0
Provides-Extra: all
Requires-Dist: httpx>=0.27; extra == 'all'
Requires-Dist: jinja2>=3.1; extra == 'all'
Requires-Dist: playwright>=1.40; extra == 'all'
Requires-Dist: pybtex>=0.24; extra == 'all'
Provides-Extra: bibtex
Requires-Dist: pybtex>=0.24; extra == 'bibtex'
Provides-Extra: browser
Requires-Dist: playwright>=1.40; extra == 'browser'
Provides-Extra: dev
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-httpserver; extra == 'dev'
Requires-Dist: tomli-w; extra == 'dev'
Requires-Dist: vcrpy; extra == 'dev'
Provides-Extra: export
Requires-Dist: jinja2>=3.1; extra == 'export'
Provides-Extra: write
Requires-Dist: httpx>=0.27; extra == 'write'
Description-Content-Type: text/markdown

# zotcli (`zot`)
![alt text](zot.png)

A "crazy" good command-line interface for your local [Zotero](https://www.zotero.org/) library. Queries `zotero.sqlite` directly — no Zotero app running and no API key required for reads. **Default: strictly read-only.** Write capabilities are opt-in and route through Zotero's own connector HTTP server — `zotero.sqlite` is **never** modified directly.

**Library stats on this machine:** 3,771 items · 200 collections · 3,201 tags · 6.2 GB storage

---

## Installation

```bash
# Read-only (default)
pip install zotcli

# Adds the write API client (httpx) — required for any zot add … command
pip install "zotcli[write]"

# Adds Playwright for paywalled-PDF retrieval via browser SSO
pip install "zotcli[browser]"

# Everything
pip install "zotcli[all]"
```

Verify:
```bash
zot --help
```

The database at `~/Zotero/zotero.sqlite` is auto-detected on WSL. Override anytime with `--db PATH`.

Depending on the operating system the Zotero folder can be in different locations. By default, zotcli looks for the database at `~/Zotero/zotero.sqlite`, which works for most setups. For Windows users, the Zotero folder is typically located in `C:\Users\<YourUsername>\Zotero`. For macOS users, it is usually found in `~/Zotero`. If you have a custom setup or want to specify a different path, you can use the `--db` option to point zotcli to the correct location of your `zotero.sqlite` file.

---

## Repository layout

```
zotcli/
├── PLAN.md                          # Original design document
├── PLAN_WRITE.md                    # Write-capability design document
├── SKILL.md                         # Agent skill descriptor
├── pyproject.toml                   # Package metadata and dependencies
├── docs/
│   ├── commands.md                  # Full command reference
│   └── architecture-write.md       # Write-path architecture overview
├── src/
│   └── zotcli/
│       ├── __init__.py
│       ├── __main__.py              # python -m zotcli entrypoint
│       ├── db.py                    # Read-only SQLite connection + auto-discovery
│       ├── config.py                # TOML config + [write]/[unpaywall]/[browser] sections
│       ├── paths.py                 # Cross-platform self-contained path resolution
│       ├── models.py                # Pydantic v2 models: Item, Collection, Creator, Attachment, Note
│       ├── queries/
│       │   ├── items.py             # Core item fetch
│       │   ├── collections.py       # Collection tree queries
│       │   ├── attachments.py       # Attachment path resolution helpers
│       │   ├── tags.py              # Tag queries
│       │   └── search.py            # Field search, author search, DOI, year, fulltext
│       ├── write/                   # Write-path package (requires zotcli[write])
│       │   ├── connector_client.py  # httpx client for /connector/*
│       │   ├── preflight.py         # Zotero liveness check
│       │   ├── session.py           # Session lifecycle + updateSession
│       │   ├── csl_json.py          # CSL-JSON ↔ connector item shape
│       │   ├── identifiers.py       # detect_kind: DOI / arXiv / PMID / ISBN / URL / citation
│       │   ├── dedup.py             # Read-only duplicate check against DB
│       │   ├── citation_pipeline.py # Free-text citation → DOI pipeline
│       │   ├── pdf.py               # MIME sniff + streaming upload
│       │   ├── browser.py           # Playwright headed window (lazy import; requires zotcli[browser])
│       │   ├── credentials.py       # File-based credential store (mode 0600)
│       │   └── resolvers/
│       │       ├── crossref.py      # DOI → CSL-JSON; bibliographic search
│       │       ├── arxiv.py         # arXiv ID → CSL-JSON
│       │       ├── pubmed.py        # PMID → CSL-JSON
│       │       ├── openlibrary.py   # ISBN → CSL-JSON
│       │       ├── openalex.py      # Citation/title fallback
│       │       ├── semantic_scholar.py  # Second fallback (rate-limited)
│       │       ├── unpaywall.py     # DOI → OA PDF URL (opt-in)
│       │       ├── ieee.py          # URL → DOI extraction helpers
│       │       └── sciencedirect.py # URL/PII → DOI extraction helpers
│       ├── export/
│       │   ├── json_.py             # Full-fidelity JSON dump
│       │   ├── csv_.py              # Flat CSV (one row per item)
│       │   ├── bibtex.py            # BibTeX with auto citation keys
│       │   └── markdown.py          # Markdown table report
│       └── cli/
│           ├── main.py              # Root Click group + global options
│           ├── render.py            # Shared Rich helpers (tables, panels, trees)
│           ├── add.py               # `zot add` group + auto-detect dispatcher
│           ├── config_cmd.py        # `zot config` group
│           ├── collections.py       # `zot collections` subcommands
│           ├── items.py             # `zot items` subcommands
│           ├── attachments.py       # `zot attachments` subcommands
│           ├── search.py            # `zot search`
│           ├── stats.py             # `zot stats` subcommands
│           └── export.py            # `zot export` subcommands
└── tests/
    ├── conftest.py                  # In-memory SQLite fixture with seeded test data
    ├── unit/                        # Unit tests (no network, no Zotero)
    ├── integration/                 # Integration tests (mocked connector + resolvers)
    └── e2e/                         # End-to-end (opt-in; requires real Zotero)
```

---

## Architecture

```
CLI layer  (cli/)
    ↓  Click commands call query functions
Query layer  (queries/)
    ↓  Batch SQL via sqlite3.Row
Database layer  (db.py)       ← read-only URI: file:zotero.sqlite?mode=ro
    ↓
zotero.sqlite

Write path (opt-in):
CLI layer  (cli/add.py)
    ↓  resolve identifiers via external APIs
Resolver pipeline  (write/resolvers/)
    ↓  CSL-JSON
Connector client  (write/connector_client.py)
    ↓  loopback HTTP
Zotero desktop app  →  zotero.sqlite + storage/
```

See [`docs/architecture-write.md`](docs/architecture-write.md) for the full write-path design.

---

## Global options

These go **before** the subcommand:

```
zot [--db PATH] [--library ID] [--format table|json|csv] [--no-color] <command>

Write-related globals (only relevant when write.enabled=true):
zot [--allow-write] [--connector-url URL] [--require-zotero/--no-require-zotero] <command>
```

---

## Examples with output

### `zot stats` — library overview

```bash
zot stats
```
```
   Library Summary
┌─────────────┬──────┐
│ Items       │ 3771 │
│ Collections │  200 │
│ Tags        │ 3201 │
│ Creators    │ 3959 │
└─────────────┴──────┘
       Items by Type
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃ Type             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│ annotation       │  2188 │
│ journalArticle   │   864 │
│ conferencePaper  │   234 │
│ webpage          │   179 │
│ book             │    77 │
│ report           │    75 │
│ preprint         │    43 │
│ thesis           │    32 │
│ bookSection      │    25 │
└──────────────────┴───────┘
```

```bash
zot stats years
```
```
              Publications by Year
┏━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Year ┃ Count ┃ Bar                            ┃
┡━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 2026 │    10 │                                │
│ 2025 │    36 │ ██                             │
│ 2024 │   110 │ █████████                      │
│ 2023 │   361 │ ██████████████████████████████ │
│ 2022 │   144 │ ███████████                    │
│ 2021 │   120 │ █████████                      │
│ 2020 │   101 │ ████████                       │
│ 2019 │    96 │ ███████                        │
└──────┴───────┴────────────────────────────────┘
```

```bash
zot stats tags --top 10
```
```
           Top 10 Tags
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃ Tag                   ┃ Items ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│ Topology              │    89 │
│ Voltage measurement   │    77 │
│ Network topology      │    75 │
│ State estimation      │    65 │
│ Distribution networks │    53 │
│ thesis                │    50 │
│ Smart meters          │    48 │
│ notion                │    47 │
│ Real-time systems     │    46 │
│ _EndnoteXML import    │    38 │
└───────────────────────┴───────┘
```

---

### `zot collections` — browse the library tree

```bash
zot collections list
```
```
Collections
├── 00_Reading Tracker (2)
│   ├── Read (12)
│   ├── Reading (15)
│   └── To Read (31)
├── Energy Management (6)
│   ├── AI based Energy Management (2)
│   ├── Demand Response (14)
│   │   └── Home Energy Management System (3)
│   ├── Energy Market (75)
│   ├── Energy Storage (4)
│   ├── Felixibility (8)
│   └── Power Flow (16)
├── PhD Research (...)
│   ├── Distribution Systems (...)
│   └── State Estimation (...)
└── ...
```

```bash
zot collections items "Energy Market"
```
```
                            Energy Market (73 items)
┏━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━┓
┃  # ┃ Key       ┃ Type               ┃ Title                               ┃ Authors      ┃ Year   ┃
┡━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━┩
│  1 │ RBUU6AM2  │ journalArticle     │ New coordination framework for      │ Hussain      │ 2023   │
│    │           │                    │ smart home peer-to-peer trading…    │ et al.       │        │
│  2 │ 8DE2V7ZZ  │ journalArticle     │ Integrating Distributed Flexibility │ Tsaousoglou  │ 2023   │
│    │           │                    │ into TSO-DSO Coordinated Markets…   │ et al.       │        │
│  3 │ GKG9XUBE  │ thesis             │ Adoption of Blockchain in European  │ Meyer        │ 2023   │
│    │           │                    │ Electricity Markets                 │              │        │
└────┴───────────┴────────────────────┴─────────────────────────────────────┴──────────────┴────────┘
```

Include all sub-collections recursively:
```bash
zot collections items "Energy Management" --recursive
```

---

### `zot items` — inspect individual items

```bash
zot items list --limit 5
```
```
┏━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┓
┃  # ┃ Key       ┃ Type               ┃ Title                                       ┃ Authors     ┃ Year   ┃
┡━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━┩
│  1 │ L54VEEWV  │ journalArticle     │ Three-phase feeder parameter estimation…    │ Yang et al. │ 2026   │
│  2 │ B8CNX4LI  │ journalArticle     │ Coordinated State Estimation of Power…      │ Sharma      │ 2025   │
│  3 │ MXYF8V3J  │ journalArticle     │ Towards Digital Twin of Distribution…       │ Idlbi       │ 2026   │
│  4 │ YN89DKH4  │ book               │ 41st European Photovoltaic Solar Energy…    │             │ 2024   │
│  5 │ 5UFZMSLU  │ journalArticle     │ UNLOCKING DATA CENTRE HOSTING CAPACITY…     │ Numair      │ 2026   │
└────┴───────────┴────────────────────┴─────────────────────────────────────────────┴─────────────┴────────┘
```

```bash
zot items show 5UFZMSLU
```
```
╭────────────────── journalArticle #6439  5UFZMSLU ──────────────────╮
│ Title    UNLOCKING DATA CENTRE HOSTING CAPACITY AND FLEXIBILITY     │
│          THROUGH DYNAMIC CABLE RATING                               │
│ Authors  Numair, Mohamed; ElKholy, Ahmed M; Martins-Britto,         │
│          Amauri G; Hertem, Dirk Van; Vanin, Marta                   │
│ Year     2026                                                       │
│ abstractNote  The unprecedented pace of Distributed Energy          │
│               Resource (DER) integration and electr…               │
│ language      en                                                    │
│                                                                     │
│ Attachments                                                         │
│   ✓ Numair et al. - 2026 - UNLOCKING DATA CENTRE…pdf               │
│                                                                     │
│ Notes (1)                                                           │
│   • Annotations(2/25/2026) (Numair et al., 2026, p. 1) …           │
╰─────────────────────────────────────────────────────────────────────╯
```

---

### `zot search` — find items

**By title keyword:**
```bash
zot search "bayesian" --field title
```
```
                                5 result(s)
┏━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃  # ┃ Key       ┃ Type            ┃ Title                                     ┃ Authors        ┃ Year   ┃
┡━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│  1 │ LVTG4KLQ  │ journalArticle  │ An Improved Recursive Bayesian Approach   │ Chen et al.    │ 2013   │
│    │           │                 │ for Transformer Tap Position Estimation   │                │        │
│  2 │ 6BR3CYQA  │ conferencePaper │ Bayesian distribution system state        │ Angioni et al. │ 2016   │
│    │           │                 │ estimation in presence of non-Gaussian…   │                │        │
└────┴───────────┴─────────────────┴───────────────────────────────────────────┴────────────────┴────────┘
```

**By author name:**
```bash
zot search --author "Numair"
```

**By DOI:**
```bash
zot search --doi "10.1016/j.epsr.2020.106394"
```

**By year range:**
```bash
zot search --year 2023 --type conferencePaper
```

---

### `zot attachments` — locate files

**Get the PDF path for a single item:**
```bash
zot attachments path 5UFZMSLU
```
```
~/Zotero/storage/RIB344FW/Numair et al. - 2026 - UNLOCKING DATA CENTRE HOSTING CAPACITY AND FLEXIBILITY THROUGH DYNAMIC CABLE RATING.pdf
```

**Find all missing attachments:**
```bash
zot attachments list --missing
```

**Open a PDF in the system viewer:**
```bash
zot attachments open 5UFZMSLU
```

---

### `zot items fulltext` — retrieve full text

```bash
zot items fulltext 5UFZMSLU
zot items fulltext 5UFZMSLU --offline
```

Retrieval order:
1. direct network access from DOI/URL (institution/network-location access),
2. configured credentials (`zot config library-auth`),
3. Playwright interactive login fallback,
4. local Zotero fulltext index,
5. metadata fallback (title/abstract/notes).

---

### `zot export` — export to files

**BibTeX:**
```bash
zot export bib --collection "Energy Market" --output refs.bib
zot export bib --item 5UFZMSLU --output ref.bib
```

**CSV / JSON / Markdown:**
```bash
zot export csv      --collection "Energy Market" --output refs.csv
zot export json     --all --output library.json
zot export markdown --all --notes --output report.md
```

---

### Workflow: search → get attachment paths

```python
from zotcli.db import ZoteroDatabase
from zotcli.queries.search import search_items, search_by_author

DB = "~/Zotero/zotero.sqlite"

with ZoteroDatabase(DB) as db:
    bayesian = search_items(db, "bayesian", fields=["title"])
    numair   = search_by_author(db, "Numair")

    seen = set()
    for item in bayesian + numair:
        if item.item_id in seen:
            continue
        seen.add(item.item_id)
        for att in item.attachments:
            if att.file_exists and "pdf" in att.content_type.lower():
                print(f"{item.key}\t{att.absolute_path}")
```

---

## Writing to your library

> **Default: strictly read-only.** Write capabilities are opt-in and route through Zotero's own connector HTTP server — `zotero.sqlite` is **never** modified directly.

**Zotero must be running** for any `zot add …` command to succeed.

### Enable write capability

```bash
# One-time setup (persists to config)
zot config set write.enabled true

# Check current status
zot config get write.enabled

# Verify Zotero is reachable
zot add status
```

### Quick-start: add items

```bash
# Add by DOI / arXiv / PMID / ISBN
zot add doi 10.1109/TPWRS.2023.1234567
zot add arxiv 2401.12345 --collection Preprints
zot add pmid 31452104
zot add isbn 978-0-262-03384-8 --collection Books

# IEEE Xplore or ScienceDirect URL (DOI is extracted automatically — no browser needed)
zot add "https://ieeexplore.ieee.org/document/9876543"
zot add "https://www.sciencedirect.com/science/article/pii/S2352467725000XYZ"

# Free-text citation string
zot add "Zhang, J., Geth, F., Heidari, R., Verbič, G. (2025) Beyond simplifications…"

# Local PDF (Zotero auto-recognises the parent reference)
zot add ~/Downloads/paper.pdf

# Smart auto-detect: zot add figures out the type automatically
zot add "10.1109/TPWRS.2023.1234567"   # detected as DOI
zot add "2401.12345"                    # detected as arXiv
zot add "/home/me/paper.pdf"            # detected as file
```

### Batch add

```bash
# papers.txt: one DOI / arXiv / URL / citation per line; # = comment
zot add batch papers.txt --collection "Smart Grid" --tag imported
```

### Import from a bibliography file

```bash
# .bib, .ris, or .json (CSL-JSON)
zot add import refs.bib --collection "Imports/2026-05"
```

### Optional: paywalled-PDF retrieval

```bash
# Step 1: register for Unpaywall (email only; opt-in)
zot add login --service unpaywall

# Step 2: add with open-access PDF
zot add doi 10.1109/X --with-pdf

# For paywalled content: authenticate via browser SSO (requires zotcli[browser])
zot add login --service ieee           # opens headed Chromium; sign in once
zot add login --service sciencedirect
zot add doi 10.1109/X --with-pdf      # re-uses saved cookies
```

### Architecture summary

All writes go through `POST /connector/saveItems` (and related endpoints) on Zotero's local HTTP server at `127.0.0.1:23119`. Zotero performs every database transaction. `zotero.sqlite` is never opened in write mode by zotcli.

See [`docs/architecture-write.md`](docs/architecture-write.md) for full details.

---

## Running tests

```bash
python3 -m pytest tests/ -q
```
```
441 passed in Xs
```

Tests use an in-memory SQLite fixture seeded with synthetic Zotero data. No real database needed. e2e tests (requiring a live Zotero) are opt-in:

```bash
python3 -m pytest tests/e2e -m e2e
```

---

## Configuration

Self-contained config at `<zotcli-home>/config.toml`. Run `zot config path` to find the directory.

```toml
[database]
path = ""                        # empty = auto-detect zotero.sqlite

[output]
default_format = "table"
color = true
page_size = 50

[write]
enabled = false                  # opt-in; set true once with: zot config set write.enabled true
connector_url = "http://127.0.0.1:23119"
require_zotero = true

[unpaywall]
enabled = false                  # opt-in; set up with: zot add login --service unpaywall
email = ""

[browser]
headless = false                 # SSO/captcha needs headed browser

[library_auth.1]
institution = "KU Leuven"
username = "alice"
password = "token"
```

Override `<zotcli-home>` with the `ZOTCLI_HOME` environment variable.

---

## Safety

- Database opened with `sqlite3://…?mode=ro` — the OS-level read-only URI flag makes direct writes impossible
- WAL journal detection warns if Zotero is currently open (pending writes may not be visible yet)
- Network retrieval is optional and only used by `zot items fulltext` unless `--offline` is passed
- Write operations route through Zotero's own connector HTTP server — never via direct SQLite mutation
- Default is read-only; writes require explicit opt-in (`zot config set write.enabled true`)
- No Zotero API key required
