Metadata-Version: 2.4
Name: poco-harmonizer
Version: 0.0.2
Summary: PoCo — trust-first, deterministic reference library harmonizer (no LLM).
License: Apache-2.0
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: fastapi>=0.110
Requires-Dist: uvicorn[standard]>=0.29
Requires-Dist: python-multipart>=0.0.9
Requires-Dist: pydantic>=2.6
Requires-Dist: habanero>=1.2.6
Requires-Dist: lxml>=5.0
Requires-Dist: httpx>=0.27
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: hypothesis>=6.100; extra == "dev"

# PoCo - Polish & Complete

PoCo is a local-first tool for cleaning and completing Zotero and EndNote
reference libraries.

It helps you turn an inconsistent reference export into a cleaner copy: consistent
journal names, safer DOI formatting, completed author names, page ranges,
publication details, book metadata, and a transparent audit trail of what changed.

The important part: **you stay in control**. PoCo shows every proposed change
before export, lets you reject or edit suggestions, and never modifies your
original library file.

PoCo does not use an LLM or generative AI. The engine is deterministic: each
suggestion comes from a scoped rule, your chosen preferences, a local lookup
table, or source-backed metadata from public services such as CrossRef, OpenAlex,
and Open Library.

## Quick Start

PoCo runs on your own computer and opens in your browser. You install it once
from a terminal, then start it any time by typing `poco`. The steps below set up
everything from scratch — no prior tools needed.

> **Before PoCo's first PyPI release**, replace `pipx install poco-harmonizer`
> in the steps below with:
> `pipx install git+https://github.com/Rkn12345/poco-reference-harmonizer.git`

### Windows

1. **Install Python.** Download it from
   [python.org/downloads](https://www.python.org/downloads/) and run the
   installer. On the first screen, **tick "Add python.exe to PATH"**, then click
   *Install Now*. (This checkbox is easy to miss and everything else depends on
   it.)
2. **Open a terminal.** Click Start, type `PowerShell`, and press Enter.
3. **Install pipx** (a small helper that installs apps like PoCo cleanly). Paste
   these lines one at a time:
   ```powershell
   py -m pip install --user pipx
   py -m pipx ensurepath
   ```
   Then **close PowerShell and open it again** so it picks up the change.
4. **Install PoCo:**
   ```powershell
   pipx install poco-harmonizer
   ```
5. **Start PoCo:**
   ```powershell
   poco
   ```
   Your browser opens with PoCo running. To stop it, go back to PowerShell and
   press `Ctrl+C`.

### macOS

1. **Open the Terminal.** Press `Cmd+Space`, type `Terminal`, and press Enter.
2. **Install Homebrew** if you don't already have it — it sets up Python and pipx
   for you. Paste this line and follow the prompts (skip if you already have
   Homebrew):
   ```bash
   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
   ```
3. **Install pipx:**
   ```bash
   brew install pipx
   pipx ensurepath
   ```
   Then **close the Terminal and open it again**.
4. **Install PoCo:**
   ```bash
   pipx install poco-harmonizer
   ```
5. **Start PoCo:**
   ```bash
   poco
   ```
   Your browser opens with PoCo running. To stop it, go back to the Terminal and
   press `Ctrl+C`.

After the first setup, you only ever need the last step — just type `poco` to
start it again.

## Why You Can Trust It

PoCo is built so you don't have to take any of the claims above on faith — each
one is checkable:

- **It's open source (Apache-2.0).** The whole engine and rule set are in this
  repository. Nothing about how a change is decided is hidden.
- **No LLM, no guessing.** Every change traces to a named rule, your preference,
  a local table, or a cited public source — see [`refharmonizer/core/`](refharmonizer/core/).
- **Your originals are never touched.** PoCo reads your file and writes a *new*
  copy; the input bytes are hashed into the run manifest so you can prove it.
- **Nothing is exported without your confirmation.** Review is a hard gate, and
  editing any decision resets it (see [Confirm](#5-confirm)).
- **It runs on your machine.** No accounts, analytics, or telemetry. Offline mode
  sends zero network requests; online enrichment only sends the lookup fields
  documented in [Privacy And Control](#privacy-and-control).
- **Every run is reproducible.** A manifest records input/output hashes, engine
  and table versions, the sources queried, and the disposition of every change.
- **It's honest about its limits.** See [Known Limitations](#known-limitations)
  rather than discovering them yourself.

## What You Can Do With It

- Import a Zotero CSL-JSON export, an EndNote XML export, the bundled sample
  library, or a read-only Zotero local API session.
- Choose how you want references to look before analysis: author names,
  journal-name style, DOI style, page ranges, title casing, publisher style,
  volume/issue completion, journal capitalization, and protected terms.
- Analyze the library for existing conventions and possible improvements.
- Review proposed changes grouped by category: titles, journal names, authors,
  identifiers, item types, publication details, book metadata, text cleanup, and
  your manual edits.
- Accept, reject, or edit suggestions before export.
- Add your own manual corrections for key fields such as title, journal title,
  DOI, pages, volume, issue, publisher, ISSN, ISBN, and year.
- Export a polished copy plus audit artifacts that explain the run.

## How The Workflow Works

### 1. Import

Start with a reference export from Zotero or EndNote, or use the bundled sample
library.

PoCo keeps the original input intact. Internally, it builds an analysis view and
stores the original records separately so accepted changes can be applied back
onto a copy at export time.

Supported inputs today:

- Zotero CSL-JSON export
- EndNote XML export
- Zotero local API read from `localhost:23119`
- Bundled sample library for trying the app immediately

### 2. Tune

Before running the engine, you choose the conventions you want PoCo to follow.
For example:

- full author names or initials
- full or abbreviated journal names
- bare DOI or DOI URL
- expanded or abbreviated page ranges
- full or abbreviated publisher names
- sentence case or headline case titles
- whether to complete volume/issue fields

These choices are not hidden defaults. They become part of the run configuration
and are recorded in the manifest.

### 3. Detect And Enrich

PoCo scans the library to detect existing conventions and identify records that
may be improved.

It can:

- normalize DOI format and safe text hygiene
- detect title-case, journal-name, and page-range conventions
- complete missing or abbreviated author names when source evidence is available
- complete pages, volume, issue, dates, ISSN/ISBN, and publisher fields
- discover missing DOIs through CrossRef bibliographic search when enabled
- flag exact-DOI duplicates
- flag retraction or correction notices when source metadata exposes them
- use local journal tables when offline mode is enabled

Every suggested change is represented as a patch with a source, confidence tier,
category, evidence, and before/after value.

### 4. Review

The review step is the main safety gate.

PoCo groups changes into readable sections and shows each proposed change in
context. You can:

- accept a suggestion
- reject a suggestion
- edit the suggested value
- inspect source evidence
- see conflicts when metadata sources disagree
- see which records were left unchanged and why

Rejecting a discovered DOI also rejects the changes that depended on that DOI.
Manual edits are logged like every other change and apply last, so your value
wins.

### 5. Confirm

Export is blocked until you confirm the review.

If you change an accept/reject decision or edit a value, the confirmation gate is
reset. This makes it difficult to accidentally export a library after changing
the review state.

### 6. Export

PoCo exports a new file. It does not edit your original library.

Export includes:

- a polished library file (`poco_library.json` for Zotero, `poco_library.ris`
  for EndNote)
- an audit log as CSV
- an audit log as JSON
- an audit log as HTML
- a run manifest as JSON

For Zotero, import the polished CSL-JSON into a new collection.

For EndNote, the polished library is exported as **RIS** — EndNote's own XML
re-import is unreliable, whereas RIS imports dependably through the built-in
*Reference Manager (RIS)* filter (`File > Import`). Import it into a new, empty
library or group to avoid duplicates. RIS does not carry EndNote's `rec-number`,
so it does not round-trip in place; the audit log records the `rec-number` ->
change mapping. The original EndNote XML is still available as an advanced
download for anyone who prefers it.

## Privacy And Control

PoCo is designed as a local-first tool.

- Your library is processed on your machine.
- The local app keeps session data in memory while it is running.
- PoCo does not provide accounts, analytics, telemetry, or a hosted database in
  the local version.
- Your original library file is never modified.
- Export only happens after you confirm the review.
- Offline mode disables public metadata lookups and uses bundled/local data.

When online enrichment is enabled, PoCo may query public metadata services such
as CrossRef, OpenAlex, and Open Library. Depending on the record, those requests
may use identifiers such as DOI, ISSN, or ISBN, or bibliographic search fields
such as title, author, and year for DOI discovery.

PoCo may also keep local convenience files on your own machine:

- API response cache: `~/.cache/refharmonizer/api_cache.sqlite`
- saved Tune preferences and protected terms:
  `~/.cache/refharmonizer/preferences.json`

These local files are used to make repeat runs faster, support auditability, and
remember your preferences. They are not uploaded to PoCo.

## Transparency And Auditability

Every run is designed to be inspectable.

The audit logs record the proposed changes with:

- record key
- field path
- old value
- new value
- category
- rule family
- confidence tier
- source
- dependency information
- label explaining the change

The manifest records:

- engine version
- ruleset version
- lookup table version
- input and output file hashes
- whether offline mode was used
- effective configuration and preferences
- DOI discovery log
- API sources queried
- accepted, rejected, skipped, and unchanged items

The goal is not just to produce a cleaner library, but to make the process
reviewable later.

## Install And Run

See [Quick Start](#quick-start) for the from-scratch, step-by-step setup. PoCo
runs entirely on your own machine — there is no hosted backend and nothing is
uploaded.

If you already have [pipx](https://pipx.pypa.io):

```bash
pipx install poco-harmonizer
poco
```

Or run it without a permanent install using [uv](https://docs.astral.sh/uv/):

```bash
uvx --from poco-harmonizer poco
```

Running `poco` opens `http://127.0.0.1:<port>` in your browser. Choose Import →
use the bundled sample library, or upload a Zotero CSL-JSON / EndNote XML export.
Your library is processed locally and your original file is never modified.

## Command Line

The CLI uses the same deterministic engine and preview-before-export posture.

```bash
# Dry run on the bundled sample. Writes nothing.
python -m refharmonizer.cli

# Dry run on your library. Writes nothing.
python -m refharmonizer.cli my-library.json

# Export a polished copy after previewing the summary.
python -m refharmonizer.cli my-library.json --apply

# EndNote XML, offline mode.
python -m refharmonizer.cli my-library.xml --offline --apply
```

Without `--apply`, the CLI prints detected conventions, proposed change counts,
examples, and skipped items, then exits without writing output.

## Tests

```bash
pip install -e ".[dev]"
pytest
```

Current verified state:

- Python test suite: 113 tests passing
- Frontend production build: passing

## Project Map

```text
refharmonizer/
  api/              FastAPI app, local sessions, review gate, downloads
  core/
    adapters/       CSL-JSON, EndNote XML, Zotero local API ingest/export
    api_clients/    CrossRef, OpenAlex, Open Library, local response cache
    detect.py       convention detection
    discover.py     DOI discovery with review gating
    enrich.py       source-backed completion and flags
    normalize.py    deterministic local cleanup rules
    invariants.py   rule contracts; unsafe patches are dropped
    patch.py        patch/evidence/confidence model
    preview.py      grouped preview and skipped-item report
    render.py       neutral reference previews
    ris.py          RIS export for the EndNote path
    audit.py        CSV/JSON/HTML audit logs
    manifest.py     reproducible run manifest
    pipeline.py     engine orchestration
  data/             bundled sample library and journal lookup table
  tests/            engine, API, adapter, preview, and invariant tests

web/                Vite + React + TypeScript frontend
```

## Development

Contributors run the two processes separately for hot-reload. Backend (Python 3.9+):

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
uvicorn refharmonizer.api.app:app --port 8000 --reload
```

Frontend (Node 18+), in another terminal — proxies `/api` to the backend:

```bash
cd web
npm install
npm run dev      # http://localhost:5173
```

To produce the single-process app that `poco` serves, build the UI into the
package, then install:

```bash
cd web && npm run build      # outputs to refharmonizer/webui/
cd .. && pip install .       # bundles the built UI into the wheel
poco
```

## Known Limitations

PoCo is honest about what it does not yet do:

- **No in-place update.** PoCo exports a clean copy for import into a *new*
  collection or library; it does not modify your Zotero or EndNote library in
  place. (A future "Connect Zotero" mode could update items via the Web API.)
- **EndNote round-trip is import-only.** The RIS export imports cleanly but cannot
  carry `rec-number`, so changes are not merged back onto your existing records.
- **Editors are not yet captured for book chapters.** The EndNote ingest reads
  authors but not secondary-author/editor fields, so chapter editors are not
  completed or carried through.
- **Input formats.** Direct `.enl` / `.enlx`, BibTeX, and Zotero database-file
  parsing are not supported. Inputs are Zotero CSL-JSON, EndNote XML export, and
  the Zotero local API.
- **Distribution.** Installs via `pipx`/`uv` and needs Python 3.9+ (see
  [Install And Run](#install-and-run)). Signed standalone installers
  (`.exe`/`.dmg`/AppImage) for users without Python are not yet provided.

Citation styling is intentionally out of scope. PoCo improves the underlying
reference metadata; your reference manager and citation style still control how
the final bibliography is formatted.
