Metadata-Version: 2.4
Name: remarkable-tools
Version: 0.8.0
Summary: A set of tools to interact with reMarkable tablets
Keywords: remarkable,pdf
Author: Gustaf Hendeby
Author-email: Gustaf Hendeby <hendeby@gmail.com>
License-Expression: MIT
License-File: LICENSE.md
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Requires-Dist: pikepdf==10.16.0
Requires-Dist: platformdirs==4.13.0
Requires-Dist: requests==2.34.2
Requires-Dist: rmscene==0.8.0
Requires-Dist: textual==8.2.8
Requires-Dist: textual-fspicker==1.0.1
Requires-Python: >=3.14, <3.16
Project-URL: Homepage, https://remarkable.hendeby.se/
Description-Content-Type: text/markdown

# remarkable

Utilities for interacting with a reMarkable tablet and post-processing exported PDFs.

This project installs two command-line scripts:

- `remarkable`: terminal UI and automation commands for browsing, downloading, and uploading documents.
- `remarkable-hlfix`: PDF post-processing tool that reduces highlight opacity.

## Installation

Install from PyPI:

```bash
pip install remarkable-tools
```

or with uv:

```bash
uv add remarkable-tools
```

To install as a standalone tool (makes `remarkable` and `remarkable-hlfix` available globally without a project):

```bash
uv tool install remarkable-tools
```

For development, install the package in your current environment:

```bash
uv sync
```

Then run scripts with `uv run`:

```bash
uv run remarkable --help
uv run remarkable-hlfix --help
```

If installed as a tool or package entry point, you can call `remarkable` and `remarkable-hlfix` directly.

## Script: remarkable

`remarkable` connects to the tablet over the reMarkable local interface (`http://10.11.99.1`) and supports both interactive and one-shot modes.

### Usage

Start the interactive browser:

```bash
remarkable
```

Start in a specific folder title path:

```bash
remarkable "Work/Meetings"
```

Download newest document and exit:

```bash
remarkable --get-newest
```

The document title and the path it was saved to are printed to stdout.

CLI downloads replace existing files with the same output name without prompting.
Documents and optional tag reports are staged in the output directory and published atomically, so a failed write does not leave a partially written destination.
On POSIX systems, replacing a regular file preserves its read/write/execute permissions; new outputs are private (`0600`).
Symlinks are replaced rather than followed, and special permission bits are not copied.
The document and its tag report are published separately; a report-write failure does not undo a successful document download.

Download newest document matching a tag suffix:

```bash
remarkable --get-newest --tag _notes
```

Download newest document into a specific directory:

```bash
remarkable --get-newest --output-dir ./downloads
```

Upload a file and exit:

```bash
remarkable --upload-file ./document.pdf
```

Uploads are not retried automatically, including connection failures and lost responses, to avoid submitting the same document twice.
Listing and download requests retain their retry/backoff policy.

Disable highlight fixing for downloaded PDFs:

```bash
remarkable --no-hl-fix
```

Embed page tags as bookmarks in downloaded PDFs:

```bash
remarkable --pdf-tag-bookmarks
```

Embed an `Annotations` tag bookmark for pages with reMarkable page material:

```bash
remarkable --pdf-annotation-bookmarks
```

This reads `.rmdoc` scene data and ignores deleted page material left behind by the device.

Write a JSON tag report with extracted page-tag data next to downloaded files
(for example `My Note.pdf` → `My Note.tags.json`):

```bash
remarkable --tag-json-report
```

Show installed/runtime version:

```bash
remarkable --version
```

Show where config files are searched (in load order):

```bash
remarkable --show-config-paths
```

Write logs to a custom location:

```bash
remarkable --log-file /tmp/remarkable-ui.log
```

### Exit codes (one-shot mode)

When running one-shot actions (`--get-newest` or `--upload-file`), `remarkable` exits
with operation-specific status codes:

| Exit code | Meaning |
|---|---|
| `0` | Success (or no documents found for `--get-newest`). |
| `2` | Invalid arguments/configuration, or initialization/connect failure while creating the device session. |
| `3` | Download failure during `--get-newest`. |
| `4` | Upload failure during `--upload-file`. |
| `5` | Specified starting directory not found when attempting to follow it. |

On these failures, a concise error message is printed to stderr indicating
whether the operation timed out, could not reach the device, or failed due to
an HTTP/network error or a local file-write error.

### Interactive keys

Global:

- `q`: quit
- `Ctrl+S`: save a screenshot of the current TUI view (`remarkable-ui-YYYYMMDD-HHMMSS.svg`)
- `F5`: refresh listing
- `h`: toggle highlight fix for PDF downloads
- `b`: toggle PDF tag-bookmark embedding for PDF downloads
- `a`: toggle PDF annotation-bookmark embedding for PDF downloads
- `H`: toggle snap-to-text → `/Highlight` annotation conversion for PDF downloads
- `t`: set a tag filter used by downloads (Enter confirms, Esc cancels)
- `/`: filter the current view (type to narrow, Enter to apply, Esc to clear)
- `←` / `→` or `Tab` / `Shift+Tab`: move between the folder and document panes
- `Backspace`: go up one folder

Selecting a folder (`Enter` or click) opens it. With a document selected:

- `d`: download as PDF (honours the HL Fix toggle)
- `D`: download as `.rmdoc`
- `u`: upload a file from disk (opens a file picker)

The status bar shows the current location (`/ Books / Papers`) and a compact
summary of PDF settings. The active filter and download progress appear only
while in use, and the tag appears only when set. Pane titles show item counts,
and switch to `matched/total` while a filter is active.

Downloads and uploads run in the background; a toast notification reports
success or failure so the UI stays responsive.
Concurrent downloads show aggregate progress and remain visible until the last active transfer finishes.
If any active transfer has an unknown length, the indicator shows received bytes instead of a percentage.
Folder refresh and navigation also run in the background, with a loading indicator in the pane borders.
The previous path, listing, and filter remain intact if a folder request fails.
While a folder request is in flight, further navigation is declined with a notification; repeated refreshes, including those after uploads, are coalesced into one follow-up refresh.
The TUI initially displays the listing fetched during connection rather than requesting it again on mount.
If a document download would replace an existing file, the TUI asks before downloading and defaults to Cancel.
The destination is checked again at publication; unchanged files reuse the approval, while newly appeared or modified files require confirmation.
Existing tag reports are confirmed separately; declining a report replacement keeps that report without undoing the saved document.
The TUI's no-clobber publication requires an output filesystem supporting hard links; otherwise the download fails safely instead of risking an overwrite.
you can always confirm active settings before downloading.

## Script: remarkable-hlfix

`remarkable-hlfix` rewrites a PDF so reMarkable-style highlights are no longer fully opaque.

Default behavior:

- Targets known reMarkable highlight colors.
- Reduces fully opaque highlight fill and stroke opacity from `1.0` to `0.4`.
- Scopes opacity changes to each highlight paint, preserving existing graphics-state scopes (`q`/`Q`) and clipping paths so subsequent non-highlight content retains its original opacity.
- Detects snap-to-text highlights (clean axis-aligned rectangles of approximately one line height) and converts them into native PDF `/Highlight` annotations. The original drawing is removed for the converted lines, so PDF readers render them as real text-markup highlights that can be selected and copied with the underlying text. Freehand (non-snapped) highlights fall back to the transparency pass. See [docs/snap-to-annotations.md](docs/snap-to-annotations.md) for the heuristic's design rationale and ideas for future improvement.

### Usage

Basic conversion:

```bash
remarkable-hlfix input.pdf output.pdf
```

Run with default filenames (`input.pdf` -> `output.pdf`):

```bash
remarkable-hlfix
```

Apply to all fully opaque filled shapes instead of only known highlight colors:

```bash
remarkable-hlfix input.pdf output.pdf --all-colors
```

Tune minimum stroke width required before applying stroke transparency:

```bash
remarkable-hlfix input.pdf output.pdf --min-stroke-width 4.0
```

Disable snap-to-text annotation conversion (keeps all highlights as drawn rectangles and only applies the transparency pass):

```bash
remarkable-hlfix input.pdf output.pdf --no-snap-annotations
```

Detect highlight palette from one page and print a paste-ready color block:

```bash
remarkable-hlfix input.pdf --calibrate-palette --calibration-page 1
```

Show installed/runtime version:

```bash
remarkable-hlfix --version
```

Show where config files are searched (in load order):

```bash
remarkable-hlfix --show-config-paths
```

Write logs to a custom location:

```bash
remarkable-hlfix --log-file /tmp/remarkable-hlfix.log
```

## Configuration

Both scripts load defaults from the same config files (later files override earlier ones).

To see the exact search order on your machine, run:

```bash
remarkable --show-config-paths
```

or:

```bash
remarkable-hlfix --show-config-paths
```

This works both in project environments (`uv run ...`) and for globally installed tools (`uv tool install ...`).

Preferred config locations follow OS conventions:

- Linux: `$XDG_CONFIG_DIRS/remarkable/config` (default includes `/etc/xdg`), then `$XDG_CONFIG_HOME/remarkable/config` (or `~/.config/remarkable/config`)
- macOS: `/Library/Application Support/remarkable/config`, then `~/Library/Application Support/remarkable/config` (Homebrew Python may add `$HOMEBREW_PREFIX/share/remarkable/config` before the system path; XDG config variables override these defaults)
- Windows: `%PROGRAMDATA%/remarkable/config`, then `%APPDATA%/remarkable/config`

Quick start (enable tag bookmarks by default):

```toml
[general]
pdf_tag_bookmarks = true
```

You can still override this per run with `--pdf-tag-bookmarks` or `--no-pdf-tag-bookmarks`.

Expected format:

```toml
[general]
hl_fix = true
pdf_tag_bookmarks = false
pdf_annotation_bookmarks = false
log_file = ""
tag = ""
url = "http://10.11.99.1"

[highlight]
alpha = 0.4
alpha_threshold = 0.999
min_stroke_width = 0.1
color_tolerance = 0.1
snap_to_annotations = false
snap_line_height_min = 4.0
snap_line_height_max = 32.0
snap_min_aspect = 0.5
snap_annotation_alpha = 1.0
snap_merge_path = false
```

All fields are optional and fall back to the defaults shown above. Command-line arguments override config values.

Known settings are validated before device access or PDF processing; invalid settings exit with code `2` and an error naming the option and, for file settings, its section and path.
Boolean settings require TOML booleans (`false`, not `"false"`); string settings require strings, and numeric settings require finite numbers (integers are accepted).
Opacity values must be within `0..1`; stroke width, color tolerance, and minimum aspect ratio must be non-negative.
Minimum line height must be positive, and maximum line height must be at least the minimum.
Custom `hl_palette` tables use known color names and three RGB components within `0..1` per color.
Unknown options continue to be warned about and ignored.

### Page tag report format

When `--tag-json-report` is enabled, each download writes a `*.tags.json`
report alongside the downloaded file.

The report JSON includes structured sections: `document`, `summary`, `pages`, and `raw`.

This report flag is command-line only (no config option and no TUI toggle).

### [general] options

| Option | Default | Description |
|---|---|---|
| `hl_fix` | `true` | Whether to apply highlight fixing when downloading PDFs via `remarkable`. |
| `pdf_tag_bookmarks` | `false` | Whether to embed page tags from reMarkable metadata as bookmarks in downloaded PDFs. |
| `pdf_annotation_bookmarks` | `false` | Whether to add an `Annotations` tag bookmark listing pages with reMarkable material. |
| `log_file` | `""` | Path to the script log file used by both `remarkable` and `remarkable-hlfix`. If empty, a platform default is used. |
| `tag` | `""` | Tag suffix filter used by `--get-newest`. |
| `url` | `"http://10.11.99.1"` | URL of reMarkable (do not change unless reMarkable changes standard endpoint). |

Default log path when `log_file = ""`:

- `remarkable`:
	- Linux: `$XDG_STATE_HOME/remarkable/log/remarkable_ui.log` (fallback: `~/.local/state/remarkable/log/remarkable_ui.log`)
	- macOS: `~/Library/Logs/remarkable/remarkable_ui.log`
	- Windows: `%LOCALAPPDATA%/remarkable/Logs/remarkable_ui.log`
- `remarkable-hlfix`:
	- Linux: `$XDG_STATE_HOME/remarkable/log/remarkable_hlfix.log` (fallback: `~/.local/state/remarkable/log/remarkable_hlfix.log`)
	- macOS: `~/Library/Logs/remarkable/remarkable_hlfix.log`
	- Windows: `%LOCALAPPDATA%/remarkable/Logs/remarkable_hlfix.log`

### [highlight] options

These control `remarkable-hlfix` behavior. Command-line flags (where available) override config values.

| Option | Default | Description |
|---|---|---|
| `alpha` | `0.4` | Target opacity applied to highlight fill and stroke. |
| `alpha_threshold` | `0.999` | Minimum opacity for a shape to be considered fully opaque and eligible for reduction. |
| `min_stroke_width` | `0.1` | Minimum stroke width (in PDF units) required before stroke opacity is reduced. Raise this to leave thin lines untouched. |
| `color_tolerance` | `0.1` | Per-channel tolerance when matching fill colors against the known highlight palette. |
| `snap_to_annotations` | `false` | Convert snap-to-text highlight rectangles into native PDF `/Highlight` annotations. Disabled by default because the detector is heuristic; enable it when the source PDF reliably uses snap-to-text highlights. Detection is per-subpath, so a path containing both a snap rectangle and other shapes only converts the rectangle. Combined fill+stroke paint operators (`B`, `b`, `B*`, `b*`) are never converted, to avoid silently dropping the stroke. |
| `snap_line_height_min` | `4.0` | Minimum rectangle height (PDF user-space units, 1/72") that qualifies as a single text line. |
| `snap_line_height_max` | `32.0` | Maximum rectangle height that qualifies as a single text line. |
| `snap_min_aspect` | `0.5` | Minimum width/height ratio required to treat a rectangle as text-line shaped. |
| `snap_annotation_alpha` | `1.0` | Opacity (`/CA`) written into the generated `/Highlight` annotation. |
| `snap_merge_path` | `false` | When a single snap-to-text path produces several rectangles (a multi-line highlight), emit them as a single `/Highlight` annotation with one `/QuadPoints` quad per line. Disabled by default so each line remains its own annotation, matching what earlier versions produced; enable it to get one annotation per user-intent highlight. |

## Troubleshooting

### remarkable cannot connect to device

Symptoms:

- Requests fail or time out.
- You see connection errors when listing folders or downloading files.

Checks:

1. Ensure the tablet is connected over USB and USB web interface access is enabled.
2. Verify the endpoint is reachable:

```bash
curl -I http://10.11.99.1/documents/
```

3. If needed, retry after reconnecting the cable and restarting the tablet's USB web interface.

### config-related startup failure

Ensure the config file uses the correct TOML structure (see [Configuration](#configuration) above).

If you are testing quickly, run with explicit flags to avoid relying on config values.

### downloaded PDF highlights look unchanged

Try one of these:

1. Make sure highlight fixing is enabled (`--hl-fix` or toggle `h` in the UI).
2. Run a direct pass with `remarkable-hlfix`.
3. If your document uses non-standard colors, run calibration first:

```bash
remarkable-hlfix input.pdf --calibrate-palette --calibration-page 1
```

4. If highlights are very thin, lower the stroke threshold:

```bash
remarkable-hlfix input.pdf output.pdf --min-stroke-width 1.0
```

### remarkable-hlfix input errors

Common causes:

- Input path is wrong.
- File is encrypted or malformed.
- Calibration page number does not exist in the PDF.

Checks:

1. Confirm file paths are correct.
2. Start with the basic command:

```bash
remarkable-hlfix input.pdf output.pdf
```

3. If that fails, try opening and re-saving the PDF with another tool, then run again.
