Metadata-Version: 2.4
Name: h2hdb
Version: 0.18.0.0
Summary: A simple H@H database
Project-URL: Homepage, https://github.com/Kuan-Lun/h2hdb
Project-URL: Source, https://github.com/Kuan-Lun/h2hdb
Project-URL: Tracker, https://github.com/Kuan-Lun/h2hdb/issues
Author: Kuan-Lun Wang
License: GNU Affero General Public License v3
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.14
Requires-Dist: h2h-galleryinfo-parser>=0.4.0
Requires-Dist: mysql-connector-python<10.0.0,>=9.7.0
Requires-Dist: pillow<13.0.0,>=12.3.0
Requires-Dist: pydantic<3.0.0,>=2.13.4
Provides-Extra: dev
Requires-Dist: black>=26.5.1; extra == 'dev'
Requires-Dist: mypy>=2.3.0; extra == 'dev'
Requires-Dist: pymarkdownlnt>=0.9.39; extra == 'dev'
Requires-Dist: ruff>=0.16.0; extra == 'dev'
Description-Content-Type: text/markdown

# H2HDB

## Description

The `H2HDB` is a comprehensive database for organising and managing H@H comic
collections. It offers a streamlined way to catalogue your comics, providing
key information such as GID (Gallery ID), title, tags and more, ensuring your
collection is always organised and accessible.

---

## Features

- [x] Add new galleries to the database
- [x] Comporess H@H's galleries to a folder
- [x] Record the removed GIDs in a separate list
- [x] Coordinate bounded downloader and database-ingest turns
- [ ] Write document (need?)

---

## Installation and Usage

1. Install [uv](https://docs.astral.sh/uv/getting-started/installation/).
   It manages the Python version and dependencies for you.
1. Install the required packages.

    ```bash
    uv pip install h2hdb
    ```

1. Run the script.

    ```bash
    uv run python -m h2hdb --config [json-path]
    ```

### Config

```json
{
    "h2h": {
        "download_path": "download",
        "cbz_max_size": 768,
        "cbz_grouping": "flat",
        "cbz_sort": "no"
    },
    "database": {
        "sql_type": "mariadb",
        "host": "localhost",
        "port": 3306,
        "user": "root",
        "password": "password",
        "database": "h2h"
    },
    "maintenance": {
        "optimize_enabled": true,
        "min_interval_seconds": 604800,
        "min_work_units": 1000,
        "min_data_free_bytes": 268435456,
        "min_data_free_ratio": 0.2,
        "lock_wait_seconds": 300
    },
    "logger": {
        "level": "INFO"
    }
}
```

- `h2h.download_path`: H@H download path. The default is `download`.
- `h2h.cbz_path`: directory for CBZ output. Unset (the default) disables CBZ
  output entirely; if given, it must be a non-empty path (`""` is rejected).
- `h2h.cbz_max_size`: maximum image size. The default is `768`.
- `h2h.cbz_grouping`: `flat`, `date-yyyy`, `date-yyyy-mm`, or
  `date-yyyy-mm-dd`. The default is `flat`.
- `h2h.cbz_sort`: `no`, `upload_time`, `download_time`, `gid`, `title`,
  `pages`, or `pages+[num]`. The default is `no`.
- `h2h.file_hash_workers`: maximum number of files read and hashed
  concurrently. The default is the smaller of `4` and the available CPU count;
  set it to `1` for serial hashing. Valid values are `1`–`32`.
- `database.sql_type`: `mariadb` or `sqlite`. The default is `mariadb`.
  Existing config files that still use `mysql` must update this field.
- `database.host`, `database.port`, `database.user`, and `database.password`
  are only used for `mariadb`.
- `database.database`: for `mariadb`, this is the database name. For `sqlite`,
  this is the path to the database file.
- `maintenance.optimize_enabled`: enables automatic optimization in the
  resident main loop. Manual `H2HDB.optimize_database()` calls remain
  unconditional.
- `maintenance.min_interval_seconds`: minimum time between automatic
  optimization evaluations. The default is seven days (`604800`).
- `maintenance.min_work_units`: changed or removed galleries accumulated
  before an evaluation. New galleries do not count. The default is `1000`.
- `maintenance.min_data_free_bytes` and `maintenance.min_data_free_ratio`:
  minimum reclaimable space a table (or the SQLite database) must satisfy.
  Both thresholds must pass; the defaults are 256 MiB and 20%.
- `maintenance.lock_wait_seconds`: one wait interval for the MariaDB
  cross-process database gate. The default is 300 seconds. A timeout is logged
  and retried rather than terminating the caller.
- `logger.level`: one of `NOTSET`, `DEBUG`, `INFO`, `WARNING`, `ERROR`, or
  `CRITICAL`.

The main entry point remains resident and keeps its 30-minute periodic scan
deadline, but it now polls the durable `gallery_ingest_state` every five seconds
while waiting. A live downloader lease defers that scan; otherwise an ingest
request wakes h2hdb without waiting for the old uninterruptible 30-minute
sleep.

The singleton coordination row moves through
`INGEST_REQUESTED → INGESTING → READY → DOWNLOADING`. A newly added row starts
at `INGEST_REQUESTED`, so an upgraded or new installation completes one
baseline scan before a downloader can claim `READY`. Downloader integrations
use these public methods:

- `claim_download_turn(lease_seconds=...)` returns a generation-fenced
  `DownloadTurn`, or `None` while h2hdb owns the turn.
- `renew_download_turn(turn, lease_seconds=...)` extends a live downloader
  lease and returns whether that token still owns the generation.
- `request_gallery_ingest(turn)` idempotently hands the generation to h2hdb.
- `finish_download_turn(turn, request)` atomically hands off a successful root
  traversal and conditionally deletes only that request token.
- `finish_missing_download_turn(turn, request, gid)` atomically fences and hands
  off a coordinated lookup; only while that exact request token is still current
  does it record the GID as removed and delete the request.
- `complete_missing_download_request(request, gid)` records a confirmed missing
  gallery and deletes a direct request in one transaction, only while that exact
  token is still current.
- `clear_removed_gallery_gid(gid)` clears a prior missing result after a later
  lookup finds the gallery again.
- `get_gallery_ingest_state()` exposes the durable phase and
  `completed_generation`; a downloader may start its next root request only
  after `completed_generation >= turn.generation`.

A fresh `DOWNLOADING` lease prevents h2hdb from starting a scan until the
downloader requests handoff; a periodic deadline does not override it. If the
downloader terminates, h2hdb takes over after the lease expires and ingests any
complete gallery folders already published. An ingest acknowledgement is
written only after repeated `synchronize_once()` calls converge to a pass with
no new or changed galleries and scheduled maintenance succeeds. A background
heartbeat renews the ingest lease throughout synchronization and MariaDB
maintenance. SQLite lock contention is retried only within the current lease;
before SQLite maintenance h2hdb renews once and stops the heartbeat so
`VACUUM`'s exclusive lock can fence competing coordination writers. A
successful SQLite optimization can acknowledge after the timestamp expires
only if its generation and owner token are still current. Another resident
treats SQLite lock contention while claiming as temporarily unavailable and
keeps polling. Owner tokens fence stale downloader and h2hdb processes from
renewing or completing a newer turn. Persisted handoff provenance distinguishes
an explicit live-token handoff from an expired downloader lease recovered by
h2hdb, so a recovered stale token cannot later report success.

One download turn covers one root `todownload_gids` request and its complete
deep traversal. A failed, cancelled, or interrupted traversal leaves that
durable request in the queue and may use `request_gallery_ingest()` to hand off
any complete files. A successful traversal uses `finish_download_turn()` so
handoff and exact-token deletion share one transaction. If the same GID was
already re-enqueued with a newer request token, deletion is a no-op: the
completed live turn still hands off immediately and the newer request remains
queued. A confirmed missing result uses `finish_missing_download_turn()` for a
coordinated root or `complete_missing_download_request()` for a direct request,
so the removed marker and exact-token deletion cannot be partially committed
and a stale token cannot write either. A later successful lookup calls
`clear_removed_gallery_gid()` to repair that marker; replaying the older
completion cannot restore it. If the turn was already handed off by the generic
failure path, a later finish call acknowledges that handoff without performing
any success or missing mutations.

Completed removal and changed-gallery batches add work to the singleton
`database_maintenance_state` row. Automatic optimization is evaluated only
after both the work and time thresholds pass, and MariaDB runs `OPTIMIZE TABLE`
only for base tables that also pass both reclaimable-space thresholds. h2hdb
clients can wrap short database work in `H2HDB.database_gate()` so it waits
while maintenance owns the same MariaDB named lock.

H2HDB records a source-filename manifest for each gallery. Adding, deleting, or
renaming a source file marks that gallery's CBZ for rebuilding. This pending
state remains in the database if CBZ output is disabled or a run is interrupted.
During ingestion, the provisional pass creates only missing CBZ files for new
galleries and preserves existing CBZ files. After all galleries have been
processed, one final pass uses the stable exclusion set to perform any required
rebuilds. Created CBZ files carry a small input-layout marker in their ZIP
comment, allowing the final pass to detect normalized renames and filename
swaps even after the database has been deleted and rebuilt. H2HDB does not hash
or scrub CBZ file contents.

---

## Q & A

- Why are some images missing from the CBZ-files?

`H2HDB` does not compress images that are considered spam according to certain
rules. If you encounter any images that you believe should have been included,
please report the issue.

- Why are some images in some CBZ files and not in other CBZ-files?

`H2HDB` learns the spam rule from the previous CBZ files. If you kill the CBZ
files containing these images, the new CBZ files will not contain these images.

---

## Credits

The project was created by [Kuan-Lun Wang](https://www.klwang.tw/home/).

---

## License

This project is distributed under the terms of the GNU General Public Licence
(GPL). For detailed licence terms, see the `LICENSE` file included in this
distribution.
