Metadata-Version: 2.5
Name: invenio-pidbox
Version: 0.2.8
Summary: InvenioRDM PIDBox integration for people, organizations, and works.
Project-URL: Homepage, https://codeberg.org/front-matter/invenio-pidbox
Project-URL: Repository, https://codeberg.org/front-matter/invenio-pidbox
Project-URL: Bug Tracker, https://codeberg.org/front-matter/invenio-pidbox/issues
License: MIT
License-File: LICENSE
Keywords: author,citation,invenio,organization,pidbox
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.14
Requires-Dist: commonmeta-py[backend]>=0.272
Requires-Dist: commonmeta-rs>=0.9.80
Requires-Dist: invenio-app-rdm>=14.0.0
Provides-Extra: tests
Requires-Dist: invenio-app>=1.4.0; extra == 'tests'
Requires-Dist: invenio-rdm-records>=14.0.0; extra == 'tests'
Requires-Dist: invenio-search[opensearch2]>=3.0.0; extra == 'tests'
Requires-Dist: pytest-black>=0.3.0; extra == 'tests'
Requires-Dist: pytest-isort>=3.1.0; extra == 'tests'
Requires-Dist: pytest<9,>=7.4.0; extra == 'tests'
Description-Content-Type: text/markdown

# invenio-pidbox

InvenioRDM integration that serves people, organizations and citations from a
commonmeta SQLite store, read through commonmeta-py and commonmeta-rs.

The store is the one a commonmeta import already maintains — ORCID persons, ROR
organizations, works and the relations between them. This package reads it, and
adds no tables of its own: no application-specific people or organizations
tables, no citation store, no schema. Everything it exposes is a view onto that
one file.

## Features

**Profiles** — people and organization profile pages for InvenioRDM, with list
and search pages, template context injected on the profile routes, and a
stylesheet and search-result components shipped as webpack bundles.

**Deposit autocomplete** — creator, affiliation and funder suggestions served
from commonmeta-rs Tantivy search, with the deposit form's overridable
components registered to route lookups to pidbox when configured.

**Record relations** — affiliations and funders resolved from the store as well
as the vocabularies, so a record may name any organization the store knows.
Opt-in, because it changes which records an instance accepts.

**Facet labels** — search facets fall back to the store for ids the vocabulary
cannot label, so an organization accepted by the relations above does not appear
in the sidebar as a bare ROR id.

**Citations** — citation counts and citing works, read from the store's
`pid_relations` table and served as commonmeta works, plus a
`pidbox:citations` custom field for the works a record itself cites.

**Bounded store access** — every read runs with a deadline behind a circuit
breaker, so a store that has gone slow degrades the feature that reads it
instead of holding request threads.

## Requirements

- Python 3.14+
- A commonmeta SQLite database readable through commonmeta-rs

Internally, commonmeta-rs reads `pid_annotations` records with source_id 4 for
people (ORCID) and source_id 3 for organizations (ROR), works from
`pid_records`, and citations from `pid_relations`.

## Installation

```bash
uv add invenio-pidbox
```

This package depends on commonmeta-py, commonmeta-rs, and invenio-app-rdm at
runtime, so installing invenio-pidbox pulls in the profile backend
automatically.

## Configuration

### Required

- `PIDBOX_SQLITE_PATH` — path to the commonmeta SQLite store, used by every
  feature below. Defaults to `/var/lib/commonmeta/commonmeta.sqlite3`.

### Deposit autocomplete

- `APP_RDM_DEPOSIT_FORM_AUTOCOMPLETE_NAMES`
- `APP_RDM_DEPOSIT_FORM_AUTOCOMPLETE_NAMES_PROVIDER`
- `PIDBOX_AUTOCOMPLETE_API`

To enable pidbox autocomplete in the deposit form, set:

```python
APP_RDM_DEPOSIT_FORM_AUTOCOMPLETE_NAMES = "search"
APP_RDM_DEPOSIT_FORM_AUTOCOMPLETE_NAMES_PROVIDER = "pidbox"
```

The endpoint URLs can be overridden with:

```python
PIDBOX_AUTOCOMPLETE_API = {
    "names": "/api/pidbox/names",
    "affiliations": "/api/pidbox/affiliations",
    "funders": "/api/pidbox/funders",
}
```

The funders endpoint returns only organizations with `"funder"` in their `types`
list, so the deposit form offers exactly the set a commit will accept.

### Record relations

```python
PIDBOX_RECORD_RELATIONS = True
```

Off by default. InvenioRDM validates every affiliation and funder id on each
record commit against the `affiliations` and `funders` vocabularies, so a record
naming an organization that was never imported into them is rejected with
`InvalidRelationValue`. With this on, the store is consulted first and the
vocabulary second, making the accepted set the union of the two rather than a
swap — an unreadable store degrades to exactly the stock behaviour.

Turning this on changes which records an instance will accept, which is why
installing the package does not.

### Facet labels

Facets label their buckets from the vocabularies, so an organization accepted by
the relations above but absent from the vocabulary shows as a raw ROR id. The
store-backed labels fill only those gaps, asking the vocabulary first so a
curated, localized title keeps winning:

```python
from invenio_pidbox.facets import StoreAffiliationsLabels, StoreFundersLabels
```

Use them where the facet is declared, in place of the stock
`AffiliationsLabels` / `FundersLabels` — for example
`value_labels=StoreFundersLabels("funders")`.

### Store access

- `PIDBOX_STORE_TIMEOUT` — seconds a single store read may take before the
  caller gives up (default `2.0`; `0` disables the deadline)
- `PIDBOX_STORE_FAILURE_THRESHOLD` — consecutive timeouts before the store is
  treated as unavailable (default `3`)
- `PIDBOX_STORE_RECOVERY_TIME` — seconds to leave an unavailable store alone
  before trying again (default `60.0`)

These exist because a store shared with a writer can get very slow: reads that
normally take a millisecond have blocked for minutes while an import ran against
the same file, and since a relation is resolved on every record commit, that was
enough to exhaust the request threads and have gunicorn kill the worker. A slow
store is therefore treated as an unavailable one.

## Routes

| Route | Purpose |
|---|---|
| `/people/`, `/organizations/` | profile list and search pages |
| `/people/<orcid>` | person profile page |
| `/organizations/<ror>` | organization profile page |
| `/api/pidbox/names` | creator autocomplete |
| `/api/pidbox/affiliations` | affiliation autocomplete |
| `/api/pidbox/funders` | funder autocomplete |
| `/api/pidbox/citations/<doi>` | works citing a DOI |

Each API route is also registered without the `/api` prefix, because the same
extension is registered on the UI app and the API app, and Invenio mounts the
API app *at* `/api` and strips the prefix.

Autocomplete requests carrying `?suggest=` are capped at 50 results; the profile
search pages are fixed to 10 results per page and 100 in total.

### Citations

```http
GET /api/pidbox/citations/<doi>
```

`<doi>` is a bare DOI or a DOI URL. The response uses the usual hits envelope,
and each hit is a commonmeta work, unchanged:

```json
{"hits": {"hits": [{"id": "https://doi.org/10.1234/a", "title": "Work a"}], "total": 3}}
```

Which relations count as a citation (`References`, `Cites`, `IsSupplementedBy` —
DataCite's definition) is decided by commonmeta-rs, not by this package.

`total` is the citation count and can exceed the number of hits: it is read from
the relation index, while hits are hydrated works, so a citing DOI whose work
has not been imported yet is counted but cannot be listed.

Listing hydrates every citing work, so `size=0` returns just the count, which is
answered from an index and stays cheap on a heavily cited DOI. Otherwise `size`
(default 10, max 100) and `page` paginate the response.

### The `pidbox:citations` custom field

Records can also carry the works they cite, as a list of
`{identifier, scheme, reference}` entries indexed with the record. This is the
record-side view of a citation, and it is independent of the store-side one
above — the field says what a record cites, the store says what cites a DOI.
They point in opposite directions, so their numbers are not expected to match.

The field is not registered automatically, because a custom field has to exist
in the search mapping before a record can use it. To enable it:

```python
from invenio_pidbox.custom_fields import (
    CITATIONS_FACET,
    CITATIONS_QUERY_FIELD,
    CitationsCF,
    PIDBOX_NAMESPACE,
)

RDM_NAMESPACES = {**RDM_NAMESPACES, **PIDBOX_NAMESPACE}
RDM_CUSTOM_FIELDS = [*RDM_CUSTOM_FIELDS, CitationsCF(name="pidbox:citations")]
RDM_FACETS = {**RDM_FACETS, "citations": CITATIONS_FACET}
```

Then create the mapping:

```bash
invenio rdm-records custom-fields init -f pidbox:citations
```

`CITATIONS_QUERY_FIELD` is a `QueryParser` mapping entry, so adding
`"citations": CITATIONS_QUERY_FIELD` to `RDM_SEARCH`'s parser mapping makes
`citations:10.5555/12345678` searchable. Declaring `CITATIONS_FACET` does not
display it — the facet also has to be named in `RDM_SEARCH`'s `facets` list.

commonmeta-rs fills the field when it pushes a record: citing works become
`IsReferencedBy` relations, the InvenioRDM serializer maps those to
`custom_fields["pidbox:citations"]`, and `commonmeta push`/`put --to inveniordm`
sends them to the REST API. Reading records back reverses it. So the store stays
the source of truth and the field is a projection of it onto each record.

That name is hardcoded in commonmeta-rs, which is why this package requires
commonmeta-rs >= 0.9.80. Earlier versions write `rs:citations`, and against them
the field is defined, indexed, and never populated.

This field comes from Rogue Scholar, where it was `rs:citations`. It is
unchanged apart from the namespace, and the two names are different fields: an
instance holding records under the old one has to migrate and reindex them, and
until it does, those records read as having no citations.

### Keeping the field current

A push writes the citations a record had *at that moment*. What makes it stale
afterwards is other records arriving, which is not an event on this record, so
nothing in InvenioRDM would ever notice. The package therefore schedules a
weekly sweep:

```text
invenio_pidbox.tasks.refresh_citations   Sundays at 03:20
```

It is registered as `pidbox_refresh_citations` in `CELERY_BEAT_SCHEDULE`, and
only if the instance has not already claimed that key — declare it yourself to
change the cadence, or point it at nothing to drop the sweep. It needs the
`--beat` worker that a standard deployment already runs.

For every record with a DOI it compares `count_sqlite_citations` — an index
lookup — against the number of citations the record already stores, and skips
the record without hydrating anything when they agree. Only a mismatch pays for
the full read, and only a genuine difference in identifiers is written back, as
a metadata edit through the records service (same version, same DOI, new
revision).

Run it by hand, optionally bounded, with:

```python
from invenio_pidbox.tasks import refresh_citations

refresh_citations(max_records=100)   # {"scanned", "hydrated", "updated", "failed"}
```

Two behaviours worth knowing. A record cited by a work the store has not
imported has a count higher than anything that can be listed, so it takes the
slow path on every run and writes nothing — correct, but not free. And the
sweep never clears a record's citations: every store read degrades to "none"
when the store is unavailable, which is indistinguishable from a DOI nothing
cites, so a record that holds citations keeps them rather than risk emptying
every record in the instance.

## How It Works

When a request path matches `/people/<orcid>` or `/organizations/<ror>`, the
extension loads the corresponding commonmeta record from SQLite and exposes
template-friendly objects such as person, organization, employment, identifiers,
relations, location, orcid, ror, and search_config. It also registers the
profile routes and a `country_name` template filter, so it runs against a
standard invenio-app-rdm installation.

For deposit autocomplete it patches invenio-app-rdm's form config at runtime to
inject the endpoint URLs, and loads overridable components that switch the
person, organization and funder remote selects to those endpoints. Awards lookup
stays on the existing awards API.

Reads are optional throughout, and each has a defined fallback: a profile falls
back to a record search, an autocomplete to no suggestions, a relation to the
vocabulary, a facet label to the bare id, a citation listing to none. An absent,
unreadable or slow store therefore costs a feature, not the page.

## Development

```bash
uv sync --extra tests
uv run pytest
```

## Entry points

| Group | Name | Value |
|---|---|---|
| `invenio_base.apps` | `invenio_pidbox` | `invenio_pidbox.ext:InvenioPidbox` |
| `invenio_base.api_apps` | `invenio_pidbox` | `invenio_pidbox.ext:InvenioPidbox` |
| `invenio_assets.webpack` | `invenio_pidbox` | `invenio_pidbox.webpack:profiles` |

## License

MIT
