Metadata-Version: 2.4
Name: wikidata-ner-classifier
Version: 0.7.0
Summary: Hierarchical Wikidata item classification and mention-focused LLM type inference
Author: Roberto Avogadro
License-Expression: MIT
Project-URL: Homepage, https://github.com/roby-avo/ner-wikidata
Project-URL: Issues, https://github.com/roby-avo/ner-wikidata/issues
Project-URL: Source, https://github.com/roby-avo/ner-wikidata
Keywords: wikidata,ner,entity-linking,knowledge-graph,classification
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# Wikidata NER Classifier 0.7.0

Classification into one shared retrieval-oriented hierarchy through three
complementary paths:

1. `WikidataNERClassifier` classifies Wikidata items deterministically from
   P31/P279 token clues and optional descriptions.
2. `OpenRouterNERClassifier` infers the type of one target mention from free
   text, a structured record, or tabular context using an LLM.
3. `HierarchicalNERPredictor` routes unlinked free-text mentions and table
   columns/cells through deterministic candidate generation, with an optional
   contrastive LLM resolver over at most ten fine types.

All paths return classes from the packaged hierarchy:

```text
coarse_type -> fine_type -> subtype -> specific_type
```

## Source-first hierarchical prediction

`HierarchicalNERPredictor` is the pre-retrieval API. It never links an entity or
returns a QID. Coarse and fine vocabularies, subtype parents, facets, and derived
retrieval types all come from the packaged rule files through `HierarchyIndex`.

Free text:

```python
from wikidata_ner import HierarchicalNERPredictor

predictor = HierarchicalNERPredictor()  # deterministic; no LLM is required
prediction = predictor.predict_free_text(
    "Chrysler Cirrus",
    "The Chrysler Cirrus is a mid-size four-door sedan model.",
)

assert prediction.retrieval_path == (
    "PRODUCT",
    "VEHICLE_WEAPON_OR_EQUIPMENT_MODEL",
    "CAR_MODEL",
)
assert prediction.retrieval_key == (
    "PRODUCT/VEHICLE_WEAPON_OR_EQUIPMENT_MODEL/CAR_MODEL"
)
```

Table column and cell:

```python
stations = [
    "Roma Termini",
    "Milano Centrale",
    "Napoli Centrale",
    "Bologna Centrale",
]

column_prediction = predictor.predict_table_column(
    "Departure station",
    stations,
    neighboring_headers=["Arrival station", "Duration"],
    table_title="Italian high-speed train connections",
)
assert column_prediction.retrieval_path == (
    "FACILITY",
    "TRANSPORT_STATION",
    "RAILWAY_STATION",
)

cell_prediction = predictor.predict_table_cell(
    "Roma Termini",
    column_header="Departure station",
    row_context={
        "Departure station": "Roma Termini",
        "Arrival station": "Milano Centrale",
        "Duration": "3h 10m",
    },
    same_column_values=stations,
    neighboring_headers=["Arrival station", "Duration"],
    table_title="Italian high-speed train connections",
    column_prediction=column_prediction,
)
```

Column samples and row fields are bounded deterministically. Column predictions
are cached and act as soft cell priors; strong cell evidence can override them.
The generated retrieval plan always retains `item_category: ENTITY`:

```python
print(prediction.retrieval_plan.to_dict())
# {
#   "mode": "fine_type_filter_specific_type_boost",
#   "filters": {
#     "item_category": "ENTITY",
#     "coarse_type": "PRODUCT",
#     "fine_type": "VEHICLE_WEAPON_OR_EQUIPMENT_MODEL",
#   },
#   "boosts": {"ner_specific_types": ["CAR_MODEL"]},
#   ...
# }
```

To add contrastive resolution, inject `LLMTypeResolver(OpenRouterClient(...))`.
The resolver receives the target context and a general zero-shot rule. The
bounded controlled labels appear only as response-schema enums; type cards,
definitions, examples, and hierarchy paths are not placed in the prompt.
Application code validates the semantic decision and constructs every retrieval
field locally.
The main configuration dataclasses are `InputContextConfig`,
`CandidateScoringConfig`, `HierarchicalPredictorConfig`,
`RetrievalPolicyConfig`, `HierarchyCompatibilityConfig`, and
`CandidateRankingConfig`.

The dependency-free similarity score is deliberately lightweight. Applications
can inject a semantic-similarity callback or replace the in-memory hierarchy
index with a vector-backed implementation without changing the predictor API.

## Fast input prediction with Cerebras through OpenRouter

For free text and tabular inputs, the Cerebras-hosted LLM makes the prediction
from the target and its context with a general zero-shot semantic rule. The
packaged hierarchy is deliberately not serialized into the prompt. Returned
coarse types, fine types, subtypes, and facets are validated locally, and
specific types plus retrieval paths are constructed by the application.

```bash
export OPENROUTER_API_KEY='...'
```

```python
from wikidata_ner import OpenRouterNERClassifier

classifier = OpenRouterNERClassifier(
    model="openai/gpt-oss-120b",
    provider="cerebras",
    allow_fallbacks=False,
    reasoning_effort="low",
)
prediction = classifier.predict_text(
    "Rome Against Rome is a 1964 sword-and-sandal film.",
    mention="Rome Against Rome",
)

assert prediction.fine_type == "FILM"
print(prediction.specific_type)  # SWORD_AND_SANDAL_FILM
print(prediction.usage)
```

The general rule is deliberately ontology-agnostic:

> Classify the target by what it denotes in context. Prefer an explicit
> target-bound type or description, then target-bound relations and structure,
> and treat surface form or general knowledge as weak evidence. Choose the most
> specific schema-admitted type clearly supported; otherwise choose a broader
> admitted type or abstain.

Inference remains two-stage to keep Cerebras schemas small: first select a broad
coarse type, then a fine type and optional refinements within that branch. The
system prompts contain no taxonomy index, rule definitions, examples, clue
lists, or candidate cards. Controlled IDs live in the strict response schema,
and Python validates parentage and builds `ner_retrieval_key`,
`ner_retrieval_path`, `ner_retrieval_tags`, and specificity fields. Existing
`ner_*`, prior, popularity, URL, QID, and previous coarse/fine fields are stripped
from structured-record prompt input; fields such as `label`, `labels`, `aliases`,
`types`, and `description` remain available as semantic evidence.

### Multi-mention inference

`predict_many()` batches independent mention/context pairs into shared model
requests. Each request reuses the same general zero-shot rule; no hierarchy is
serialized into the batch prompt. Controlled coarse/fine labels remain in the
strict output schema and are checked again locally. Use
`MentionTask` when contextual text needs an explicit target:

```python
from wikidata_ner import (
    MAX_MENTIONS_PER_BATCH,
    MentionTask,
    OpenRouterNERClassifier,
)

assert MAX_MENTIONS_PER_BATCH == 8

classifier = OpenRouterNERClassifier(
    model="openai/gpt-oss-120b",
    provider="cerebras",
    allow_fallbacks=False,
    reasoning_effort="low",
    max_mentions_per_batch=8,
    max_batch_characters=80_000,
)

predictions = classifier.predict_many(
    [
        MentionTask(
            data="Rome Against Rome is a 1964 sword-and-sandal film.",
            mention="Rome Against Rome",
        ),
        MentionTask(
            data="Ada Lovelace was an English mathematician and writer.",
            mention="Ada Lovelace",
        ),
        MentionTask(
            data={"label": "Dune", "description": "1965 science-fiction novel"},
        ),
    ],
)

assert [prediction.fine_type for prediction in predictions] == [
    "FILM",
    "HUMAN",
    "BOOK_OR_WRITTEN_WORK",
]

coverage = classifier.hierarchy_coverage_report()
assert coverage["complete"] is True
assert coverage["prompt_embeds_hierarchy"] is False
assert coverage["strategy"] == "zero_shot_semantic_routing_local_validation"
assert coverage["source_file"] == "B_full_rule_spec.json"
assert coverage["coarse_type_count"] == 21
assert coverage["fine_type_count"] == 187
```

For each chunk, the classifier makes one shared coarse request and then one
shared fine/refinement request for each coarse branch present. If eight mentions
resolve to two coarse branches, this is three requests instead of the 16 requests
made by individual two-stage prediction. Results retain input order; numbered
task IDs keep identical surface forms with different contexts separate.

The absolute `MAX_MENTIONS_PER_BATCH` is 8. This keeps the controlled coarse and
fine enums inside Cerebras's expanded strict-schema budget even without a
hierarchy catalog in the prompt. The constructor can set a lower
instance ceiling with `max_mentions_per_batch`; omitting `batch_size` then uses
that ceiling. Longer iterables are chunked automatically, and the configured
prepared-context budget may split a chunk earlier. A method call cannot exceed
the instance ceiling or the hard library ceiling. A single large task is still
sent alone. Set `batch_size=1` to use the individual-request path.

Batch outputs use a fixed object with one required key per task and a shared
`$defs` result schema. This is intentional: Cerebras strict output supports
schema references but not `minItems`/`maxItems`, so array bounds cannot reliably
require one result for every mention. The output does not repeat
`target_mention`; the required result key binds each object back to the locally
stored target. This avoids asking the model for a field forbidden by the strict
batch schema. Cerebras expands the referenced result for every task when
counting property and enum strings. Batch schemas therefore constrain the exact
object shape and retain the primary coarse/fine enum. Every
returned coarse type, fine type, subtype, and facet is validated against the
packaged taxonomy locally.

Evidence strength is also enforced locally. A positive result cannot carry
`NONE`; such a result abstains instead. Model confidence is capped at 0.90 for
`CONTEXTUAL` evidence and 0.65 for `SURFACE_ONLY` evidence. Lowercase evidence
labels are normalized before validation.

Fine-stage batch output retains up to two locally validated secondary fine
types when the evidence is genuinely ambiguous. Unknown IDs, types from another
coarse branch, and the selected primary type are removed locally.

Every prediction exposes lossless NER tags at several granularities:

```python
print(prediction.ner_tag)       # most precise primary tag
print(prediction.ner_tags)      # flat coarse/fine/specific/facet tags
print(prediction.ner_tag_sets)

# {
#   "primary": ("SWORD_AND_SANDAL_FILM",),
#   "coarse": ("CREATIVE_WORK",),
#   "fine": ("FILM",),
#   "subtype": (),
#   "specific": ("SWORD_AND_SANDAL_FILM",),
#   "facets": ("GENRE:SWORD_AND_SANDAL",),
#   "flat": (...),
#   "hierarchical": (
#       "COARSE:CREATIVE_WORK",
#       "FINE:FILM",
#       "SPECIFIC:SWORD_AND_SANDAL_FILM",
#       "FACET:GENRE:SWORD_AND_SANDAL",
#   ),
# }
```

If secondary fine types are returned, they appear after the primary under
`ner_tag_sets["fine"]` and as `FINE_ALTERNATIVE:<ID>` in the hierarchical tag
set. These tags are derived locally and require no additional model request.

Token and timing values under `prediction.usage` describe the shared request,
so do not sum them across predictions from the same batch when calculating
cost. The usage metadata includes `shared_batch`, `batch_size`, and
`batch_task_id` for this reason. It also records the request ID, routed model,
and exact system-prompt, user-prompt, and strict-schema character counts.
For Cerebras requests it also records `cerebras_expanded_string_budget`, the
provider-relevant budget after shared definitions are conservatively expanded.

Tabular data has a dedicated batch helper. Each `TableCellTask` applies the same
bounded preprocessing as `predict_table_cell()` before entering the shared LLM
requests:

```python
from wikidata_ner import TableCellTask

table_tasks = [
    TableCellTask(
        cell="Rome Against Rome",
        column_header="title",
        row_context={
            "work_type": "film",
            "director": "Giuseppe Vari",
        },
        same_column_values=["Dune", "Solaris", "Arrival"],
        table_name="works",
    ),
    TableCellTask(
        cell="Dune",
        column_header="title",
        row_context={
            "work_type": "novel",
            "author": "Frank Herbert",
        },
        same_column_values=["Solaris", "Neuromancer", "Foundation"],
        table_name="works",
    ),
]

predictions = classifier.predict_table_cells(table_tasks)
```

Every task can provide its own header, row context, column samples, table name,
description, `max_row_fields`, and `max_column_samples`. Empty row values,
duplicate samples, and the target itself are removed. Each returned prediction
retains its own `table_preview` and `context_report`, including omitted-context
counts. `TableCellTask` is also accepted directly by `predict_many()` when text,
record, and table tasks need to share one input iterable.

Inspect the exact zero-shot batch request without spending an API call:

```python
preview = classifier.preview_batch_prompts(
    table_tasks[:MAX_MENTIONS_PER_BATCH],
    assumed_coarse_type="PRODUCT",
)

print(preview["coarse_request_characters"])
print(preview["fine_request_characters"])
assert preview["hard_max_mentions_per_batch"] == 8
```

## What changed in 0.7.0

- Replaced hierarchy-heavy LLM prompts with a general zero-shot semantic rule;
  controlled labels remain constrained by response schemas and validated locally.
- Added source-first hierarchical prediction for free text, table columns, and
  table cells before Wikidata candidate retrieval.
- Added canonical retrieval paths, confidence-aware retrieval plans, hierarchy
  compatibility scoring, and candidate ranking safeguards.
- Added shared multi-mention OpenRouter requests, bounded table-cell contexts,
  prompt/schema preflight reporting, and schema-safe batching of up to 8 targets.
- Added deterministic NER tags and locally constructed retrieval keys without
  allowing the model to generate QIDs or retrieval metadata.

## What changed in 0.6.0

- Added 23 controlled occupation types for real people, including
  `POLITICIAN`, `ACTOR`, `MUSICIAN`, `WRITER`, `ATHLETE`, and `SCIENTIST`.
- Human occupations are multi-valued: one person can expose several compatible
  `specific_types` without forcing an arbitrary single occupation.
- Enabled the same human-specific taxonomy for deterministic Wikidata input and
  locally validated OpenRouter mention inference.
- Generic humans still resolve to `HUMAN` when no occupation is supported.

## What changed in 0.5.1

- Added order-preserving native `predict_batch()` for mappings and generators.
- Added an instance-local bounded LRU that reuses coarse/fine branch decisions,
  including abstentions, across batch calls.
- Kept description and context refinement independent for every item.
- Indexed subtype, facet, and composite retrieval rules by selected fine branch.
- Added cache statistics and explicit cache clearing.

See [CHANGELOG.md](CHANGELOG.md) for release history.

## What changed in 0.5.0

- Added mention-focused type inference for free text and structured/tabular data.
- Added a dependency-free OpenRouter client with strict JSON-schema output.
- Added complete coarse-to-fine LLM inference over the packaged hierarchy, with
  branch-local subtype and facet selection in the second stage.
- Added OpenRouter provider pinning for fast Cerebras inference with fallbacks
  disabled when deterministic latency is required.
- Added exact mention-span marking, target-focus validation, and safe abstention.
- Added an auditable, bounded context report and table preview for cell inference.
- Retained the deterministic Wikidata token/clue classifier unchanged.

The QID is retained as an identifier and is never used as a lookup key.

## Two classification paths

### Wikidata items: deterministic token clues

Use this path when P31/P279 labels are already available:

```python
from wikidata_ner import WikidataNERClassifier

classifier = WikidataNERClassifier()
prediction = classifier.predict(
    qid="Q3441181",
    types=[{"id": "Q11424", "name": "film"}],
    description="1964 sword-and-sandal film directed by Giuseppe Vari",
)
```

The primary branch is selected with the library's deterministic token/clue
rules. Descriptions can refine that branch but cannot replace its P31/P279
anchor.

### Input mentions: LLM inference through OpenRouter

Use this path when the input is a mention and its type must be inferred from
context:

```bash
export OPENROUTER_API_KEY='...'
```

```python
from wikidata_ner import OpenRouterNERClassifier

classifier = OpenRouterNERClassifier(
    model="openai/gpt-oss-120b",
    provider="cerebras",
    allow_fallbacks=False,
    reasoning_effort="low",
)

prediction = classifier.predict_text(
    "Rome Against Rome is a 1964 sword-and-sandal film directed by "
    "Giuseppe Vari; the story is set partly in Rome.",
    mention="Rome Against Rome",
)

print(prediction.fine_type)       # FILM
print(prediction.specific_type)   # SWORD_AND_SANDAL_FILM
```

Only `Rome Against Rome` is classified. The later `Rome` is contextual evidence
about a different mention and cannot become the prediction target.

For a structured record:

```python
prediction = classifier.predict_record(
    {
        "label": "Rome Against Rome",
        "types": [{"name": "film"}],
        "description": "1964 sword-and-sandal film",
    }
)
```

For a table cell:

```python
prediction = classifier.predict_table_cell(
    "acetylsalicylic acid",
    column_header="active ingredient",
    row_context={
        "drug": "Aspirin",
        "molecular_formula": "C9H8O4",
    },
    same_column_values=["ibuprofen", "paracetamol", "naproxen"],
)

print(prediction.table_preview)
print(prediction.context_report["context_usage"])
```

The cell is always the target. Headers, row attributes, and same-column samples
are evidence about the cell, never alternative targets. The returned
`context_report` renders the exact selected table information, explains how each
context component was interpreted, and reports whether fields or samples were
omitted. The LLM receives that information as structured JSON; the Markdown
preview is human-readable and is not duplicated in the prompt.

By default, table context is bounded to 12 non-empty same-row fields and 8
distinct same-column samples. Empty values, duplicate samples, and the target
itself are removed from the sample set. Adjust the limits only when the table
requires it:

```python
prediction = classifier.predict_table_cell(
    cell,
    column_header="title",
    row_context=relevant_row_fields,
    same_column_values=column_examples,
    max_row_fields=8,
    max_column_samples=5,
)
```

For best accuracy, pass fields that describe or relate directly to the target
cell—such as a type/category, description, unit, identifier, creator, location,
or parent relation. Avoid unrelated display metadata and entire unfiltered
rows.

For contextual free text, `mention=` is required. You can disambiguate repeated
surface forms with an exact character span:

```python
prediction = classifier.predict_text(
    text,
    mention="Rome",
    mention_span=(start, end),
)
```

Use `preview_prompts(...)` to inspect the normalized mention, exact prompts, and
JSON schemas without making an API call:

```python
preview = classifier.preview_prompts(
    text,
    mention="Rome Against Rome",
    assumed_coarse_type="CREATIVE_WORK",
    assumed_fine_type="FILM",
)
```

The default predictor uses two schema-constrained zero-shot calls:

1. Select exactly one controlled coarse branch or abstain.
2. Select one fine type and only its legal subtypes and facets inside that
   branch.

Passing a model slug does not select a hosting provider on OpenRouter. Use
`provider="cerebras"` with `allow_fallbacks=False` when Cerebras latency is
required. Each stage reports the routed provider and wall-clock duration under
`prediction.usage`.

OpenRouter is called at
`https://openrouter.ai/api/v1/chat/completions` with strict JSON-schema output,
`provider.require_parameters=true`, temperature zero, and optional response
healing. The package continues to have no runtime dependencies. You may inject a
custom `client=` for testing or infrastructure integration.

`MentionPrediction` supports the same retrieval conveniences as deterministic
predictions:

```python
payload = prediction.to_dict()
fields = prediction.to_retrieval_fields(prefix="ner")
query_filter = prediction.elasticsearch_filter()
```

The selected OpenRouter model must support structured outputs. Pin a model slug
in production and store the returned model, prompt version, taxonomy version,
usage, evidence, and confidence with each result.

## Deterministic evidence policy

The default evidence policy is now:

1. `types[].name` selects `coarse_type` and `fine_type`.
2. `ancestor_types[].name`, when supplied, provides lower-weight class ancestry.
3. Direct type labels and `description` refine only the selected branch.
4. `context_string` is ignored by the classifier by default because it often
   contains related people, organizations, countries, genres, and formats.
5. Description evidence cannot change a `FILM` branch into a location, company,
   person, or another unrelated branch.
6. Unsupported specificity is not invented.

The packaged configuration contains:

- 187 fine-type rules;
- 295 structural subtype rules;
- 56 controlled facet rules;
- 7 branch-local composite-type templates.

## Installation

```bash
python -m pip install wikidata-ner-classifier
```

Python 3.10 or newer is required. The library has no runtime dependencies.

## Deterministic basic use

```python
from wikidata_ner import WikidataNERClassifier

classifier = WikidataNERClassifier()

prediction = classifier.predict(
    qid="Q3441181",
    types=[{"id": "Q11424", "name": "film"}],
    description="1964 sword-and-sandal film directed by Giuseppe Vari",
)

print(prediction.to_dict())
```

Relevant output:

```json
{
  "coarse_type": "CREATIVE_WORK",
  "fine_type": "FILM",
  "subtype": null,
  "specific_type": "SWORD_AND_SANDAL_FILM",
  "specific_types": [
    "SWORD_AND_SANDAL_FILM"
  ],
  "facets": {
    "genre": [
      "SWORD_AND_SANDAL"
    ]
  },
  "refinement_sources": [
    "description"
  ]
}
```

The description adds specificity only inside the already established `FILM`
branch.

## Native batch prediction

Use `predict_batch()` when classifying many items. It accepts any iterable of
item mappings, returns normal `Prediction` objects in input order, and retains
each QID:

```python
items = [
    {
        "qid": "Q3441181",
        "types": [{"id": "Q11424", "name": "film"}],
        "ancestor_types": [],
        "label": "Rome Against Rome",
        "description": "1964 sword-and-sandal film",
        "context_string": None,
    },
]

predictions = classifier.predict_batch(items, cache_size=100_000)
```

Batch prediction normalizes the direct and ancestor type labels, groups
identical ordered signatures, and performs the full coarse/fine rule scan once
per missing signature. The bounded cache is a true least-recently-used cache and
is local to the classifier instance, so custom rules and configuration never
share entries with another classifier. Successful decisions and abstentions are
both cached.

Only the primary coarse/fine decision is reused. Subtypes, facets, composite
specific types, evidence, and refinement sources are calculated independently
for every item, so descriptions and context strings remain item-specific.
Results are exactly equivalent to calling `predict()` on every item.

By default, the cache key is the ordered normalized direct and ancestor label
signature. If `use_description=True`, the normalized description is also part
of the key. If `use_entity_label=True`, the normalized entity label is also part
of the key. Description/context settings used only for branch-local refinement
do not widen the primary key.

Inspect or reset the cache with:

```python
info = classifier.branch_cache_info()
print(info.hits, info.misses, info.maxsize, info.currsize)

classifier.clear_branch_cache()
```

Passing `cache_size=0` disables reuse across calls while still deduplicating
repeated signatures inside the current batch. Changing `cache_size` on a later
call immediately evicts the least recently used entries until the cache fits.
The implementation is synchronous, dependency-free, and deterministic; callers
can place independent classifier instances in an external process pool.

## Alpaca or Elasticsearch entities

Both source objects and complete Elasticsearch hits are accepted:

```python
prediction = classifier.predict_entity(hit_or_source)
```

```python
{
  "qid": "Q3441181",
  "types": [{"name": "film"}],
  "description": "1964 sword-and-sandal film directed by Giuseppe Vari"
}
```

```python
{
  "_id": "Q3441181",
  "_source": {
    "qid": "Q3441181",
    "types": [{"name": "film"}],
    "description": "1964 sword-and-sandal film directed by Giuseppe Vari"
  }
}
```

A complete Elasticsearch response can be processed with:

```python
predictions = classifier.predict_elasticsearch_response(response)
```

## Why both subtype and specific type exist

A subtype describes a structural kind. A facet describes an independent
characteristic. A specific type is a retrieval-oriented composition.

```python
prediction = classifier.predict(
    "Q1",
    [
        {"name": "film"},
        {"name": "feature film"},
        {"name": "comedy film"},
    ],
)
```

This can produce:

```json
{
  "fine_type": "FILM",
  "subtype": "FEATURE_FILM",
  "specific_type": "FEATURE_FILM",
  "specific_types": [
    "FEATURE_FILM",
    "COMEDY_FILM"
  ],
  "facets": {
    "genre": [
      "COMEDY"
    ]
  }
}
```

`FEATURE_FILM` and `COMEDY_FILM` are compatible. They may be combined by the
retriever rather than forced into a single mutually exclusive label.

Human occupations use the same compatibility model:

```python
person = classifier.predict(
    "Q7259",
    [{"name": "human"}],
    description="British politician and writer",
)

assert person.fine_type == "HUMAN"
assert person.specific_type == "POLITICIAN"
assert person.specific_types == ("POLITICIAN", "WRITER")
assert person.facets["occupation"] == ("POLITICIAN", "WRITER")
```

The controlled human occupation types are `ACADEMIC`, `ACTIVIST`, `ACTOR`,
`ARCHITECT`, `ARTIST`, `ATHLETE`, `BUSINESSPERSON`, `EDUCATOR`, `ENGINEER`,
`EXPLORER`, `FILMMAKER`, `INVENTOR`, `JOURNALIST`, `LEGAL_PROFESSIONAL`,
`MEDICAL_PROFESSIONAL`, `MILITARY_PERSONNEL`, `MUSICIAN`, `POLITICIAN`,
`PUBLIC_OFFICIAL`, `RELIGIOUS_FIGURE`, `ROYALTY`, `SCIENTIST`, and `WRITER`.

## Example refinements from the Alpaca query

| Type labels | Description | Fine type | Subtype | Most specific retrieval type |
|---|---|---|---|---|
| `film` | `1951 film directed by Luigi Zampa` | `FILM` | none | `FILM` |
| `film` | `1964 sword-and-sandal film ...` | `FILM` | none | `SWORD_AND_SANDAL_FILM` |
| `album` | `album by Holger Czukay` | `MUSICAL_WORK_SONG_OR_ALBUM` | `MUSIC_ALBUM` | `MUSIC_ALBUM` |
| `literary work` | `Alternative history, military science fiction story` | `BOOK_OR_WRITTEN_WORK` | `FICTION_STORY` | `MILITARY_SCIENCE_FICTION_LITERARY_WORK` |
| `pencil drawing` | `1953 work of art ...` | `VISUAL_ARTWORK_PHOTOGRAPH_OR_COMIC` | `PENCIL_DRAWING` | `PENCIL_DRAWING` |
| `human` | `British politician and writer` | `HUMAN` | none | `POLITICIAN`, `WRITER` |

A generic description cannot justify an invented subtype. A generic film remains
`FILM` when neither its type labels nor description contain a safe refinement.

## Retrieval indexing helpers

Store the prediction alongside each entity using stable keyword fields:

```python
fields = prediction.to_retrieval_fields(prefix="ner")
```

Example fields:

```json
{
  "ner_coarse_type": "CREATIVE_WORK",
  "ner_fine_type": "FILM",
  "ner_subtype": null,
  "ner_specific_type": "SWORD_AND_SANDAL_FILM",
  "ner_specific_types": [
    "SWORD_AND_SANDAL_FILM"
  ],
  "ner_facets": {
    "genre": [
      "SWORD_AND_SANDAL"
    ]
  }
}
```

A deterministic Elasticsearch filter can be generated with:

```python
query_filter = prediction.elasticsearch_filter(
    field="ner_specific_types",
    require_all=True,
)
```

For several compatible specific types, `require_all=True` emits one term filter
per type. Use `require_all=False` to emit a `terms` disjunction.

Index-time and query-time predictions should use the same library and rule-file
version.

## Description and context controls

Description refinement is enabled by default:

```python
classifier = WikidataNERClassifier(
    use_description_for_refinement=True,
    use_context_string_for_refinement=False,
)
```

Disable it when only class labels should be considered:

```python
classifier = WikidataNERClassifier(
    use_description_for_refinement=False,
)
```

Noisy context refinement is available only as an explicit opt-in:

```python
classifier = WikidataNERClassifier(
    use_context_string_for_refinement=True,
)
```

The separate `use_description=True` option allows description text to add
low-weight support to the primary coarse/fine scorer. It is disabled by default.
Descriptions therefore do not rescue a missing or unknown type anchor unless the
caller explicitly changes that policy.

## Live Alpaca notebook

Open `examples/alpaca_live_test.ipynb`.

The notebook:

1. issues the supplied Alpaca Elasticsearch request;
2. extracts QID, type labels, and description;
3. ignores `context_string` during classification;
4. displays `predicted_subtype`, `predicted_specific_type`, all compatible
   `specific_types`, facets, and confidence values;
5. demonstrates a retrieval filter generated from the prediction.

Set the bearer token before starting Jupyter:

```bash
export ALPACA_TOKEN='your-token'
```

The notebook also supports a hidden token prompt when the environment variable is
not set.

## CLI

```bash
wikidata-ner response.json > predictions.json
cat response.json | wikidata-ner
```

Relevant flags:

```text
--no-description-refinement
--context-refinement
--description-for-primary
--entity-label
```

## Validation

The source package includes unit tests for:

- direct type-label classification;
- description-only branch refinement;
- context exclusion by default;
- explicit context opt-in;
- subtype and facet compatibility;
- generic fallback behavior;
- QID independence;
- Elasticsearch hit input;
- retrieval-field generation;
- generated Elasticsearch filters.


## Mention-focused OpenRouter notebook

`examples/openrouter_mention_focused_ner.ipynb` classifies one explicit target
mention at a time from free text, structured records, or table cells. Context is
used only as evidence for that target. Contextual free text requires `mention=`
or an exact span; table helpers make the selected cell the target. The installable
library now exposes the same workflow through `OpenRouterNERClassifier`; the
notebook remains useful as an expanded prompt inspection and evaluation example.
