Metadata-Version: 2.4
Name: census-search
Version: 0.7.0
Summary: CLI tool for tracing individuals across Irish National Archives census records (1926, 1911, 1901, 1851, 1841, 1831, 1821)
License-Expression: LicenseRef-Polyform-Small-Business-1.0.0
License-File: LICENSE
Keywords: ireland,census,genealogy,national-archives,family-history
Author: Joe Corrigan
Author-email: meltingturret@gmail.com
Requires-Python: >=3.12
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: End Users/Desktop
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Utilities
Requires-Dist: click (>=8.1,<8.4)
Requires-Dist: httpx (>=0.27)
Requires-Dist: playwright (>=1.44)
Requires-Dist: pydantic (>=2)
Requires-Dist: rich (>=13)
Requires-Dist: typer[all] (>=0.26)
Project-URL: Repository, https://github.com/meltingturret/census-search
Description-Content-Type: text/markdown

# census-search

CLI tool for tracing individuals across Irish National Archives census records (1926, 1911, 1901, and the pre-Famine fragments 1851, 1841, 1831, 1821).

Searches the **1926 census** by name, then links backwards to **1911** and **1901** using confidence scoring. Results are always displayed as tables. When a single 1926 record is found, the full household is shown automatically.

The **1821–1851 commands** browse the surviving census fragments held by the National Archives. These are partial records — not all counties or returns survived the 1922 Four Courts fire.

## License

Licensed under the [Polyform Small Business License 1.0.0](LICENSE).

- **Free to use** for individuals and organisations with under $1M/year gross revenue.
- **Commercial license required** for organisations with over $1M/year revenue — see [COMMERCIAL_LICENSE.md](COMMERCIAL_LICENSE.md).

## Install

Requires Python 3.12+ and [Poetry](https://python-poetry.org/).

```bash
bash setup.sh
```

Or manually:

```bash
poetry env use python3.12
poetry install
poetry run playwright install chromium
```

## Commands

### `link` — search and link across all three censuses

The primary command. Without `--birth-year`, returns all 1926 matches as a browse table. With `--birth-year`, age-filters results and links each match across 1911 and 1901 with a confidence score. Household members are always shown and automatically linked back to 1911 & 1901.

```bash
# Browse all male Corrigans in Kilkenny (no birth year)
poetry run census-search link Corrigan --county Kilkenny --sex Male

# Link across all three censuses using a known birth year
poetry run census-search link Corrigan --first-name James --birth-year 1882 --county Kilkenny --sex Male

# Search across multiple counties
poetry run census-search link Corrigan --first-name James --birth-year 1882 --county "Kilkenny,Tipperary"

# Search for name variants (Joe, Joseph, Jos) in one pass
poetry run census-search link Corrigan --first-name "Joe,Joseph,Jos" --birth-year 1917 --county Kilkenny --sex Male
```

| Flag | Short | Default | Description |
|------|-------|---------|-------------|
| `--birth-year` | `-b` | | Known or estimated birth year — omit to browse all ages |
| `--first-name` | `-f` | | First name — comma-separate variants (e.g. `Joe,Joseph,Jos`) |
| `--county` | `-c` | | County or comma-separated counties (e.g. `Kilkenny` or `Kilkenny,Tipperary`) |
| `--sex` | `-s` | | `Male` or `Female` — enforced client-side |
| `--service-number` | | | Army service number — triggers TNA military records search |
| `--age-tolerance` | | 3 | ±years for age matching across censuses |
| `--max` | `-n` | 30 | Max results per census year |
| `--no-headless` | | | Show browser window (useful for debugging) |

#### Output examples

**Browse without birth year** — returns all matches as a table:

```
$ poetry run census-search link Corrigan --county Kilkenny --sex Male

Corrigan  8 result(s)  add --birth-year to link across 1911 & 1901
 #   Surname    First Name   Age  Sex   County    Townland / Street  DED              Birthplace
 1   Corrigan   James         44  Male  Kilkenny  Lamogue            Kilmaganny
 2   Corrigan   Patrick       52  Male  Kilkenny  Main Street        Kilkenny Urban
 3   Corrigan   Thomas        31  Male  Kilkenny  Ballyline          Scotsborough
 ...
```

**Link with birth year** — one row per census year with a confidence score:

```
$ poetry run census-search link Corrigan --first-name James --birth-year 1882 --county Kilkenny --sex Male

James Corrigan  (born ~1882 ±3yr)
 Year  Surname    First Name   Age  Sex   County    Townland / Street  DED          Birthplace  Match
 1901  Corrigan   James         19  Male  Kilkenny  Lamogue            Kilmaganny                 91%
 1911  Corrigan   James         29  Male  Kilkenny  Lamogue            Kilmaganny                 95%
 1926  Corrigan   James         44  Male  Kilkenny  Lamogue            Kilmaganny                  —

Household  Lamogue, Kilmaganny, Kilkenny
 #   Surname    First Name   Age  Sex     Relationship  County    Townland / Street  DED
 1   Corrigan   Mary          39  Female  Wife          Kilkenny  Lamogue            Kilmaganny
 2   Corrigan   Brigid        14  Female  Daughter      Kilkenny  Lamogue            Kilmaganny
 3   Corrigan   Patrick        9  Male    Son           Kilkenny  Lamogue            Kilmaganny
```

#### Military records

Supply `--service-number` to search the TNA Discovery API for WO 372 medal cards, WO 97 service records, PIN 82 widow's pensions, and PIN 26 disability pension files.

```
$ poetry run census-search link Hennessy --first-name Patrick --birth-year 1888 --sex Male --service-number 3989

Patrick Hennessy  (born ~1888 ±3yr)
 Year  Surname    First Name   Age  Sex   County  Townland / Street  DED       Birthplace  Match
 1901  Hennessy   Patrick       13  M     Clare   Henry Street       Kilrush Urban  Co Clare  92%
 1911  Hennessy   Patrick       23  M     Clare   Hector Street      Kilrush Urban  Co Clare  95%
 1926  Hennessy   Patrick       38  Male  Clare   Aglish             Glenroe         —

Military Records  TNA WO 372 / WO 97
 #  Type        Reference        Regiment              Service No  Rank     Dates
 1  Medal card  WO 372/9/144384  Royal Irish Regiment  3989        Private  1914-1920

Dependants & Pensions  TNA PIN 82 / PIN 26
 #  Type          Reference    Regiment / Unit       Cause of Death / Disability  Dates
 1  Pension file  PIN 26/18129  Royal Irish Regiment  —                            1915-1947

Household  Aglish, Glenroe, Clare
 #  Surname    First Name  Age  Sex     Relationship  County  Townland  DED
 1  Hennessy   Cecilia      25  Female  Wife          Clare   Aglish    Glenroe
 2  Hennessy   Mary         79  Female  Mother        Clare   Aglish    Glenroe
 3  Hennessy   Katie        38  Female  Sister        Clare   Aglish    Glenroe
```

```
$ poetry run census-search link Corrigan --first-name James --county Dublin --birth-year 1885 --sex Male --service-number 70925

Military Records  TNA WO 372 / WO 97
 #  Type        Reference       Regiment                      Service No  Rank    Dates
 1  Medal card  WO 372/5/31903  Royal Garrison Artillery Depot  70925      Gunner  1914-1920

Dependants & Pensions  TNA PIN 82 / PIN 26
 #  Type             Reference    Regiment / Unit           Cause of Death / Disability  Dates
 1  Widow's pension  PIN 82/38/2  Royal Garrison Artillery  Phthisis pulmonalis          [1915-1925]
```

**Multiple first name variants** — searches all variants and shows whichever name appears in the record:

```
$ poetry run census-search link Corrigan --first-name "Joe,Joseph,Jos" \
    --birth-year 1917 --county "Kilkenny,Tipperary" --sex Male

Joe / Joseph / Jos Corrigan  (born ~1917 ±3yr)
 Year  Surname    First Name   Age  Sex   County    Townland / Street  DED          Birthplace  Match
 1901  Corrigan   Joseph         0  Male  Kilkenny  Lamogue            Kilmaganny                 78%
 1911  Corrigan   Joe            6  Male  Kilkenny  Lamogue            Kilmaganny                 82%
 1926  Corrigan   Joseph         9  Male  Kilkenny  Lamogue            Kilmaganny                  —
```

**Multi-county search** — merges results from both counties:

```
$ poetry run census-search link Purcell --first-name Mary --birth-year 1887 --county "Kilkenny,Tipperary" --sex Female

Mary Purcell  (born ~1887 ±3yr)
 Year  Surname  First Name  Age  Sex     County    Townland / Street  DED                 Birthplace  Match
 1901  Purcell  Mary         14  Female  Kilkenny  Ballyline          Scotsborough                      92%
 1911  Purcell  Mary         22  Female  Kilkenny  Brownstown         Kilkenny Rural                    88%
 1926  Purcell  Mary         38  Female  Kilkenny  Balief Upper       Clomantagh                         —
```

When a household is found it is shown below as a separate table, and each member is automatically linked back to 1911 & 1901.

---

### `browse` — browse by county/DED (no name required)

```bash
# Browse all records in a county
poetry run census-search browse --county Dublin

# Browse a specific DED
poetry run census-search browse --county Kerry --ded "Tralee Urban"

# Filter by surname within a county
poetry run census-search browse Corrigan --county Kilkenny

# Increase result limit
poetry run census-search browse Corrigan --county Kilkenny --max 100
```

| Argument / Flag | Short | Default | Description |
|-----------------|-------|---------|-------------|
| `surname` | | | Surname to filter by (optional positional argument) |
| `--county` | `-c` | | County to browse |
| `--ded` | `-d` | | District Electoral Division |
| `--max` | `-n` | 30 | Max results to return |

---

### `1901` / `1911` — browse those censuses directly (no 1926 anchor needed)

```bash
# Browse all Corrigans in Tipperary in 1901
poetry run census-search 1901 Corrigan --county Tipperary --max 300

# Browse all Corrigans in Tipperary in 1911
poetry run census-search 1911 Corrigan --county Tipperary --max 300

# Filter by first name and sex
poetry run census-search 1911 Corrigan --first-name James --county Kilkenny --sex Male

# Multiple name variants, multiple counties
poetry run census-search 1901 Murphy --first-name "Mary,Maria" --county "Kilkenny,Tipperary"
```

| Argument / Flag | Short | Default | Description |
|-----------------|-------|---------|-------------|
| `surname` | | | Surname to search (optional positional argument) |
| `--first-name` | `-f` | | First name — comma-separate variants (e.g. `Joe,Joseph`) |
| `--county` | `-c` | | County or comma-separated counties |
| `--sex` | `-s` | | `Male` or `Female` — enforced client-side |
| `--max` | `-n` | 30 | Max results to return |

#### Output example

```
$ poetry run census-search 1911 Corrigan --county Tipperary --max 300

📂 Browsing 1911 census — Corrigan

142 record(s) — showing 142
 #   Surname    First Name   Age  Sex   County     Townland / Street  DED          Birthplace
 1   Corrigan   James         29  Male  Tipperary  Main Street        Clonmel
 2   Corrigan   Mary          24  Female Tipperary Barrack Street     Clonmel
 ...
```

---

### `1851` / `1841` / `1831` / `1821` — pre-Famine census fragments

Browse the surviving pre-Famine census returns held by the National Archives. These are partial records — coverage varies by county and year due to destruction in the 1922 Four Courts fire.

```bash
# 1851 — full personal detail (age, sex, relation, occupation, birthplace)
poetry run census-search 1851 Murphy --county Antrim

# 1841 — same columns as 1851
poetry run census-search 1841 Corrigan --county Kilkenny --sex Female

# 1831 — household-level stats (no individual age/sex)
poetry run census-search 1831 Corrigan

# 1821 — includes age, occupation, relation to head
poetry run census-search 1821 Murphy --county Meath
```

| Argument / Flag | Short | Default | Description |
|-----------------|-------|---------|-------------|
| `surname` | | | Surname to search (optional positional argument) |
| `--first-name` | `-f` | | First name |
| `--county` | `-c` | | County |
| `--sex` | `-s` | | `Male` or `Female` — 1841/1851 only (those years have individual sex records) |
| `--max` | `-n` | 30 | Max results to return |

**Available fields by year:**

| Year | Age | Sex | Relation | Occupation | Birthplace | Notes |
|------|-----|-----|----------|------------|------------|-------|
| 1851 | ✓ | ✓ | ✓ | ✓ | ✓ | Most complete fragment |
| 1841 | ✓ | ✓ | ✓ | ✓ | ✓ | |
| 1831 | — | — | — | — | — | Household aggregate stats only |
| 1821 | ✓ | — | ✓ | ✓ | — | Individual records, no sex column |

---

## How it works

1. **Search 1926**: Playwright opens a Chromium browser, intercepts the underlying API call, and extracts structured results. Filtering by first name, sex, and age is done client-side.
2. **Household**: When a single 1926 record is matched, all household members on the same PDF form (`aform_name`) are fetched automatically. Household members are always linked back to 1911 & 1901 automatically.
3. **Search 1911 & 1901**: Uses a direct REST API — no browser required. Results are filtered by surname, age window (`expected_age ± tolerance`), and county. Multiple counties can be searched and merged.
4. **Phonetic matching**: Surname variants are caught using Soundex (e.g. Corrigan/Corigan, Purcell/Pursell) and fuzzy string similarity.
5. **Confidence scoring**: Each candidate is scored across surname, first name, age consistency, county, sex, and relationship. The best match above the threshold is shown with a percentage.
6. **Relationship-aware linking**: Relationship compatibility across years is factored into scoring (e.g. Son↔Scholar, Daughter↔Scholar score higher than unrelated pairs).

## Notes

- The 1926 census site is JavaScript-rendered. Playwright/Chromium is required. On first run, Playwright downloads Chromium (~150 MB).
- The 1901/1911 search uses a direct REST API — no browser needed for that step.
- The 1821–1851 commands scrape the National Archives search page (plain HTML, no browser required). Coverage is sparse — many returns did not survive.
- Age tolerance of ±3 years is the default — census ages were often approximate.
- Census ages are not always accurate; the confidence score reflects how well each linked record fits across all available fields.

