Decisions

decisions/ holds what people decided. Its files are small enough to read, compare and edit, and they name things by stable keys: keywords by their text, people and organisations by their ids. A re-extraction or a re-collection never moves a decision onto something else; a decision whose key disappears is listed, never dropped silently.

decisions/
  people.csv            roles, identities, merges
  people_pairs.csv      pairs of people judged two people, or left for later
  organisation_pairs.csv  the same, for organisations
  organisations.csv     levels, parents, merges, names, set by people
  affiliations.csv      affiliations added or removed by people
  params.json           the parameters people set
  keywords.csv          keep, exclude, merge, and each keyword's category
  themes.json           the theme tree, keyed by keyword text
  maps.json             map versions, the pinned one
  snowball.csv          collaborators proposed, round by round, and what was decided
  stopwords.json        additions to and removals from the function-word lists
  curation-notes.md     the curator's notes and standing rules for the AI copilot
  prompts/              prompt overrides, one file per prompt
  history/              every earlier version

Writing decisions

A writer passes the fingerprint (SHA-256) of the version it read. If the file changed since, the write is refused and nothing is lost. An accepted write moves the previous version to decisions/history/<file>/<UTC time>-<action>.<ext> first, then writes atomically. project.json follows the same rule. Restoring a version is a write like any other, so it is itself undoable.

A CSV decision file of 256 KB or more (the people.csv of a large project) keeps most earlier versions as a delta instead, <UTC time>-<action>.csv.delta: gzipped JSON (format: "cartolex-history-delta/1"), saying how the latest earlier version kept whole (of, its name) turns into this one. ops lists, in order, runs [start, count] of that version’s records (a record is a line, or several when a quoted field holds a line break) and the text of the records it lacks; base and result are the SHA-256 of the version it starts from and of the version it gives. One version in 50 is kept whole, and so is one of which more than half is new. A delta depends only on files of the history, which never change: editing the current file by hand leaves every earlier version readable.

people.csv

One row per person the project knows. Columns:

column	meaning
person_id	the key, as in sources/tables/people.parquet
role	mapped (their texts shape the lexicon and the map, and they appear on it), context (their texts shape the lexicon and the space with a weight; they are not on the map), projected (placed on the finished map), excluded, undecided
set	for projected: the overlay set
identity	confirmed, auto (a single match above the threshold, accepted automatically), none (no record exists), pending
records	the service records that are this person, ;-separated (openalex:A…;orcid:0000-…)
merged_into	when two rows are one person, the person_id that remains (see below)
note	free text
decided_at	UTC time of the last change to the row

Merges. A merge is a decision, and it can be undone: the merged row keeps its own role, identity and records, so emptying merged_into gives it back as it was. While it is merged, the person it is merged into stands for both: their records are their own and those of every row merged into them (each counted when its row’s identity is confirmed or auto), and so are their texts (a text both rows wrote counts once), their affiliations, their coverage and their sheet; the collection asks the services for all those records, and corpus.assemble reads the merged rows’ texts as theirs, never a merged row on its own. merged_into names the person that remains; a chain (a row merged into a row that is itself merged) is followed to its end, and a loop leaves its rows on their own. Two rows whose ORCIDs differ are not merged unless someone says they are one person anyway. corpus.assemble records the merges as an input of their own (decisions/people.csv#merges, in its run.json), present only when there are merges.

people_pairs.csv and organisation_pairs.csv

Pairs proposed as one person (one organisation) and judged otherwise, so that the proposal does not come back. Columns: a, b (the two ids, the smaller first), decision (distinct: two people, never proposed again; later: not decided, kept in the list of pairs to review and out of every automatic merge), note, decided_at. A merge is not a pair decision: it is merged_into. Undoing a merge may record its pair here.

Keywords are decided by the mapped people: a keyword is kept only if enough mapped people use it. context texts count in the statistics, never in that threshold.

organisations.csv and affiliations.csv

organisations.csv changes what the sources say about organisations: org_id, level, parents (;-separated), name, merged_into, note, decided_at. An empty cell leaves the source’s value. affiliations.csv adds or removes affiliations: person_id, org_id, start_year, end_year, action (add or remove), note, decided_at; a removal without start_year removes every affiliation of the person to the organisation, with one only those starting that year.

Every reader applies them (cartolex.project.organisations): the corpus (the unit of each person), the organisations’ list and detail, the people’s units, a person’s sheet, the coverage by organisation and the atlas. A merged organisation is read as the one it is merged into: its affiliations are that organisation’s, it is not listed on its own, and a parent merged elsewhere is replaced by the one that remains. A merge can be undone: emptying merged_into gives the organisation back as it was.

params.json

Only the parameters people set. Every other value comes from a default, most of them computed from the project’s sizes by a simple rule; the effective value and where it came from are written into each stage’s run.json.

{
  "format": "cartolex-params/1",
  "seed": 20260928,
  "pinned_year": null,
  "stages": {
    "keywords.extract": {"counting_unit": "person"},
    "themes.group": {"top_groups": 15}
  }
}

seed fixes every random choice; pinned_year fixes the year that date windows count back from (none: the current year). A parameter a stage does not know, or an impossible value (as many topics as keywords), is refused when the file is read, with the reason.

collect (optional, added within version 1 and left out of the file when empty) holds the collection’s parameters people set, by step, each a whole number:

"collect": {"snowball": {"cap": 200, "max_authors": 25}, "coverage": {"good": 3}}

parameter	default	meaning
snowball.cap	200	collaborators’ rounds are taken whole while the people proposed stay within this number; the round that would pass it is left out, and named
snowball.max_authors	25	a work with more authors is left out of the co-author graph
coverage.good	3	a person is well covered from this many texts with an abstract (or a full text)

An unknown step or parameter, or a value below its smallest (1, and 2 for max_authors), is refused when the file is read.

ai (optional, added within version 1 and left out of the file when empty) holds the route people chose for each AI step of a build, from the build page:

"ai": {"keywords.triage": "copilot", "themes.curation": "none"}

step	routes	meaning
keywords.triage	none, copilot, api	the keyword clean-up, between the extraction and the vocabulary
themes.curation	none, copilot	the theme curation, between the grouping and the themes applied; it has no API route

none (the default) builds without AI help. copilot: the build runs up to the step and pauses there until an assistant’s result is imported and accepted, or the person continues without it. api: the clean-up runs by the provider’s API, the opt-in stage keywords.triage, whose enabled switch (stages) is the one the build reads: choosing api sets it, choosing another route clears it, and enabled: true reads as api whatever ai says. An unknown step or route is refused when the file is read.

similarity (optional, added within version 1 and left out of the file at its default space) is how two people, or two organisations, are compared: the headline of Compare on the map, the nearest of each and the exported distances. It needs no rebuild.

value	meaning
space	the cosine of their vectors in the space of the themes (the default)
keywords	the cosine of their keyword profiles (each keyword’s share of their use)
jaccard	the share of their keywords both use, among those either uses
themes	Σ min of their top-level theme shares

keywords.csv

column	meaning
term	the keyword, as shown
language	its language
decision	keep, exclude or merge
target	for merge: the keyword it merges into
reason	why, in words
source	person, ai-handoff, ai-copilot or ai-api
decided_at	UTC time
category	optional (added within version 1, written only when a row fills it): what a kept or merged keyword names, concept, method (techniques, instruments, models, data sources), object, place (a kind of place or setting) or field (a discipline’s name); why an excluded one is not a keyword, never (never informative in any field) or here (not informative in this field only)

A proposal imported from an AI (a copilot’s result, or, from an earlier version, an answer to a handoff) is kept as it came in history/ai/, and only the changes someone accepts reach keywords.csv.

The curator’s notes for the AI copilot: curation-notes.md

A Markdown file the curator writes (Settings › Project, or either copilot dialog): teams, keywords that belong together or apart, standing context. Under the heading ## Standing rules for the AI copilot, one line per rule agreed with a copilot, with the task it is for:

Two teams work on the same coast: keep their waves and tides together.

## Standing rules for the AI copilot

- triage: research discourse is always excluded, sure
- themes: never merge the two policy themes

Every copilot bundle carries the notes and its task’s rules, as context for the assistant (never instructions); importing a result adds the new rules it brought. It is written like every decision file (the version read, the history); a line under the heading that is not a rule is kept in the notes.

Rejected automatically: cartolex-rejects/1

Some candidates are never keywords, in any field. Two lists name them, and the extraction puts the candidates they name in the rejected band (never judged by an AI, never in the lexicon):

• cartolex’s list, shipped with the package in cartolex/_data/rejects/<lang>.json: {"format": "cartolex-rejects/1", "language": "en", "terms": ["…"]};

• this computer’s cache, in the app’s own folder (<data dir>/rejects/<lang>.jsonl, never in a project): a first line {"format": "cartolex-rejects/1", "language": "en"}, then one JSON object per term and project: term, language, route (ai-api, ai-handoff, ai-copilot), date (UTC day), project (a fingerprint of the project, never its name). An AI’s never answers enter it: every one by API (its prompt gives K, G and F only when sure), and the accepted exclusions of a handoff or a copilot that the judge marked sure (an answer that does not say is unsure, and stays out).

A project’s own answers never reject its own candidates: only other projects’ do. A person’s decision on a term wins: the term is not rejected in that project, and a keep or a merge (a put back) takes it out of the cache. The rejects parameter of keywords.extract (params.json) switches both lists off for a project. A change of the cache makes no result out of date: it applies at the next extraction (force keywords.extract to apply it now). The extraction records what it applied in its results (derived/keywords.extract/rejects.json: per language, each term and its origin, list or earlier).

cartolex rejects export [--min-projects 2] [--out folder] [--projects …] makes a list to ship from the cache: the terms seen in at least that many projects, without any that looks like a proper name (a capital letter, a digit) or holds a word of a person’s or an organisation’s name in the user’s projects (the projects given, else those the app opened lately). The maintainers review every term before adding a list to the package; the vocabulary scan (tools/vocab_scan.py) covers it like every file of the repository.

themes.json: the theme tree

The tree has a depth from 1 to 4 levels of nodes. Its nodes have stable ids and names in each interface language; each keyword text sits on one node, at any level: a keyword on a higher node is broader than every node below it.

{
  "format": "cartolex-themes/1",
  "depth": 2,
  "levels": [
    {"names": {"en": "Theme", "fr": "Thème", "pt": "Tema"}},
    {"names": {"en": "Topic", "fr": "Sujet", "pt": "Tópico"}}
  ],
  "nodes": [
    {"id": "n1", "parent": null, "names": {"en": "Coastal hazards"}, "order": 1},
    {"id": "n7", "parent": "n1", "names": {"en": "Storm surge"}, "order": 1}
  ],
  "keywords": {"coastal flooding": "n7", "extreme events": "n1", "storm surge model": "n7",
               "tidal inlet": "n7", "wave climate": "n7"},
  "attribution": {"wave climate": 1},
  "set_aside": {"numerical results": {"from": "n7", "reason": "too general", "attribution": 0}},
  "review": {"tidal inlet": "to_check"},
  "based_on": {"run": "themes.group/20260928T101500Z-3f2a", "vocabulary": "sha256:…"},
  "saved": {"at": "2026-09-28T10:20:00Z", "action": "move 3 keywords to n7"}
}

key	meaning
depth, levels	the number of levels (1 to 4) and their names, from the top level down
nodes	every node: a stable id (letters, digits, - and _), its parent (null on the top level), its names per language (possibly none), its order among its siblings (ties by id)
keywords	each placed keyword and the node it is on, at any level
attribution	for a placed keyword that does not count at its node’s level: how many levels, from the top, its usage counts toward (see below)
set_aside	each keyword set aside: from, the node it was set aside from (null when it never had a place), reason, and the attribution it had, if any
review	to_check for a keyword a rebase added, reviewed once someone checked it, kept once someone kept it at its node from the borderline list (a review too, and the only one that leaves it out of that list); a keyword with nothing to check is absent
based_on	the run and the vocabulary the tree was built or last rebased on
saved	when this version was saved, and the action that saved it

• Every keyword of the current vocabulary is on exactly one node or set aside, never both; review names only keywords the tree holds.

• A node’s level is its distance from the top (1 for a top-level node); no node sits below the tree’s depth. A node may hold keywords and child nodes at the same time. In the example, « extreme events » sits on the theme itself: it belongs to coastal hazards, not to storm surge in particular.

• Attribution. A keyword’s usage counts toward its node and every node above it (a keyword on a top-level node counts toward that node only). attribution lowers that: n counts it toward levels 1 to n only, 0 shows the keyword without counting it anywhere, and a keyword absent from the map counts down to its node’s level. n is below the level of the keyword’s node (0 to level − 1), and only placed keywords appear. In the example, « wave climate » is shown under storm surge but counts toward coastal hazards only.

• The carry rule. A move, a merge or a split that gives a keyword another node drops an attribution other than 0; 0 always stays. A keyword that keeps its node keeps its attribution (moving a node moves its keywords with it). A set-aside keyword keeps its attribution in its entry; putting it back on the node it came from gives it back, anywhere else only 0 comes back.

• Depth changes keep keywords on their nodes; the nodes’ levels shift. Inserting a level gives each node just above it that has child nodes one new child, named like it, that takes them over (at the top: one new root over every top-level node; at the bottom: nothing, the level starts empty), and an attribution counting toward the new level’s position becomes n + 1. Removing a level dissolves its nodes into their parents, which receive their child nodes and keywords; the keywords of a removed top-level node are set aside with the reason its node was removed with its level. An attribution counting toward the removed level becomes n − 1 (a keyword that counted toward the removed top level only then counts nowhere); a keyword moved to the parent with n equal to the parent’s level loses its attribution.

• from may name a node that no longer exists (merged away, or removed by a rebase or a save); putting the keyword back then needs a target. cartolex never gives a new node such an id.

• cartolex writes the file in one form: nodes in tree order (depth first, siblings by order then id), keywords, attributions, set-aside keywords and review states sorted by text; a set-aside entry without an attribution has no attribution key.

• Saving removes empty nodes. Every save removes each node with no keyword in its subtree, itself included (set-aside keywords do not count), and names the removed nodes in its action (move 2 keywords to n7; remove empty nodes n3, n5). A saved file therefore never holds an empty node; a tree being edited may, until it is saved.

• based_on.vocabulary is the fingerprint of a vocabulary: sha256: and the SHA-256 of the JSON list of its distinct keywords, sorted, written without spaces (["a","b"]) in UTF-8.

• After a new extraction or a new grouping the tree is rebased onto the new vocabulary: kept keywords stay where they are (set-aside ones stay set aside); each new keyword goes to the node proposed for it, at any level, marked to_check (set aside, with the reason new keyword, no place proposed, when no node is proposed); vanished keywords are removed, set-aside ones too; a node whose subtree held keywords and holds none after the rebase is removed, with the nodes under it. Nothing else changes: no node is renamed, moved or renumbered, and surviving keywords keep their attribution. The rebase is saved as a new version, and its reconciliation list names every added or removed keyword and every removed node. That list is the difference between the version before the rebase and the version it wrote, so the history keeps it.

• Level names are the defaults for the depth (Theme; Theme › Topic; Field › Theme › Topic; Domain › Field › Theme › Topic, in English, French and Portuguese) until someone renames them. When the depth changes, a name still equal to its default follows the defaults of the new depth; a name someone gave stays.

• Every change is a new version: saved records when and by which action it was saved, and decisions/history/themes.json/<UTC time>-<action>.json holds the version that <action> replaced at that time. Restoring a version saves it again as a new version, with the action restore <version>.

The operations on a tree and their rules are described in The theme tree.

maps.json: map versions

{
  "format": "cartolex-maps/1",
  "pinned": "v3",
  "versions": [
    {"id": "v3", "shows": ["people"], "layout": {"method": "umap", "seed": 7}, "base": null,
     "created_at": "2026-09-28T11:00:00Z", "note": "after the September curation"},
    {"id": "v4", "shows": ["people"],
     "layout": {"method": "umap", "seed": 8, "dimensions": 3}, "base": null,
     "created_at": "2026-10-07T09:00:00Z", "note": "in space", "built": true}
  ]
}

shows lists what the map shows: people, organisations:<level>, texts, possibly several. base names a base from project.json when the version is placed on another project’s map. A rebuild keeps the pinned version’s layout; another layout is tried as a new version beside it.

layout.dimensions is 2 (a flat map, the default) or 3 (a map in space): only the umap method draws in space, and not with its flat recipes (layout: tsne_anchored, tsne, tree); a version that asks for it otherwise is refused when the file is read. built (default false) marks a version to be built with the pinned one: each build of the map draws every built version and places the people, keywords, organisations, texts, projected people and time windows on it (see Derived results, caches, outputs and logs). The pinned version is always built, whatever its mark, and stays the reference: distances, exports and the themes’ places come from it.

layout.method is one of:

method	the people’s map	params
umap	UMAP of the people in the space	n_neighbors, min_dist, metric, n_epochs, spread, set_op_mix_ratio, local_connectivity, repulsion_strength, negative_sample_rate, layout
tsne	t-SNE of the people (the optional openTSNE package: pip install 'cartolex[tsne]')	perplexity (30), metric
tree	the applied theme tree: themes as discs, people inside their heaviest theme	fill (0.62: the share of a disc its children cover), gap (0.04), lean (0.4), sharp (8)

In every method the keywords are placed on the people’s map by their nearest people (map.layout’s neighbours and link_radius in params.json), and seed makes the map the same on every run. Layouts compared compares them.

snowball.csv

One row per collaborator proposed: round, person_id, seeds (the seeds they wrote with, ;-separated), path (how they connect to a seed), joint_texts, last_joint_year, fit, decision (mapped, context, projected, no, later), decided_at. A round is taken whole or not at all, up to the cap set in params.json (collect.snowball.cap). seeds names the people of the round before (the seeds, for round 1) the collaborator wrote with, path the chain of person_ids from a seed to them, joined by >, and fit their topical fit to the seeds (0 to 1, see Collecting people and texts). A collaborator is proposed with the decision context (their texts shape the lexicon, they are not on the map); no makes them excluded in people.csv, later undecided.

stopwords.json and prompts/

stopwords.json adds to or removes from the packaged function-word lists, per language: {"format": "cartolex-stopwords/1", "add": {"fr": ["…"]}, "remove": {}}. prompts/<name>.txt replaces a packaged prompt; it must keep the placeholders the packaged prompt uses, or it is refused when read.