Metadata-Version: 2.3
Name: translator_tom
Version: 2.0.0
Summary: TRAPI Object Models: A performant python data model and centralized utilities for the Translator Reasoner API.
Author: Willow Callaghan
Author-email: Willow Callaghan <43009413+tokebe@users.noreply.github.com>
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: bmt>=1.4.8
Requires-Dist: linkml-runtime>=1.7.0
Requires-Dist: orjson>=3.11.3
Requires-Dist: ormsgpack>=1.11.0
Requires-Dist: pydantic>=2.12.0
Requires-Dist: pydantic-core>=2.41.1
Requires-Dist: pydantic-settings>=2.13.1
Requires-Dist: pyyaml>=6.0
Requires-Dist: stablehash>=0.3.0
Requires-Dist: typing-extensions>=4.15.0
Requires-Python: >=3.10
Project-URL: Homepage, https://github.com/NCATSTranslator/TRAPIObjectModeling
Project-URL: Repository, https://github.com/NCATSTranslator/TRAPIObjectModeling
Project-URL: Issues, https://github.com/NCATSTranslator/TRAPIObjectModeling/issues
Description-Content-Type: text/markdown

# TRAPI Object Modeling: `translator_tom`

A library for statically typed, fast serialize/deserialize TRAPI in Python, for Translator-wide use.

Models based on Pydantic provide deserialize with basic validation, serialize, and statically-typed construction with very reasonable performance, as well as utility methods based on architectural descisions, such as message/result/kg/etc. merging and standard TRAPI manipulation.

Allows for easy FastAPI standup.

## TRAPI versions

`translator_tom` provides models for multiple TRAPI versions. When importing directly from `translator_tom`, you automatically import models for the latest version.

```python
from translator_tom import Response                  # TRAPI 2.0 (latest)
from translator_tom.model_dicts import ResponseDict  # TRAPI 2.0 (latest)
```

To pin a specific version, import it from its version subpackage:

```python
from translator_tom.v2_0 import Response             # TRAPI 2.0
from translator_tom.v1_6 import Response             # TRAPI 1.6
from translator_tom.v1_6.model_dicts import ResponseDict
```

Each version has the same general API: models, model_dicts, diff, semantic validation (WIP). Some items are version-agnostic (Biolink, CURIEs, `TOMBase`, etc.) and shared between the two (`translator_tom.utils`).

### Converting TRAPI versions

The TRAPI 2.0 package provides a utility for converting 1.6 models to 2.0 models:

```python
from translator_tom import up_version

my_v1_response = ... # Some TRAPI 1.6 response

my_v2_response = up_version(my_v1_response)
```

## Model Usage

The main ways you interact with a Model are as follows:

- `Model.from_json()` and `Model.to_json()`
- `Model.from_dict()` and `Model.to_dict()`
- `Model.from_msgpack()` and `Model.to_msgpack()`
- Validated instantiation: `Model()`
- Direct construction: `Model.model_construct()`

### JSON Validation

These models can be used for validation of straight JSON:

```python
from translator_tom import Query

query_json = """
{
  "submitter": "TOM tester",
  "message": {
    "query_graph": {
      "nodes": {
        "n0": { "ids": [ "PUBCHEM.COMPOUND:726218" ] },
        "n1": { "ids": [ "NCBIGene:3778" ] }
      },
      "edges": {
        "e0": {
          "subject": "n0",
          "object": "n1",
          "predicates": [ "biolink:related_to" ]
        }
      }
    }
  }
}
"""

query = Query.from_json(query_json)

# Access is now statically typed and editor provides hints + completions
query_graph = query.message.query_graph
assert query_graph is not None
assert len(query_graph.nodes) == 2  # True
```

Similarly, you can validate from JSON with a FastAPI endpoint:

```python
from fastapi import FastAPI
from translator_tom import Query

app = FastAPI()

@app.post("/query")
def query(body: Query) -> str:
    return f"Got {len(body.message.query_graph.nodes)} query nodes!"
```

### Dict Validation

You can also validate dicts:

```python
from translator_tom import Query

query_dict = {
  "submitter": "TOM tester",
  "message": {
    "query_graph": {
      "nodes": {
        "n0": { "ids": [ "PUBCHEM.COMPOUND:726218" ] },
        "n1": { "ids": [ "NCBIGene:3778" ] }
      },
      "edges": {
        "e0": {
          "subject": "n0",
          "object": "n1",
          "predicates": [ "biolink:related_to" ]
        }
      }
    }
  }
}

query = Query.from_dict(query_dict)

query = Query(**query_dict)  # Also works (less clear, not recommended)
```

### Construction

There are two ways to construct instances within Python:

The first is just calling the model like any class. This ensures everything you pass is validated (meaning you don't need to construct every model and can just pass dicts, though static type checkers will complain).

```python
from translator_tom import Query

query = Query(
    submitter="TOM tester",
    # Could use types, or just pass a dict (static checkers will complain, though!)
    message={
        "query_graph": {
            "nodes": {
                "n0": {"ids": ["PUBCHEM.COMPOUND:726218"]},
                "n1": {"ids": ["NCBIGene:3778"]},
            },
            "edges": {
                "e0": {
                    "subject": "n0",
                    "object": "n1",
                    "predicates": ["biolink:related_to"],
                }
            },
        }
    },
)
```

Another way is to use `Model.model_construct()`.

> [!WARNING]
> This does no validation, so it's faster, but you **have** to pass correct construction for everything. No types will be coerced to their correct models. Only use it for internal construction where you know everything is already valid (a static type checker such as ty or pylance is highly recommended!). An example:

```python
from translator_tom import Biolink, Curie, Message, QEdge, QNode, Query, QueryGraph

# Using each type provides hints and type checking, making internal TRAPI construction
# safer.
query = Query.model_construct(
    submitter="TOM tester",
    message=Message(
        query_graph=QueryGraph(
            nodes={
                # Used a helper function to ensure curie formatting (optional)
                "n0": QNode(ids=[Curie("PUBCHEM.COMPOUND", "726218")]),
                "n1": QNode(ids=["NCBIGene:3778"]),
            },
            edges={
                "e0": QEdge(
                    subject="n0",
                    object="n1",
                    # Used a helper function to ensure biolink prefix formatting (optional)
                    predicates=[Biolink("related_to")],
                )
            },
        )
    ),
)
```

### Convenience Methods

TOM provides many convenience methods, similar to those in reasoner-pydantic.

```python
from translator_tom import KnowledgeGraph

my_kg = KnowledgeGraph.new()  # Init an empty knowledge graph

other_kg = get_some_kg()  # Imagine a function returns another KG with data...
_old_new_mapping = other_kg.normalize()  # Normalize edge IDs (keeps a mapping of old->new)

my_kg.update(other_kg, pre_normalized="other")  # Handles merging appropriately, can skip redundant normalization
```

There are many more, it's recommended to look at the models themselves as they are self-documenting (every model has docstrings equivalent to the descriptions in the original spec!). Common examples are `.<field>_list` and `.<field>_dict` for optional container fields for easy iteration without None-guarding, `.new()` for quick instantiation with sensible defaults (mostly of container-like models, but also for LogEntry with automatic timestamping), etc.

More in-depth utility methods include `.normalize()` for Message/KnowledgeGraph/Result/AuxiliaryGraph, `.prune()` for KnowledgeGraph, etc.

## TypedDict Usage

This library also provides `TypedDict` models, which can be used for internal static typing without class instantiation overhead, at the cost of some code verbosity.

- `*DictUtil.from_json()` and `*DictUtil.to_json()`
- `*DictUtil.from_msgpack()` and `*DictUtil.to_msgpack()`
- Direct construction: `*Dict()`

### JSON Reading

Unlike with models, the model_dicts don't validate by default.

```python
from translator_tom.model_dicts import QueryDictUtil, QNodeDictUtil


query_json = """
{
  "submitter": "TOM tester",
  "message": {
    "query_graph": {
      "nodes": {
        "n0": { "ids": [ "PUBCHEM.COMPOUND:726218" ] },
        "n1": { "ids": [ "NCBIGene:3778" ] }
      },
      "edges": {
        "e0": {
          "subject": "n0",
          "object": "n1",
          "predicates": [ "biolink:related_to" ]
        }
      }
    }
  }
}
"""

query = QueryDictUtil.from_json(query_json)  # returns type QueryDict

# These key accessors now have hints+completions in type-aware editors
query_graph = query["message"]["query_graph"]
assert query_graph is not None  # Type narrowing
assert len(query_graph["nodes"]) == 2  # True
n0_ids = query_graph["nodes"]["n0"].get("ids") or []  # `ids` is optional
assert n0_ids == ["PUBCHEM.COMPOUND:726218"]  # True

# DictUtils also provide safe accessors:
n0 = query_graph["nodes"]["n0"]
assert QNodeDictUtil.ids_list(n0) == ["PUBCHEM.COMPOUND:726218"]  # True


# A 'lite' version of validation may be optionally used
# This doesn't mutate the parsed dict, but throws ValidationError if it fails.
# Significantly faster than model validation; but not as thorough
query = QueryDictUtil.from_json(query_json, validate=True)
```


### Casting and direct instantiation

Oftentimes you'll just want to cast a model_dict:

```python
from typing import cast

from translator_tom.model_dicts import QueryDict

query_plain = {
    "submitter": "TOM tester",
    "message": {
        "query_graph": {
            "nodes": {
                "n0": {"ids": ["PUBCHEM.COMPOUND:726218"]},
                "n1": {"ids": ["NCBIGene:3778"]},
            },
            "edges": {
                "e0": {
                    "subject": "n0",
                    "object": "n1",
                    "predicates": ["biolink:related_to"],
                }
            },
        }
    },
}

# cast is free at runtime; it only tells the type checker to treat query_plain as a QueryDict.
query = cast("QueryDict", query_plain)
```

You can also just pass an already-existing dict to the dict constructor, although it produces a shallow copy:

```python
from translator_tom.model_dicts import QueryDict

# An existing dict you've annotated as a QueryDict (checked against it here).
query_plain = {
    "submitter": "TOM tester",
    "message": {
        "query_graph": {
            "nodes": {
                "n0": {"ids": ["PUBCHEM.COMPOUND:726218"]},
                "n1": {"ids": ["NCBIGene:3778"]},
            },
            "edges": {
                "e0": {
                    "subject": "n0",
                    "object": "n1",
                    "predicates": ["biolink:related_to"],
                }
            },
        }
    },
}

query = QueryDict(**query_plain)
```

### Direct construction

You can also use the model_dicts directly as construction guides:

```python
from translator_tom.model_dicts import (
    MessageDict,
    QEdgeDict,
    QNodeDict,
    QueryDict,
    QueryGraphDict,
)

# Each constructor provides key hints and type checking
query = QueryDict(
    submitter="TOM tester",
    message=MessageDict(
        query_graph=QueryGraphDict(
            nodes={
                "n0": QNodeDict(ids=["PUBCHEM.COMPOUND:726218"]),
                "n1": QNodeDict(ids=["NCBIGene:3778"]),
            },
            edges={
                "e0": QEdgeDict(
                    subject="n0",
                    object="n1",
                    predicates=["biolink:related_to"],
                )
            },
        )
    ),
)
```

## Semantic Validation (WIP)

A very WIP item is Semantic Validation:

```python
from translator_tom.validation import semantic_validate

warnings, errors = semantic_validate(some_model)  # Any TOM model
```

This returns a list of warnings and errors with clear descriptions and tuples describing their locations.

> [!WARNING]
> This feature is WIP and does not do every bit of semantic validation you might expect.

## Scripts

TOM provides some module-level scripts, for your convenience:

- `tom-parse`: Parse a given JSON into a given TOM model to check that it parses.
- `tom-validate`: Run semantic validation (WIP) against a given JSON/TOM model.
- `tom-up-version`: Upgrade a TRAPI 1.6 JSON to TRAPI 2.0.
- `tom-diff`: Diff two JSONs of a given TOM model.

## Design Decisions

There are a view caveats to using TOM, listed below:

### Implicit Enums

In some sections (such as `knowledge_type`), only certain values are used by existing systems, despite the field being an open string. In these cases TOM explicitly defines enums, as this may help to catch early validation errors. Some areas where the TRAPI spec defines short-codes that are not well-policed do not define enums.

### Literal over Enum

Python Literals are slightly faster for serialization, so internally, literals are used. Enums are still provided, largely for documentation access. All literals use the standard names you'd expect from TRAPI, while enums have `Enum` as a suffix.

### Using None where None is allowed, despite default

In TRAPI, some properties are non-required, but default to an empty list. TOM defaults these to None, to save serialized space. This is in-line with intended TRAPI 2.0 changes, and doesn't break interoperability.

### Hash calc + representation

Hashing is used in several cases, including KG normalization. TOM uses `stablehash` to ensure hashes are stable, and outputs hashes as unpadded base64url, with 120 bits truncation, by default. This offers a nice tradeoff of collision safety and hash shortening; all hashes are exactly 20 characters long.

### Differences from reasoner-pydantic

- Extra fields do not contribute to hashes
- Knowledge Node hash does not take `attributes` or `categories` into account
- MetaAttribute hash does not take into account name fields
- Message does not auto-normalize, and results do not auto-merge. You have to manually call the appropriate methods.
- BiolinkEntity, BiolinkPredicate, and BiolinkQualifier are now sub-types on the Biolink utility class.
  - This causes one issue: BiolinkPredicate and BiolinkEntity don't show up the JsonSchema generated from these models (but the patterns are preserved )
