
This is a toolset for manipulating single cell expression matrices. The functionality
is organized in modality accessors. To help generative agents to better understand
the pipeline in the runtime, I decided to implement a pipeline dependency graph for
AI to inspect next possible move.

A previous implementation is done in dependency_graph.py, history_schema.py and history.py
as well as in the function wrappers in accessor.py. You should read carefully the 
implemented logic to get an idea of the system. But now, you can freely design a new
system in replacement of this. The old system is less organized and obsoleted.

The tools are in exprmat/reader/rna, exprmat/reader/spatial, and exprmat/reader/atac.

-----

Build an introspective and command history system:

For each adata calling tool functions registered in the accessor, append a history
record if successful after the tool run. Also provide introspection to registerred
tools in accessors by implementing:
```
- accessor.get_history(sample, uid)
- accessor.get_interpretation(sample, uid)
```
and the introspective properties of tools
```
- accessor.list_tools()
- accessor.list_available_tools(sample)
- accessor.introspect_parameters(tool)
- accessor.introspect_dependencies(tool)
- accessor.introspect_slots(sample)
- accessor.introspect_slots_by_classes(sample, classes)
```
A tool may depend on several data slots required, of certain type. A registry 
for each slots are maintained along with history system. Histories are recorded
per-adata in .uns['commands'], and slot registry is recorded in .uns['slots']
A tool will write their output (if there is) to the adata in certain slots, thus
inserting slot registry to inform later tool calls. Sometimes the tool write is
conditional, only when writing the slot do they insert registry.

When a tool creates a new slot, it should write into the slot registry to provide
information and capability of the slot data. The slot registry seems to be
```
{
    'group': 'obs',
    'name': 'velocity.pseudotime',
    'class': ['float', 'coordinate'],
    'description': 'Pseudotime inferred from RNA velocity'
}
```
The class attribute specifies whether the data slot may be useful as other tool
required. A tool may request a data with 'coordinate' in the class properties.

The tool history will be added like
```
{ # for computation
    'uid': '8f3e1a2b',
    'tool': 'umap',
    'qualname': 'exprmat.reader.rna.reduction.rna_umap',
    'kind': 'computation',
    'version': schema_version,
    'timestamp': '2026-04-27T12:00:00Z',
    'sample': 'integrated',
    'modality': 'rna',
    'scope': 'integrated',
    'parent_uid': None,
    'split': None,

    'interpretation': {
        'description': '...',
        'features': ['...',],    # highlights
        'notes': ['...',]        # caveats
    }
}
```
```
{ # for plottings
    'uid': 'a1b2c3d4',
    'tool': 'plot_embedding',
    'qualname': 'exprmat.reader.rna.reduction.rna_plot_embedding',
    'kind': 'plot',
    'version': schema_version,
    'timestamp': '2026-04-27T12:01:14Z',
    'sample': 'integrated',
    'modality': 'rna',
    'scope': 'split',
    'parent_uid': '7766aabb',
    'split': {'key': 'leiden', 'value': '3'},

    'interpretation': {
        'description': '...',
        'features': ['...',],    # highlights
        'notes': ['...',]        # caveats
    }
}
```
Another thing is that the developer of the tool may provide interpretation functions,
like where they interpret key findings of a graphical output, to a model with no visual
capability. If unspecified, the interpretation will be automatically generated with blank

Any tool should set up the requirement chain like this:
```
@requires([
    require_obs(param = 'key_pseudotime', require = ['coordinate'], desc = 'Pseudotime'),
    require_obs(param = 'key_leiden', require = ['categorical'], desc = 'Cell type categories')
])
@writes([
    # only when .uns['trace'] actually exists in the adata, will this be registered.
    output_uns(name = 'trace', classes = ['dict', 'data.trace'], desc = 'Unstructured data')
])
@interprets(interpret_trace)
def trace(adata, sample, spatial, key_pseudotime = '...', key_leiden = '...', a = 0, kwargs):
    ...
```
This explicitly declares the dependency and consequence of the tool to the anndata.

For the introspection of parameters, which are static, it is directly reused from the
docstring. You should implement docstrings in common capitalization and insert docstrings
where the tool do not have one, and format existing or new docstrings into numpy-like style.

`introspect_slots` function provide a JSON description of each slot. or by classes 
which match all the specified classes.

-----

Possible writers:

    output_obs(param = '...', classes = [...], desc = '...') # if specified from a parameter
    output_obs(name = '...', classes = [...], desc = '...')  # if the name is static.

    output_var(...)
    output_obsm(...)
    output_obsp(...)
    output_varm(...)
    output_uns(...)
    output_layers(...) # layer name can be 'X'
    output_image(param = '...', classes = [...], desc = '...')


Possible data families (classes):

    For obs:
        # object contains any value types (e.g. strings, IDs, names) that may not be 
        # considered as a sensible category
        # each element must have a basic data class at the start of the list.
        Basic data classes: 
            'category', 'float', 'int', 'bool', 'object'
        Sequential float: 
            'coordinate' (1d axis that represent a sequential order, e.g. distances axis, pseudotime axis etc.)
    
    For obsm:
        Basic: ...
        Embedding coordinates:
            'coordinate:nd', 'coordinate:<n>' (n is the actual dimension)
        Scoring:
            also a coordinate, plus 'scoring:nd', 'scoring:<n>'
        Weights:
            'proportions': for matrics that row sum is one.
            'weights': for matrices that col sum is one.
        Specific:
            'knn', for knn indices obsm
            'knnd', for knn distances for the indices
            ... adds more as you infer from the tool
    
    For varm:
        Basic: ...
        Others:
            'weights'
            'proportions'
    
    For obsp:
        Basic: ...
        Others:
            'adjacency': binary matrix of adjacency
            'connectivity': float matrix of adjacency, correlation is also connectivity.

    For uns:
        Basic: 'dict', ...
        Commonly each tool have different incompatible format.
        Register a tag in the table below for each tool you work on.
        

    For obsm:
        Embedding coordinates:
            ['float', 'coordinate:nd', 'spectral']     Spectral embedding (atac_spectral)
        Scoring:
            ['float', 'coordinate:nd', 'scoring:nd']   Generic scoring coordinate
        Weights:
            ['float', 'proportions']                   Row-sum = 1
            ['float', 'weights']                       Col-sum = 1
        Specific:
            ['int', 'knn']                              KNN index matrix
            ['float', 'knnd']                           KNN distance matrix

    For varm:
        ['float', 'motif', 'motif-matches']             Peak-by-motif binary matrix
        ['float', 'weights']                            Per-feature weights
        ['float', 'proportions']                        Per-feature proportions

    For obsp:
        ['float', 'connectivity']                       Float adjacency / correlation
        ['bool', 'adjacency']                           Binary adjacency matrix

    For layers:
        ['int', 'counts']                               Raw integer counts
        ['float', 'linear']                             Linear-normalised (e.g. CPM)
        ['float', 'normal']                             Non-linear normalised (e.g. log)
        ['float', 'zscores']                            Scaled z-scores

    For images:
        ['image', 'mask']                               Binary 2-D mask
        ['image', 'rgb']                                Three-channel RGB image
        ['image', 'layers']                             Multi-channel (not RGB, not mask)
    
    For layers:
        Basic: ...
        For integer counts: use ['int', 'counts']
        For linearly normalized: use ['float', 'linear']
        For non-linearly normalized: use ['float', 'normal']
        For scaled z-scores: use ['float', 'zscores']

    For images:
        If layers is static, declare it, otherwise dynamic, left empty.
        ['image', 'mask']: a 2D mask
        ['image', 'layers', 'rgb']: a 3-channel RGB image
        ['image', 'layers']: if not rgb and not mask.


Possible requirements:

    external anndata is not to be a requirement, just skip it.

    require_obs
    require_var
    require_obsm
    require_obsp
    require_varm
    require_uns
    require_layers
    require_features
    require_image

If the tool attempt to create a new anndata in the experiment
commonly seen by `on_complete = append_from_result` in the tool registry,
Use output_experiment, and with a # TODO: comment to warn me to check.