Metadata-Version: 2.4
Name: ontoeval
Version: 0.1.0
Summary: Benchmarking AI agents against ontology PRs
Author-email: Chris Mungall <cjm@berkeleybop.org>
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: argilla>=2.8.0
Requires-Dist: click>=8.1.8
Requires-Dist: diskcache>=5.6.3
Requires-Dist: joblib>=1.5.1
Requires-Dist: pandas>=2.3.0
Requires-Dist: pydantic-ai>=0.2.15
Requires-Dist: pydantic>=2.11.5
Requires-Dist: pyyaml>=6.0.2
Description-Content-Type: text/markdown

# ontobench

This repo has code for creating benchmarks for ontology changes.

For each pull request:

- fetch metadata from github
- find linked issue(s)
- fetch the commit IDs of the "before" and "after" states
- create a diff/patch from the two IDs

All of this information will be stored in a pydantic data model, and exported to JSON

Note that commits are typically simple changes in the *-edit.obo file, but may involve other files

See workdir/ for some example repos that are checked out.


Examples

go-ontology

* 13117 - multiple stanzas