Metadata-Version: 2.4
Name: github-repo-atlas
Version: 0.2.0
Summary: Automated, read-only auditing and knowledge-base generation for scientific open-source GitHub repositories (structure, docs, FAIR readiness, transparency, attribution, agent-readiness).
Author: Damien Huzard
License-Expression: MIT
Project-URL: Homepage, https://github.com/dhuzard/github-repo-atlas
Project-URL: Documentation, https://github.com/dhuzard/github-repo-atlas#readme
Project-URL: Repository, https://github.com/dhuzard/github-repo-atlas
Project-URL: Issues, https://github.com/dhuzard/github-repo-atlas/issues
Keywords: fair,open-science,reproducibility,documentation,audit,agent-readiness,metadata,transparency
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Documentation
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer>=0.12
Requires-Dist: pydantic>=2.6
Requires-Dist: jinja2>=3.1
Requires-Dist: rich>=13.7
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Dynamic: license-file

# github-repo-atlas

**A neutral Python CLI, reusable GitHub Action, and agent-skill toolkit for
initializing and auditing trustworthy scientific open-source repositories.**

`github-repo-atlas` can scaffold a new repository or check an existing one. It
also scans the repositories you can reach on GitHub, extracts their structure
and documentation, audits them for scientific transparency, FAIR readiness,
attribution quality, reproducibility, and agent-readiness, and generates a rich
Markdown knowledge base — one card per repo, plus cross-repo synthesis, a
scoreboard, and reusable "vibe coding" lessons.

It is designed for a working scientist who wants (1) a durable technical memory
of all their repositories and (2) a library of reusable scientific-transparency
templates.

---

## What this project does

For every repository it can see, the atlas:

1. **Inventories** it via the GitHub CLI (`gh`) → `data/raw/repos.json`.
2. **Clones/updates** a local read-only copy in `cache/repos/OWNER__REPO/`.
3. **Extracts** structure, docs, dependencies, and git history →
   `data/processed/repos/OWNER__REPO.json`.
4. **Audits** a "Repo Trust Pack" (README, LICENSE, CITATION.cff, CREDITS,
   FAIR.md, AI_DECLARATION.md, AGENT.md, …) with deterministic heuristics.
5. **Scores** it across 7 explainable dimensions.
6. **Tracks trends** across runs — each audit snapshots the scores, so you can
   see deltas (▲/▼) and trajectories over time.
7. **Generates** a Markdown card, proposed transparency drafts, a scoreboard,
   cross-repo synthesis/lessons, a trends page, and a **workshop kit** for
   teaching good practices from your own repositories.

## Why it exists

Scientific software is only as trustworthy as its documentation, provenance, and
attribution. The atlas makes those qualities **measurable and visible** across a
whole portfolio, and lowers the cost of fixing gaps by generating draft
transparency files you can review and adopt.

## Safety model

This is the most important section. By design, the atlas:

- **Never modifies your repositories.** It only reads, clones, and fast-forwards
  (`git pull --ff-only`) local copies. It never pushes, commits to, or opens PRs
  against a target repo.
- **Never prints or copies secrets.** Secret detection is **by filename only**
  (e.g. `.env`, `*.pem`); file contents are never read or shown.
- **Writes everything inside this atlas repo** (`data/`, `cache/`, `docs/`,
  `packs/`). Target repositories are untouched.
- **Skips archived repos and forks by default** (opt in with `--include-archived`
  / `--include-forks`).
- **Requires no paid APIs and no API keys.** The deterministic V1 uses only the
  GitHub CLI you already authenticate. An LLM enhancement hook exists but is
  **optional and disabled by default** (`ATLAS_ENABLE_LLM=false`).
- **Never auto-validates** CRediT roles or AI declarations, and never treats a
  generated draft as authoritative.

Any future ability to open PRs to target repos would be a separate, explicitly
opt-in feature — it does not exist in this version.

## Requirements

- **Python 3.11+**
- **[GitHub CLI](https://cli.github.com/) (`gh`)**, authenticated — the only way
  the atlas reaches GitHub.
- Optional: **[Repomix](https://github.com/yamadashy/repomix)** for compact LLM
  packs (the audit works fine without it).

Python dependencies (installed automatically): `typer`, `pydantic`, `jinja2`,
`rich`, and `pytest` for tests.

## Installation

Install the isolated CLI from PyPI:

```bash
pipx install github-repo-atlas
atlas --help
```

To test the latest unreleased `main` branch instead:

```bash
pipx install "git+https://github.com/dhuzard/github-repo-atlas.git@main"
```

To contribute from a source checkout:

```bash
git clone https://github.com/dhuzard/github-repo-atlas.git
cd github-repo-atlas
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\Activate.ps1
pip install -e ".[dev]"
```

## Quick start: initialize a repository

Create a Python research repository with tests, CI, citation metadata, agent
instructions, and FAIR, TRUST, reproducibility, attribution, and AI-use
declarations:

```bash
atlas init behavioral-analysis \
  --path ./behavioral-analysis \
  --owner my-lab \
  --author "Jane Researcher" \
  --description "Reproducible analysis of behavioral experiment data" \
  --with-data

cd behavioral-analysis
atlas check . --fail-on never
```

The first check intentionally identifies factual placeholders requiring human
review. After completing them from evidence, enable the strict quality gate:

```bash
atlas check . --fail-on warning
```

For the generated file tree, Windows PowerShell commands, GitHub publication,
and declaration-review workflow, see **[Initiate a new repository with the CLI
or Codex in VS Code](docs/initiating-a-new-repository.md)**.

### Initialize with Codex in VS Code

Open the parent directory in VS Code and ask Codex:

```text
Use the atlas CLI to initialize a new repository named behavioral-analysis
inside ./behavioral-analysis.

Owner: my-lab
Author: Jane Researcher
Description: Reproducible analysis of behavioral experiment data
Include the data declarations.

Initialize git, create a virtual environment, install the generated project,
run pytest, and run `atlas check . --fail-on never`. Report every declaration
that still requires factual human input. Do not invent authorship, funding,
identifiers, validation results, consent, compliance, AI use, or reuse rights.
```

Codex performs the workflow; the deterministic CLI remains the quality gate.

## GitHub authentication

The atlas uses `gh`, not a token file:

```bash
gh auth login          # one-time, interactive
gh auth status         # verify you are logged in
```

You can check your environment at any time without scanning anything:

```bash
python -m atlas.cli doctor
```

Under the hood the inventory step runs the equivalent of:

```bash
gh api '/user/repos?visibility=all&affiliation=owner,collaborator,organization_member&per_page=100&sort=updated' --paginate
```

If `gh` is missing or unauthenticated, the atlas **fails clearly** and tells you
to run `gh auth login` — it never continues with fabricated data.

## First run

Start small (5 repos, no Repomix):

```bash
python -m atlas.cli full --max-repos 5 --skip-repomix
```

Then the full run:

```bash
python -m atlas.cli full
```

Scope which accounts/orgs are included via `.env` (`ATLAS_GITHUB_LOGINS`) or the
`--logins` flag, e.g. `--logins dhuzard,neuronautix`.

### Individual stages

Every stage is idempotent and runnable on its own:

```bash
python -m atlas.cli inventory            # -> data/raw/repos.json
python -m atlas.cli clone                # -> cache/repos/OWNER__REPO
python -m atlas.cli extract              # -> data/processed/repos/*.json (+ audit + scores + trend)
python -m atlas.cli audit                # print Trust Pack summary
python -m atlas.cli generate-cards       # -> docs/repo_cards/*.md
python -m atlas.cli generate-trust-pack  # -> docs/transparency_pack/OWNER__REPO/*.md
python -m atlas.cli synthesize           # -> scoreboard + synthesis + lessons + trends
python -m atlas.cli trends               # print the score trend across runs
python -m atlas.cli workshop             # -> docs/workshop/ teaching kit
python -m atlas.cli backfill-history     # seed trend baselines from existing reports (no re-scan)
```

Flags: `--max-repos N`, `--include-archived`, `--include-forks`,
`--skip-repomix`, `--logins a,b`.

## Output structure

```text
data/raw/repos.json                     # raw GitHub inventory
data/processed/repos/OWNER__REPO.json    # deterministic per-repo report
cache/repos/OWNER__REPO/                 # local read-only clone
packs/repomix/OWNER__REPO.repomix.md     # optional LLM pack
docs/repo_cards/OWNER__REPO.md           # human-readable repo card
docs/transparency_pack/OWNER__REPO/*.md  # proposed AI-assisted drafts (review!)
docs/audits/scoreboard.md                # portfolio table (with Δ + trend columns)
docs/audits/transparency_audit.md        # ecosystem transparency coverage
docs/audits/trends.md                    # cross-run score trends
docs/synthesis/cross_repo_synthesis.md   # portfolio-level synthesis
docs/lessons/*.md                        # vibe coding lessons + templates
docs/workshop/*.md                       # teaching kit (curriculum, case studies, ...)
data/reports/history/                    # append-only trend snapshots (committable)
```

## How to interpret the scores

Seven deterministic scores, each **0–100** and decomposed into weighted
components (visible in every repo card and in `data/processed/`):

| Score | What it measures |
|---|---|
| **Repo Usability** | Can a newcomer install, run, and test it from the README? |
| **Agent Readiness** | Can an AI coding agent work safely (AGENT.md, tests, CI, clear commands, no secret risk)? |
| **Transparency** | Coverage of the Trust Pack documentation set. |
| **FAIR Readiness** | Findable / Accessible / Interoperable / Reusable signals. |
| **Attribution Quality** | Contributors, CRediT roles, software/funding credit, AI disclosure. |
| **Reproducibility** | Pinned environment, tests, CI, documented run steps. |
| **Scientific Trust** | Validation, limitations, reproducibility, citation, AI disclosure. |

**Important:** these scores measure *documentation and engineering hygiene* — a
proxy for trustworthiness. They are **not** a certification of scientific
validity. A high score means "the repo plausibly documents the right things",
never "the science is correct".

## Cross-run trend tracking

Every run appends a compact snapshot of each repo's scores to
`data/reports/history/` (per-repo `*.jsonl` + a portfolio `runs.jsonl`). From
that history the atlas computes:

- **Δ since last audit** on the scoreboard and each card (▲/▼/—).
- **Sparklines** (`▁▂▃▄▅▆▇█`, scaled 0–100) showing each repo's trajectory.
- A **trends page** (`docs/audits/trends.md`) with the portfolio average over
  time, a per-repo trajectory table, and "most improved" / "declining" lists.

History is the one artefact that **cannot** be regenerated from a fresh scan, so
it is git-committable by default (the rest of `data/` is ignored). Identical
consecutive snapshots are de-duplicated, so re-running without changes adds no
noise. Use `--no-history` to skip recording, or `trends` to print the table.

**Seeding from existing reports:** if you have reports from before trend tracking
existed, run `python -m atlas.cli backfill-history` once to establish baselines
without re-scanning — the next audit will then show real deltas.

## Teaching & workshops (`workshop` command)

`python -m atlas.cli workshop` turns your audited portfolio into a ready-to-teach
kit in `docs/workshop/`, so you can run trainings on good "vibe coding" practices
using **your own repositories as the evidence**:

| File | What it is |
|---|---|
| `README.md` | Index + suggested formats (90-min, half-day, team retro). |
| `curriculum.md` | 6 modules (runnable → agent-ready → reproducible → FAIR → attribution → AI/ethical debt), each with a real exemplar repo, a contrast repo, a reusable template, and an exercise. |
| `case_studies.md` | Your flagship repo, your biggest opportunity, and improvement stories drawn from trends. |
| `playbook.md` | The 7 durable principles + quick wins ranked by how often they apply across your repos. |
| `exercises.md` | Time-boxed hands-on tasks that produce a measurable score change. |
| `slides.md` | A Marp/Reveal-compatible slide outline. |

Why teach from your own repos: participants can open the repo, see the score,
see exactly which file is missing, add it, and watch the score move on the next
run. The maturity ladder (Level 1 Ad hoc → Level 5 Exemplary) and the trends page
make that feedback loop the centrepiece of the session.

**The teaching loop:** measure → pick the lowest dimension → apply that module's
template → re-audit → show the delta. Run one improvement live and the trends
page records your first "improvement story" during the session.

## Improving existing repos & starting new projects

The atlas is read-only by default, but it ships two **explicit, opt-in** commands
to act on what it finds. Neither ever commits or pushes — you review and commit.

### Check any local repository — `check`

Run the declaration quality checker without inventorying or cloning GitHub:

```bash
# Human-readable report; only unresolved placeholders fail by default
atlas check /path/to/repo

# CI annotations + JSON; missing/weak recommended declarations fail
atlas check /path/to/repo \
  --format github \
  --output declaration-quality.json \
  --fail-on warning
```

The checker recognizes root `fair.md` / `trust.md`, uppercase variants, and the
existing `docs/FAIR.md` / `docs/TRUST.md` Trust Pack layout. It checks
declaration structure and unresolved placeholders; it does **not** certify
scientific correctness, FAIRness, security, or epistemic trust.

### Start a new project the right way — `init`

Scaffold a new repository that is agent-ready, FAIR and transparent from the
first commit (it would score highly on the atlas immediately):

```bash
atlas init my-new-tool --owner dhuzard --description "What it does"
```

This creates README, LICENSE, CITATION.cff, CREDITS.md, CONTRIBUTING.md,
SECURITY.md, the full `docs/` Trust Pack (FAIR, AI_DECLARATION, DESIGN, TRUST,
REPRODUCIBILITY, LIMITATIONS, ETHICAL_DEBT, AGENT), a root `AGENTS.md`, a Python
package + smoke test, a CI workflow, `.gitignore` and `.env.example`. Add
`--with-data` for DATA/MODEL cards + governance. It also includes a declaration
quality workflow using this repository's reusable action. The scaffold's own
smoke test passes out of the box; the declaration workflow intentionally stays
red until you replace the factual TODOs. Until a `v1` tag is published, the
generated workflow uses `@main` and includes a reminder to pin a release or SHA.

### Improve an existing repo — `apply-pack`

After `generate-trust-pack`, materialize a repo's AI-assisted drafts into *your
working copy* of that repo (not the atlas's clone cache) for review:

```bash
# 1. preview (default: writes nothing)
python -m atlas.cli apply-pack --repo dhuzard/HCMO --into /path/to/your/HCMO
# 2. actually add the missing files
python -m atlas.cli apply-pack --repo dhuzard/HCMO --into /path/to/your/HCMO --write
# then, in YOUR repo:
git checkout -b add-trust-pack && git add -A && git commit -m "Add Trust Pack drafts"
# open a PR yourself — the atlas never pushes
```

Safety: dry-run unless `--write`; only adds missing files unless `--force`;
refuses to write into `cache/repos/`; never runs git. The drafts keep their
"AI-assisted — requires human review" banner so you review before adopting.

## How to use the generated transparency files

`docs/transparency_pack/OWNER__REPO/` contains **proposed drafts** for missing or
weak Trust Pack files (FAIR.md, AI_DECLARATION.md, DESIGN.md, TRUST.md,
REPRODUCIBILITY.md, CREDITS.md, ETHICAL_DEBT.md, AGENT.md). Each begins with:

> Status: AI-assisted draft generated from repository structure and metadata.
> Requires human review before being treated as authoritative.

Workflow: **review → edit → copy into the target repo yourself.** The atlas
never places them there for you. Blank, reusable versions of all templates live
in `templates/trust_pack/`.

## Reusable quality gate

Other repositories can use the composite GitHub Action after checking out their
source:

```yaml
permissions:
  contents: read

steps:
  - uses: actions/checkout@v5
  - uses: dhuzard/github-repo-atlas@v0.2.0
    with:
      fail-on: warning
```

Update the version deliberately when adopting a new release.
Security-sensitive consumers should pin the action to a reviewed full commit
SHA. The CLI remains the portable option for GitLab, local pre-commit, HPC, and
non-GitHub environments.

## Framework source refresh

[`framework/sources.toml`](framework/sources.toml) is the source-of-truth
registry for every recommended Markdown declaration. FAIR and TRUST are
upstream-governed by
[`Neuronautix/FAIR.md`](https://github.com/Neuronautix/FAIR.md) and
[`Neuronautix/TRUST.md`](https://github.com/Neuronautix/TRUST.md); the other
declarations currently point to their locally governed templates. Each source
can later move to an independent specification repository without changing the
pipeline.

```bash
# Exit 0: current; 2: check failed; 3: reviewable drift
atlas framework-check \
  --report framework-source-report.md \
  --json-output framework-source-report.json \
  --snapshot-dir .framework-sources

# Only after implementation + human review
atlas framework-check --accept
```

The weekly and manually triggered
[`framework-refresh.yml`](.github/workflows/framework-refresh.yml) workflow
builds a reproducible evidence bundle. On explicit manual request,
`agentic_proposal=true` sends that bundle to the official Codex GitHub Action in
read-only mode and opens its proposal as an issue. It never auto-accepts a new
specification, edits standards, or merges a change. Configure the
`OPENAI_API_KEY` repository secret only if you want that optional step.

## Codex and Claude skills

Two portable Agent Skills are checked into both discovery locations:

- `audit-repo-declarations` — assess, initialize, or improve a repository
  without inventing scientific claims.
- `refresh-declaration-framework` — turn upstream drift into a tested,
  human-reviewed framework update.

Codex discovers them under [`.agents/skills/`](.agents/skills/); Claude Code
discovers the byte-identical copies under
[`.claude/skills/`](.claude/skills/). The skills deliberately depend on the
CLI's deterministic evidence rather than reimplementing checks in prompts.

For broader distribution, keep the skills in the open `SKILL.md` format, tag
releases of this repository, and package them as a Codex/ChatGPT plugin or
Claude plugin only as an additional install channel. The CLI + JSON report +
GitHub Action should remain the stable public contract for developers and
scientists who do not use either agent.

## Community adoption model

The most sustainable split is:

1. **Conventions:** version FAIR.md and TRUST.md independently with specification,
   schema, template, changelog, examples, and conformance fixtures.
2. **Conformance tooling:** publish this atlas as a Python package/CLI and a
   versioned GitHub Action; keep checks explainable and offline where possible.
3. **Agent workflows:** distribute the same open Agent Skills for Codex and
   Claude; agents propose changes, deterministic checks verify them.
4. **Governance:** use normal issues, RFC pull requests, releases, CITATION.cff,
   and archived release DOIs. Treat adoption evidence and false-positive reports
   as inputs to the next specification version.

This avoids making an LLM vendor a prerequisite for using the declarations and
gives scientific users a citeable, versioned, reviewable standard.

## What is deterministic vs AI-assisted

- **Deterministic (this version):** the entire pipeline — inventory, extraction,
  Trust Pack audit, all 7 scores, cards, scoreboard, synthesis, and the
  *structure* of the transparency drafts. Given the same repos, you get the same
  output. No LLM is involved.
- **AI-assisted (optional, off by default):** the framework refresh workflow can
  ask Codex to review a deterministic drift bundle and open a proposal issue.
  It requires an explicit manual trigger and `OPENAI_API_KEY`; it cannot accept
  or merge a standards update.

## Limitations

- The audit uses **filename + keyword heuristics**. It detects the *shape* of
  good documentation, not its correctness.
- Secret detection is **filename-only** — it will miss secrets hidden inside
  otherwise-innocuous files, and may flag harmless files by name.
- Contributor roles are **inferred from git authorship** and must be
  human-validated; they are never asserted as fact.
- Scores are proxies; treat them as a to-do prioritiser, not a grade.
- `gh` rate limits and repository size affect scan time.

## Roadmap

- ✅ Trend tracking across runs (score deltas over time) — **done**.
- ✅ Workshop / teaching kit derived from your portfolio — **done**.
- ✅ `init` scaffolder for new projects + `apply-pack` for existing repos — **done**.
- ✅ Portable `check` command + reusable GitHub Action — **done**.
- ✅ Upstream registry, reviewed lock, scheduled drift evidence, and opt-in
  Codex proposal — **done**.
- ✅ Cross-compatible Codex/Claude Agent Skills — **done**.
- Optional, opt-in LLM enrichment of draft prose (still human-reviewed).
- Richer FAIR signals (DOI/Zenodo detection, metadata standard recognition).
- Trend charts (SVG sparklines) embedded in cards.
- Optional `--pr` flag on `apply-pack` to open a review PR via `gh` (opt-in).
- Per-language deepening of install/run/test inference.

## Development

```bash
python -m pytest                 # run the test suite
python -m atlas.cli doctor       # check gh / repomix availability
```

## License

MIT (see `LICENSE`). This atlas reports on other repositories; it makes no claim
about their licenses beyond what it reads from their metadata.
