Metadata-Version: 2.4
Name: oracle-scan
Version: 0.1.2
Summary: Scan a repository for leaked secrets, vulnerable dependencies, risky code and repository hygiene problems.
Author-email: darkblightHQ <darkblightHQ@users.noreply.github.com>
Maintainer-email: darkblightHQ <darkblightHQ@users.noreply.github.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/darkblightHQ/oracle
Project-URL: Repository, https://github.com/darkblightHQ/oracle
Project-URL: Issues, https://github.com/darkblightHQ/oracle/issues
Project-URL: Changelog, https://github.com/darkblightHQ/oracle/blob/master/CHANGELOG.md
Project-URL: Security, https://github.com/darkblightHQ/oracle/security/policy
Keywords: security,secrets,secret-scanning,sast,sca,scanner,vulnerability,devsecops,github,cli
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Information Technology
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.27
Requires-Dist: rich>=13
Requires-Dist: textual>=1.0
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Dynamic: license-file

# oracle

![CI](https://github.com/darkblightHQ/oracle/actions/workflows/ci.yml/badge.svg?branch=master)
[![PyPI](https://img.shields.io/pypi/v/oracle-scan.svg)](https://pypi.org/project/oracle-scan/)
[![PyPI downloads](https://img.shields.io/pypi/dm/oracle-scan.svg)](https://pypi.org/project/oracle-scan/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
![Python 3.11+](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-3776AB.svg)
![No network required](https://img.shields.io/badge/scan-offline--friendly-success.svg)

Scan a repository for leaked secrets, vulnerable dependencies, risky code
patterns, and the repository-hygiene mistakes that make the first three
inevitable.

`argus` scans the outside of a system and `lattice` maps what is exposed. This
one takes whatever repository code you point it at - local checkout or GitHub
clone - and answers four questions at once:

| question | detection |
|---|---|
| *is a credential sitting in the tree?* | provider formats (AWS, GitHub, Stripe, OpenAI, Slack...) plus a high-entropy backstop, user-defined regex rules from a TOML file, and a generic rule for sensitively-named assignments |
| *did a credential used to sit there?* | the same secret rules re-run over `git log -p` added lines |
| *are the pinned dependencies known-vulnerable?* | exact pins matched against a bundled advisory set, optionally live via osv.dev |
| *is the code doing something structurally unsafe?* | SAST-lite rules for eval, shell=True, HTML injection sinks, disabled TLS verification, SQL concatenation ... |
| *would a real attacker shrug?* | committed `.env`/keys, `pull_request_target`, write-all workflows, secret baked into a Dockerfile layer, no SECURITY policy ... |

## Why it is shaped this way

Four decisions do most of the work.

**The same line can be scanned twice.** Secrets in the working tree are
transient: someone deletes `settings.py` after the leak gets noticed, and the
incident writes itself off as fixed. But the commit still exists, and it has
already been harvested by every scanner in orbit. When a repository has git
history, `oracle` streams `git log -p --all` and re-runs the *secret* rules
against every *added* line, tagging each finding with the commit that
introduced it. Tree findings and history findings share one detection engine
and one fingerprint space, so a secret that is still in the tree and also in
history is one finding, not two.

**Deposits are pinned, or not asserted.** A manifest range like `>=1.0,<2.0`
does not say what is installed; claiming a vulnerability for one would be a
confident lie. Only exact pins are matched. Ranges are recorded and left
alone. The bundled advisory database is a curated, deterministic set so a scan
works offline; `--osv` adds a live batch query against osv.dev for current
coverage.

**Fingerprints ignore lines, not meaning.** The baseline fingerprint is built
from the rule, the path and the credential-or-snippet - deliberately not the
line number or the commit. A reformat that shifts a leaked secret down ten
lines does not expire your baseline and re-alarm CI on a known finding.

**Renderers cannot leak by accident.** Findings carry the raw matched value in
memory, but every renderer redacts it - keeping just the first/last few
characters plus the length so an owner can still identify it - unless you
explicitly pass `--show-secrets`. That is a statement about the code, not a
convenience flag you are expected to type for every scan.

## Install

```sh
pip install oracle-scan
```

The command is `oracle`:

```sh
oracle --help
```

From a checkout, for development (tests, lint, editable install):

```sh
git clone https://github.com/darkblightHQ/oracle
cd oracle
python -m venv .venv
.venv/bin/pip install -e ".[dev]"
```

To run the editable build from anywhere, either activate its venv for your
session:

```sh
source .venv/bin/activate
oracle --help
```

or put a link on your PATH once:

```sh
ln -sf "$(pwd)/.venv/bin/oracle" ~/.local/bin/oracle
oracle --help
```

## Quick start

```sh
oracle                                        # open the interactive console
oracle scan .                                 # scan the current directory
oracle scan owner/repo                        # clone and scan a GitHub repo
oracle scan ./some/path --json | jq .summary  # data on stdout, chatter on stderr
oracle scan . --osv                           # also ask osv.dev about dependencies
oracle scan . --sarif -o sarif.json           # SARIF for a code-scanning pipeline
oracle report . -o report.md                  # write a markdown report
oracle scan . --write-baseline baseline.json  # allowlist today's findings
oracle scan . --baseline baseline.json        # suppress previously reviewed ones
```

Data goes to stdout and chatter to stderr, so `oracle scan . --json | jq`
works as expected - the default format is JSON when stdout is not a terminal,
and exit codes are stable enough to gate a script.

Exit code 1 means "findings at or above your threshold": the default
`--fail-on high` fails CI on anything critical or high. `--fail-on never`
scans without gating, `--fail-on critical` only blocks on the worst.

## The console

Bare `oracle` opens a small Textual console over the same engine the CLI uses:
type a path or an `owner/repo`, scan, and browse findings - worst first - with
a detail pane underneath. While a scan runs a status line shows a spinner with
live progress (files done, findings so far) and the scan button locks; when it
finishes the line reports the summary and elapsed time. The tree walk,
per-file analysis, advisory pass and the `git log` history pass all run off
the event loop, so the window stays responsive even on large repositories, and
a long scan can be abandoned with `escape`.

![The oracle console: scanning a repository, then browsing findings with the detail pane](https://raw.githubusercontent.com/darkblightHQ/oracle/master/docs/demo.gif)

| key | does |
|---|---|
| `q` | quit |
| `ctrl+r` | re-scan the current target |
| `escape` | cancel the in-flight scan |
| enter / click | open the finding: actual source line, caret on the match, the matched value, message, fix and references |

The detail pane is the one place a raw credential is shown by design: this is
an interactive review surface on the operator's own machine, and the value is
read on demand and never written to a file. Reports and `--json` output keep
the strict redaction contract.

On a terminal the console opens automatically; in a script or CI it degrades
to usage help so a piped `oracle` never hangs. Set `ORACLE_FORCE_TUI=1` to
force it, and `oracle --tui` to ask for it explicitly.

## Commands

### `scan TARGET...`

The whole tool: secrets, dependency advisory matching, SAST-lite and
repository hygiene, plus git-history secret recovery when the target is a git
repository. Multiple targets are scanned and merged into one result.

A target is either a local directory or a GitHub reference. Bare `owner/repo`
takes precedence over a local relative path of the same shape - if you mean a
local directory, write `./owner/repo`. GitHub refs are

```
owner/repo
https://github.com/owner/repo
https://github.com/owner/repo.git
https://github.com/owner/repo/tree/main/src
git@github.com:owner/repo.git
ssh://git@github.com/owner/repo
```

Remote repositories are cloned into `~/.cache/oracle/repos` and reused on the
next scan (`--refresh` fetches first); history scanning gets the full object
database either way. Private repositories work by passing a token via
`--token`, `GITHUB_TOKEN` or `GH_TOKEN` - it is inserted into the clone URL and
never printed.

Selective switches (`--no-secrets`, `--no-deps`, `--no-sast`,
`--no-hygiene`, `--no-history`) disable a pass; `--min-severity` filters the
report; `--exclude GLOB` prunes paths (repeatable, matched against the
relative path and the basename). Entropy can be tuned via
`--no-entropy`, `--entropy-threshold` and `--entropy-min-length`. Custom
rules are loaded via `--rules FILE` (repeatable); run `oracle rules --rules
FILE` to list them. Files over 2 MiB and vendored directories
(`node_modules`, `.git`, `vendor`, `dist`, caches...) are skipped outright.

### `rules [ID...]`

Every secret, SAST and hygiene rule this tool knows, with severity, so the
`--exclude` / `--baseline` / triage conversations happen against a real
inventory. `oracle rules aws --json` filters by rule-id substring; `oracle rules --rules
custom.toml --json` lists built-ins plus your custom rules.

### `report TARGET [-o FILE]`

Scan one target and write a markdown report (the default output, so `report`
always ends in a useful file). The same redaction contract applies: full
credentials only with `--show-secrets`.

## Detection coverage

- **Secrets** - AWS access keys, GitHub PATs and fine-grained tokens, Stripe
  live keys, OpenAI keys, Slack tokens and incoming webhooks, SendGrid,
  Twilio, Mailgun, Telegram, Discord webhooks, Google API keys, Shopify,
  npm/PyPI tokens, Hugging Face, Postman, Square, Mapbox, Vault, Datadog,
  git-grade private-key blocks, JWTs and credentials in URLs, plus an
  assignment-based **entropy backstop** that catches long random values even
  when the variable name is uninformative. **Custom regex rules** can be loaded
  from TOML via `--rules` for your own formats. Placeholder values
  (`changeme`, `your-api-key`, all-digit short strings, single-symbol runs,
  sequential alphabet/number strings, obvious base64/image magic) are
  rejected.
- **Dependencies** - `requirements.txt`, `Pipfile.lock`, `poetry.lock`,
  `pyproject.toml`, `package.json`, `package-lock.json`, `yarn.lock`,
  `pnpm-lock.yaml`, `go.mod`, `Gemfile.lock`, `Cargo.lock`, `composer.lock`.
  The bundled advisory set is a curated sample (36 entries) documented in
  `data/advisories.json`; `--osv` queries osv.dev live and `--advisories FILE`
  adds your own entries in the same JSON shape.
- **Code** - a small, language-tagged rule set for python, javascript/
  typescript, go, java, shell and generic patterns. Not a real SAST engine:
  every rule is a pattern that production code virtually never contains, so
  the false-positive rate stays low.
- **Repo hygiene** - committed `.env`/key files (`.env.example` and friends
  are allowed), GitHub Actions triggers and permissions, Dockerfile secrets,
  and the repo-level facts that matter even when no file is dangerous:
  LICENSE, SECURITY policy, `.gitignore` ignoring `.env`.

### Formats

`--format` / `--json` / `--sarif` / `--md`, or `-o FILE`. JSON is the
structured contract (same shape as `to_dict` in the codebase); SARIF is
2.1.0 with one `ruleId` per rule and a stable `partialFingerprint`; Markdown
groups findings by severity and names the fix for each.

## Discovered secret?

Rotate it. A leaked credential is compromised the moment it leaves the repo,
not when it is fixed. `oracle` can find them; only the owner can make them
stop working.

## Politeness

Scan only repositories you are entitled to scan. The tool does no scanning of
the network beyond git clone/fetch of the target you name, and it reuses a
cached clone across runs. `--osv` is the only always-online extra and it is
opt-in; queries are batched into one POST per run.

## Exit codes

| code | meaning |
|---|---|
| 0 | scan complete; nothing at or above `--fail-on` (default `high`) |
| 1 | findings at or above `--fail-on` |
| 2 | usage error - unknown command or flag, missing target, unreadable clone |
| 130 | interrupted |

## Development

```sh
pytest          # 121 tests, none of which touch the network
ruff check .
```

The test suite builds small fixture repositories on disk (`tests/conftest.py`)
and asserts on rule ids, redaction, baseline round-trips, exit codes, SARIF
shape and history recovery - including a secret deleted from the working tree
that still must be found in git history, entropy false positives, and custom
rule validation and execution.

## Contributing

New detection rules, output formats and ecosystem coverage are all welcome.
Start with [CONTRIBUTING.md](CONTRIBUTING.md) for the layout of the code and
the output contract you should not break.

- **Found a vulnerability in oracle itself?** Report it privately via
  [SECURITY.md](SECURITY.md), not a public issue.
- **Adding a rule?** Include a positive *and* a negative test case.
- **Questions and half-formed ideas** are fine in
  [Discussions](https://github.com/darkblightHQ/oracle/discussions).

If `oracle` saves you a bad afternoon, a star helps other people find it.
