Metadata-Version: 2.5
Name: warp-server
Version: 0.4.0
Summary: WARP signature server API
Requires-Python: >=3.12
Requires-Dist: aiosqlite>=0.20
Requires-Dist: click>=8.1
Requires-Dist: fastapi>=0.115
Requires-Dist: flatbuffers>=25.12.19
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: pydantic>=2.0
Requires-Dist: python-multipart>=0.0.9
Requires-Dist: sqlalchemy[asyncio]>=2.0
Requires-Dist: uvicorn[standard]>=0.30
Description-Content-Type: text/markdown

# warp-server

FastAPI implementation of the WARP signature server API.

## Development

```bash
git submodule update --init  # fetch warp/ schemas
uv sync
uv run warp-server        # start dev server on :8000
uv run pytest              # run tests
```

## Installation with `uvx`

If installed via `uvx`, use `--from warp-server` to run either command:

```bash
uvx --from warp-server warp-server           # start the server
uvx --from warp-server warp-ctl bootstrap    # manage users/sources/keys
```

### Regenerating FlatBuffer bindings

The Python FlatBuffer bindings are generated from the `.fbs` schemas in `warp/`. To regenerate:

```bash
brew install flatbuffers   # if not already installed
flatc --python -o src/warp_server/gen_flatbuffers warp/*.fbs
```

## Bootstrap: Create a User with an API Key

The server uses Bearer token auth via API keys. Use the `warp-ctl` CLI to manage users and keys.

### Quick bootstrap (admin + key in one step)

```bash
uv run warp-ctl bootstrap
# Created admin user id=1 username=admin
# API key: <key>
```

Options: `--email`, `--username` to customize the admin account.

### Managing users

```bash
# create a regular user
uv run warp-ctl user create --email alice@example.com --username alice

# create an admin
uv run warp-ctl user create --email ops@example.com --username ops --role Admin

# list all users
uv run warp-ctl user list

# delete a user
uv run warp-ctl user delete 2
```

### Managing API keys

```bash
# create a key for user id 1
uv run warp-ctl key create --user-id 1 --name dev-key

# list all keys (or filter by --user-id)
uv run warp-ctl key list
uv run warp-ctl key list --user-id 1

# revoke a key
uv run warp-ctl key revoke 3
```

### Managing sources

```bash
# create a source owned by user 1
uv run warp-ctl source create --name my-signatures --user-id 1

# list sources
uv run warp-ctl source list
```

### Ingesting `.warp` files directly

For large files or batch imports, use `warp-ctl ingest` to bypass the HTTP server
and insert directly into the SQLite database:

```bash
# ingest a single file (creates the source if it doesn't exist)
uv run warp-ctl ingest /path/to/file.warp --source my-signatures --user-id 1

# ingest multiple files at once
uv run warp-ctl ingest *.warp --source my-signatures --user-id 1

# with optional commit metadata
uv run warp-ctl ingest file.warp --source my-signatures --user-id 1 \
  --name "libc v2.38" --description "glibc signatures"
```

### Using the API key

```bash
# verify auth
curl -H "Authorization: Bearer <key>" http://localhost:8000/api/v1/users/me

# create a source via the API
curl -X POST http://localhost:8000/api/v1/sources \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-source", "user_ids": []}'
```

## Web UI

A built-in web interface is available at `/web-ui` for managing the server:

```
http://localhost:8000/web-ui
```

Log in with your username and API key. The UI lets you:

- Browse and search symbols
- Create and delete sources
- Manage users (admin only)
- Create API keys
- Browse functions and commits
- Upload, browse, and download BNDBs

## BNDB Sharing

The server supports uploading and downloading Binary Ninja Database (`.bndb`) files.
Files are stored on disk as gzip-compressed blobs in a configurable directory, keeping
the SQLite metadata database lightweight.

Any authenticated user can upload or download BNDBs. Only the uploader (or an admin)
can delete them. The API returns `can_delete` for the authenticated user; the
server enforces the same rule on deletion, including for uploads in progress.
Uploading another BNDB with the same name as one owned by the same user updates that
record while keeping its UUID stable.

| BNDB action | User | Admin |
|-------------|------|-------|
| List and download | All BNDBs | All BNDBs |
| Upload and resume | Own uploads | Own uploads |
| Delete a BNDB | Own BNDBs | Any BNDB |
| Cancel an upload | Own uploads | Any upload |

### API

```bash
# upload a BNDB
curl -X POST http://localhost:8000/api/v1/bndbs \
  -H "Authorization: Bearer <key>" \
  -F "file=@firmware.bndb" \
  -F "name=firmware.bndb" \
  -F "description=Extracted from router firmware"

# list / search BNDBs
curl -X POST http://localhost:8000/api/v1/bndbs/query \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{"name": "firmware", "limit": 20, "page": 1}'

# get metadata
curl -H "Authorization: Bearer <key>" \
  http://localhost:8000/api/v1/bndbs/<uuid>

# download
curl -H "Authorization: Bearer <key>" -o output.bndb \
  http://localhost:8000/api/v1/bndbs/<uuid>/download

# delete (owner or admin)
curl -X DELETE -H "Authorization: Bearer <key>" \
  http://localhost:8000/api/v1/bndbs/<uuid>
```

For larger files, clients use `POST /api/v1/bndbs/upload/init` with `name`,
`total_chunks`, `size_bytes`, `chunk_size`, and optionally the full-file `sha256`.
Each `PUT /api/v1/bndbs/upload/<upload_id>/<index>` includes a multipart `file`
and an `X-Chunk-SHA256` header. `GET /api/v1/bndbs/upload/<upload_id>` returns
received indices so interrupted uploads can resume. `POST .../complete` validates
the chunks and publishes the BNDB. Sessions survive server restarts and expire
after `WARP_UPLOAD_TTL_MINUTES` without activity. Clients can cancel a session
with `DELETE /api/v1/bndbs/upload/<upload_id>`.
The server also sweeps expired sessions and old orphan files hourly.

BNDBs can also be managed from the **BNDBs** tab in the web UI.

### Storage

BNDB files are stored as `<uuid>.<sha256>.bndb.gz` in the directory configured by
`WARP_BNDB_STORAGE_DIR` (default `./bndb_storage`). The directory is created
automatically on server startup. Compression is transparent — clients always
upload and download raw `.bndb` files. Existing `<uuid>.bndb.gz` files remain
readable and are replaced on the next upload with the same name.

## Configuration

The server is configured via environment variables, all prefixed with `WARP_`.

| Variable | Default | Description |
|----------|---------|-------------|
| `WARP_DEBUG` | `false` | Enable debug mode: exposes Swagger UI at `/docs`. |
| `WARP_DATABASE_URL` | `sqlite+aiosqlite:///./warp.db` | SQLAlchemy async database URL. |
| `WARP_HOST` | `127.0.0.1` | Bind address. |
| `WARP_PORT` | `8000` | Listen port. |
| `WARP_LOG_LEVEL` | `info` | Uvicorn log level (`debug`, `info`, `warning`, `error`). |
| `WARP_RELOAD` | `false` | Enable auto-reload on file changes (dev only). |
| `WARP_BNDB_STORAGE_DIR` | `./bndb_storage` | Directory for uploaded BNDB files (created automatically). |
| `WARP_MAX_BNDB_BYTES` | `8589934592` | Maximum raw size of one BNDB. |
| `WARP_MAX_BNDB_CHUNK_BYTES` | `8388608` | Maximum raw size of an upload chunk. |
| `WARP_MAX_BNDB_USER_BYTES` | `21474836480` | Maximum total raw BNDB size per user. |
| `WARP_CORS_ORIGINS` | `[]` | JSON list of allowed CORS origins, e.g. `'["https://app.example.com"]'`. |
| `WARP_TRUSTED_PROXIES` | `[]` | JSON list of trusted reverse-proxy IPs allowed to set `X-Forwarded-For`. |
| `WARP_MAX_LOGIN_ATTEMPTS` | `5` | Failed attempts before an IP or user account is locked. |
| `WARP_LOCKOUT_DURATION_MINUTES` | `30` | Minutes before a locked IP or account is automatically unlocked. |
| `WARP_SESSION_TOKEN_TTL_HOURS` | `168` | Lifetime of the token issued by `POST /api/v1/auth/login`. |
| `WARP_MAX_REQUEST_BODY_BYTES` | `536870912` | Reject requests whose `Content-Length` exceeds this. `0` disables. |
| `WARP_MAX_DECOMPRESSED_BYTES` | `536870912` | Cap on a gzip-decompressed request body. |
| `WARP_MAX_WARP_CHUNK_BYTES` | `268435456` | Cap on a single decompressed WARP chunk. |
| `WARP_MAX_UPLOAD_CHUNKS` | `100000` | Most chunks a single chunked upload may declare. |
| `WARP_MAX_ACTIVE_UPLOADS` | `1000` | Most concurrent in-flight chunked uploads, server-wide. |
| `WARP_UPLOAD_TTL_MINUTES` | `60` | Abandoned chunked uploads are deleted after this long. |
| `WARP_MAX_EXPORT_ROWS` | `50000` | Cap on rows loaded by bulk queries with no page size (e.g. `format=flatbuffer`). |
| `WARP_MAX_PAGE_SIZE` | `200` | Largest page size a paginated query may request. |

> **Warning:** `WARP_DEBUG=true` exposes Swagger UI at `/docs` and ReDoc at `/redoc`. Never enable it in production.

> **Note:** The IP lockout applies only to unrecognized API keys. A locked IP can still
> authenticate with a valid key, so bad tokens cannot lock out other clients sharing that
> address. Set `WARP_TRUSTED_PROXIES` when running behind a reverse proxy, otherwise every
> client is accounted for under the proxy's address.

## Docker

```bash
docker build -t warp-server .
docker run -p 8000:8000 -v warp-data:/data warp-server
```

The Docker image stores both SQLite and BNDB files under `/data`. When upgrading
an older container, copy its `/app/bndb_storage` directory into the mounted
volume at `/data/bndb_storage` **before removing that container**. Older images
kept BNDB files outside the volume.

Run `warp-ctl` commands against the same volume:

```bash
docker run -v warp-data:/data warp-server sh -c "warp-ctl bootstrap"
```

Environment variables (`WARP_DATABASE_URL`, `WARP_CORS_ORIGINS`, etc.) can be
passed with `-e`:

```bash
docker run -p 8000:8000 -v warp-data:/data \
  -e WARP_CORS_ORIGINS='["http://localhost:3000"]' \
  warp-server
```

## Migrations

A single `migrate` command runs all pending database migrations:

```bash
# dry-run — shows what would change
uv run warp-ctl migrate

# apply all pending migrations
uv run warp-ctl migrate --apply
```

For the `shared-types` upgrade, stop the server and back up the database before applying
the migration, then restart with the new code. The migration rebuilds the types table
atomically while preserving type GUIDs and blobs. It recovers upload memberships from
the original owner and surviving function references. Old duplicate uploads containing
only types cannot be recovered as separate memberships because earlier versions did not
store that provenance.

Migrations are idempotent and safe to re-run. Current migrations:

- **hash-keys** — hashes any plaintext API keys (SHA-256). After migration, previously issued raw keys become invalid; create new ones via `warp-ctl key create` or the web UI.
- **lockout-columns** — adds `locked`, `failed_login_count`, and `locked_at` columns to the `users` table for account lockout support.
- **purge-auto-names** — deletes functions, their comments/constraints, and symbols with
  auto-generated names (`sub_`, `nullsub`, `outlined_sub_`, `_OUTLINED_FUNCTION`), including
  any number of leading `j_` prefixes such as `j_j_sub_1fc07f124`. Uses the same filter as
  ingestion and can be re-run to clean these names even if an older version already ran.
  Named thunks such as `j_j_memcpy` are retained. Preview with `warp-ctl migrate`, then
  apply with `warp-ctl migrate --apply`.
- **add-indexes** — creates the query indexes declared on the models but missing from
  older databases (function GUID/source/commit/symbol, symbol name, and the cascade
  targets). New databases get them automatically; existing ones need this migration.
  Expect it to take a while and to grow the database file on a large deployment.
- **shared-types** — separates globally deduplicated type content from commit/source
  membership. Deleting an upload preserves types used by other uploads; deleting the
  final membership removes unreferenced content. Type metadata reports the earliest
  surviving upload, and source-filtered search finds types shared with that source.
