Metadata-Version: 2.4
Name: agent-virtual-shell
Version: 0.1.0
Summary: A virtual shell-over-database filesystem for AI agents: a strict, append-only memory layer with a frozen shell dialect.
Project-URL: Homepage, https://github.com/camilohjimenezdev/virtual-shell
Project-URL: Repository, https://github.com/camilohjimenezdev/virtual-shell
Project-URL: Issues, https://github.com/camilohjimenezdev/virtual-shell/issues
Author-email: Camilo Jiménez <camilohjimenez@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,ai,filesystem,llm,memory,postgres,shell
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: postgres
Requires-Dist: asyncpg>=0.29; extra == 'postgres'
Provides-Extra: s3
Requires-Dist: aioboto3>=12.0; extra == 's3'
Description-Content-Type: text/markdown

# virtual-shell

> A virtual shell-over-database filesystem for AI agents.

`virtual-shell` gives an AI agent a private filesystem with a shell-style
interface (`ls`, `cat`, `grep`, `echo`) where the backend is a **database, not
a disk**. Agents are trained on millions of real terminal sessions, so a shell
is the interface they use most reliably — but a real shell executes arbitrary
code and a real disk needs per-agent compute. This package keeps the familiar
*language* and replaces everything underneath with a strict, parameterized,
append-only storage layer.

It is a **memory layer only**. It reads, writes, searches, and organizes files.
It never executes programs, accesses the network, or spawns processes.

> **Status: early development (spec phase).** The behavior is fully specified in
> `openspec/` (change: `add-virtual-shell`) and being implemented incrementally
> across milestones M1–M4. This is a Python port of the v1.0 technical
> specification; the spec is the source of truth where the two disagree.

## Documentation

- **[New here? Read the plain-language guide.](docs/explainer.md)** — what it is, why, and how it's built, for dummies.
- **[Architecture snapshot](docs/architecture.md)** — module map, data model, boundaries, extensibility seams.
- **[Technical specification v1.0](docs/spec/virtual-shell-spec-v1.0.html)** — the normative source of truth (open in a browser).
- **OpenSpec change** `add-virtual-shell` — the proposal, design, capability specs, and step-by-step tasks under [`openspec/changes/`](openspec/changes/add-virtual-shell/). Browse with `openspec show add-virtual-shell`.

## Why

- **Phrasebook, not translator.** A whitelist parser recognizes a frozen shell
  dialect and compiles it to a typed AST. There is no general shell
  interpretation and no path by which agent text reaches SQL as anything other
  than a bind parameter.
- **Append-only.** Every write is a new version row; deletes are tombstones.
  Nothing an agent does is destructive. Retention is a separate, operator-run
  compaction policy.
- **Identity is injected.** Tenant and agent IDs come from the host application;
  the dialect has no syntax for identity, and no syntax to escape a scope.
- **Teaching errors.** Every rejection tells the model what to do instead, so it
  self-corrects in one turn.
- **Two doors, one table.** Agents use the `Shell` (strings in/out). Humans and
  UIs use the `Inspector` (structured JSON, read-only). Same storage, same scope
  rules.

## Install

```bash
pip install agent-virtual-shell            # core, zero runtime dependencies
pip install "agent-virtual-shell[postgres]"   # + asyncpg convenience extra
pip install "agent-virtual-shell[s3]"          # + aioboto3 for the blob tier
```

> Installs as **`agent-virtual-shell`** on PyPI but imports as **`virtual_shell`**
> (`from virtual_shell import create_shell`). The exact name `virtual-shell` is
> blocked by PyPI's similarity guard, so the distribution carries the `agent-` prefix.

The Postgres adapter accepts any object satisfying a small async `Queryable`
protocol, so it works with `asyncpg`, a SQLAlchemy async session, or your own
wrapper — no hard dependency.

## Quick start (planned API)

```python
from virtual_shell import create_shell, Scope
from virtual_shell.adapters import MemoryAdapter

shell = create_shell(adapter=MemoryAdapter(), scope=Scope(tenant_id="t1", agent_id="a1"))

print((await shell.exec('echo "hello" > /notes/todo.md')).stdout)
# wrote 6 bytes to /notes/todo.md (v1)
print((await shell.exec("cat /notes/todo.md")).stdout)
# hello
```

`shell.exec` never raises: every input — including pathological ones — resolves
to a `ShellResult` with `ok`, `exit_code`, `stdout`, and a teaching `stderr`.

## Architecture

```
 agent (LLM)                         dashboard / human
      |                                     |
 tool call: shell(cmd)              host app's HTTP API
      |                                     |
   Shell  (string in/out)            Inspector (read-only JSON)
   tokenize → parse → interpret      tree / read / history / changes
      \                                   /
       \         scope injected by host  /
        v                               v
        StorageAdapter (interface)
          MemoryAdapter    — reference impl, tests/dev
          PostgresAdapter  — parameterized SQL, dedicated `virtual_shell` schema
          TieredAdapter    — wraps any adapter + BlobStore for large files
```

## Database

Tables live in a **dedicated Postgres schema** (`virtual_shell` by default) in
whatever database you point the adapter at — including one shared with a host
application. Apply `schema.sql` once:

```bash
psql "$DATABASE_URL" -f schema.sql
```

### Wiring up the PostgresAdapter

`PostgresAdapter` imports no database driver — it accepts any object satisfying
the async `Queryable` protocol (`query(sql, params) -> rows`, native `$1, $2, …`
placeholders). A wrapper is ~5 lines:

```python
# asyncpg
import asyncpg
from virtual_shell.adapters import PostgresAdapter

pool = await asyncpg.create_pool(dsn)

class AsyncpgQueryable:
    async def query(self, sql, params=()):
        return [dict(r) for r in await pool.fetch(sql, *params)]

adapter = PostgresAdapter(AsyncpgQueryable())            # schema="virtual_shell", table="files"
```

```python
# SQLAlchemy async (asyncpg driver, $-style params)
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession

class SqlAlchemyQueryable:
    def __init__(self, session: AsyncSession): self._s = session
    async def query(self, sql, params=()):
        # translate $1.. to :p1.. for SQLAlchemy bind params
        bound = {f"p{i}": v for i, v in enumerate(params, 1)}
        stmt = text(__import__("re").sub(r"\$(\d+)", r":p\1", sql))
        return [dict(r._mapping) for r in (await self._s.execute(stmt, bound)).fetchall()]
```

`compact(adapter, scope, RetentionPolicy(keep_versions=N))` prunes old versions
(the only operation that deletes rows; never the latest version or a latest
tombstone).

## Inspector (read-only dashboard API)

The second door: structured, read-only JSON for humans and UIs — same adapter,
same scope rules, no parser. Build one per request with the scope from *your*
auth, and wrap the four methods as HTTP endpoints:

```python
from virtual_shell import create_inspector, Scope

inspector = create_inspector(adapter, Scope(tenant_id=customer.id, agent_id=agent.id))

await inspector.tree("/")              # GET …/fs/tree?prefix=     → metadata, never content
await inspector.read(path, version)    # GET …/fs/file?path=&version= → inline record or {blob, url}
await inspector.history(path)          # GET …/fs/history?path=    → all versions, tombstones flagged
await inspector.changes(cursor)        # GET …/fs/changes?cursor=  → append-only feed (poll every 2–3 s)
```

`read` returns an `InlineRead` (full content) for inline files, or a `BlobRead`
(metadata + a presigned GET URL) for blob-tier files. `changes(None)` starts
"from now"; pass the returned cursor on the next poll to get only new events.
The Inspector is strictly read-only — any future human edit must be routed as an
ordinary adapter write (a new version).

## Large files (blob tier)

Wrap any adapter in a `TieredAdapter` to offload files over a threshold (default
100 KB) to a `BlobStore`, keeping an index (preview, type metadata, search text)
in the row. Bytes go to the blob store; *Postgres finds the file, a blob scan
finds the line.*

```python
import aioboto3
from virtual_shell import TieredAdapter, S3BlobStore, create_shell, Scope

session = aioboto3.Session()
async with session.client("s3") as s3:
    blobs = S3BlobStore(client=s3, bucket="agent-memory")
    adapter = TieredAdapter(pg_adapter, blobs)          # inline_threshold_bytes=102400
    shell = create_shell(adapter, Scope(tenant_id=customer.id, agent_id=agent.id))
```

`S3BlobStore` takes an injected client (the AWS SDK stays an optional peer).
Keys are content-addressed (`<tenant>/<sha256>`), so identical bytes are stored
once and `mv` copies no bytes. `grep` is two-stage: a prefix search finds large
files by index; `grep "term" /path/to/large/file` scans that one file's lines.

**Recommended bucket configuration** (host's responsibility, not code):
server-side encryption (SSE-S3 or SSE-KMS), a private bucket policy, and a
lifecycle rule transitioning objects to Infrequent Access at ~90 days and
Glacier at ~365 days. Content-addressed keys are immutable, so transitions are
safe.

## Development

```bash
uv sync --extra dev
uv run pytest
uv run mypy
uv run ruff check
```

## License

MIT © Camilo Jiménez
