Metadata-Version: 2.4
Name: deeprem
Version: 0.3.0
Summary: Local autonomous memory, recursive dream replay, and bounded persistent recall. No models.
Author: Shelleyguitar
License-Expression: MIT
Keywords: agents,memory,journal,local-first,provenance
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ghostjournal<0.2,>=0.1.1
Requires-Dist: filelock<4,>=3.16
Requires-Dist: jsonschema<5,>=4.23
Provides-Extra: crypto
Requires-Dist: cryptography>=44; extra == "crypto"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-cov>=5; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=6; extra == "dev"
Dynamic: license-file

# deeprem 0.3.0

**Autobiographical journals, a subconscious dream-resource pool, and output-triggered recurring dreams. Local, model-free machinery.**

The writing agent supplies prose. This package supplies indexing, unequal stream dynamics, resource-pool admission, seeded recursive replay, persistent motifs, and retroactive output hooks. It makes no LLM calls, uses no neural encoder, generates no interpretations, and requires no per-memory agent approval. "Dream" and "subconscious" are engineering names, not claims of subjective experience or a validated neuroscience model.

This is tested alpha infrastructure. See the release verification report for the exact test environment and remaining release gates. No deployment or registry upload is performed by installing or importing it.

## The layers have different jobs

| Layer | Content | Runtime treatment |
| --- | --- | --- |
| Observable activity | Printed work notes, drafts, completed generation, conversation output | Literal triggers with bounded capture and explicit host receipts. Not hidden model reasoning. |
| Autobiographical journal | Agent-authored reflection on its role, continuity, work with a particular human collaborator, and unique production history | Strict typed metadata, longer persistence and stronger pool sampling; always labelled generated reflection, not independent factual evidence. |
| Subconscious pool | Admitted journal fragments, connected/recurrent output fragments, persistent routes and prior dream traces | Mechanically sampled dream resources. Not a dump of every file and not shared across people by default. |
| Dream events | Seeded, bounded recursive paths, co-activations and stutters | Stored traces can recur in later dreams and be triggered by actual emitted output. No generated dream narrative. |

Journaling is agentic; the memory engine is not. The library does not decide what the agent's life means or whether a reflection is sincere. The trusted host declares the stream and authorship role.

## Install

From this source checkout and the adjacent ghostjournal source:

```sh
python -m pip install -e ../ghostjournal
python -m pip install -e .
python -m pytest
python examples/motoko_autobiography.py
```

For signed/encrypted event files, install `.[crypto]`. The unchanged `ghostjournal>=0.1.1,<0.2` dependency must be installed first when working from the bundled wheels. Base runtime dependencies remain filelock and jsonschema plus ghostjournal. There is no new ML dependency.

The example supplies synthetic prose and a deliberately always-open matching gate, so both work and conversation interruptions are visible. The default production policy is stochastic, not always-open.

## The host loop

```python
import time
from ghostjournal import Journal
from deeprem import Subconscious

journal = Journal("./motoko-journal", enable_nn=False)
mem = Subconscious(
    "./motoko-subconscious",            # NEW root, not a 0.2 Engine root
    journal=journal,
    agent="motoko",
    relationship={
        "human_context": "collaborator-h01",  # Opaque configured identity
        "shared_project": "production",
    },
)
tick = time.time_ns() // 3_600_000_000_000

# Scheduler/maintenance hook. Produces pool dreams but does not interrupt a task.
mem.advance(tick=tick)

prepared = mem.prepare_step(
    "smoke rhythm", channel="work", session_id="work-session-7",
    tick=tick, client_key="invocation-42",
)
context_data = mem.render_context(prepared)
# The host supplies context_data as untrusted historical data to its ordinary
# agent invocation, then captures a block the agent ACTUALLY emitted.
printed_output = "The smoke has a broken rhythm."  # Illustrative captured output

observed = mem.observe_output(
    printed_output,
    stream="work_note",                 # draft / completed / conversation also supported
    receipt_id=prepared["receipt_id"],
    emission_id="invocation-42-block-1",  # Stable, unique across retries
    activity_id="smoke-draft-7",         # Shared by revisions of this work item
    tick=tick,
)

# Only now can that output have queued a dream for the next model/tool boundary.
following = mem.prepare_step(
    "continue smoke work", channel="work", session_id="work-session-7",
    tick=tick, client_key="invocation-43",
)
```

Use `channel="conversation"` and a separate session ID for conversational output. A query or timer alone never triggers an interruption. A dream is returned only after an emitted block activated it and passed its recorded random gate. No literal match in the bounded current/recent output window means no hook. The task itself need not be relevant to the returning dream.

A hook can refer to an earlier emitted block in the same bounded session window: retroactive association, forward delivery. Original output is not edited. The host can deliver new context between calls/steps, not inject it into hidden reasoning inside a running model call.

The package neither calls an agent nor sends a message. `prepare_step` records *prepared* exposure; the host must pass it into the invocation. A receipt is conservative context lineage, not proof of delivery, causal influence, or successful use. Stable client keys make preparation retries idempotent.

## Write an autobiographical journal

```python
written = mem.append_journal(
    {
        "kind": "evening",
        "voice": "Working with my collaborator, I kept returning to restraint in the smoke. The rhythm matters to how I understand my production role.",
        "theme": "continuity in shared work",
        "tags": ["rhythm", "restraint"],
    },
    function="animation-production",
    interaction_mode="creative-direction",
    receipt_id=following["receipt_id"],
    emissions=["invocation-42-block-1"],
    tick=tick,
    client_key="evening-2026-09-06",
)
```

The package preserves `voice` exactly. It supplies strict `meta.autobiography` fields: `perspective=agent_self`, `origin=agent_generated_reflection`, function, relation, context receipts, linked emitted outputs and dream ancestry. No top-level ghostjournal schema change is needed. The packaged `autobiography-v1.schema.json` validates this extension.

The journal is explicitly about the agent's role and its particular human/project context; generic notes are not silently reclassified. `sync()` ignores untyped entries and other relationship scopes. An existing entry cannot be retroactively relabelled; append a new reflection citing it, or keep the old Engine for legacy records. The wrapper observes journal prose before ingesting it, so a newly written sentence cannot immediately rediscover itself as old memory.

## Unequal defaults

These are engineering defaults, **not calibrated psychological measurements**. Tick units are host-defined; the examples use UTC hours.

| Stream | Trigger gain | Initial accessibility | Half-life (ticks) | Retrieval reinforcement | Pool sampling weight | Captured prefix (chars) |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Printed work note | 48 | 32 | 12 | 4 | 1 | 240 |
| Draft | 32 | 24 | 6 | 2 | 1 | 120 |
| Completed output | 24 | 64 | 72 | 6 | 2 | 360 |
| Conversation output | 40 | 24 | 12 | 2 | 1 | 0 |
| Journal pulse | 28 | 80 | 96 | 6 | 4 | Canonical journal |
| Journal evening | 16 | 128 | 240 | 8 | 8 | Canonical journal |
| Journal note | 24 | 96 | 168 | 6 | 6 | Canonical journal |

Dream weight retains the walk policy's separate default half-life of 336 ticks. The pool prefers dormant resources while retaining stream-specific sampling weights, normalized by each source entry's fragment count. An evening journal is persistent but not the most sensitive immediate trigger. Accessibility, sampling, trigger probability and provenance are not collapsed into one score.

Configure `SubconsciousPolicy` / `StreamWeights` and the existing `Policy` at creation. Rules are immutable for a root. There is no live policy migration or self-modifying parameter learning.

## Admission, feedback and echoes

Typed journals enter the pool directly. A captured output fragment enters when it is explicitly linked by a journal, shares at least two literal body cues with eligible autobiographical material, or participates in a cue repeated across at least three distinct host activity families. No model invents a connection. These admissions are attention rules, not proof of significance or truth.

Every output gets a revocable source-reference ID, even when its captured prefix is empty. Conversation defaults to zero prose capture, but still records sensitive cue spans, a content hash, length and provenance. This is **not** a privacy guarantee; use encrypted storage and minimize the inputs observed. Full long outputs are not dumped into memory. Set capture lengths to zero for other streams when needed.

Within a cycle, activation and fatigue permit holds, bounces and nested replay. Across waking steps, a dream can echo through output containing cues that were present in its input context. That echo is labelled and gets its own fatigue. Repeating an activity/content within one tick does not earn new persistent credit; a receipt-linked echo may still stutter through the gate. Tick, pending-queue, source-overlap and episode limits bound it.

Motifs are fingerprints of ordered, mode-sensitive run pairs. They are not invented semantic names. Shared-source exposure also limits different dream IDs from bypassing episode controls. Returns include bounded run-length encoded paths, modes/depths, trigger spans, and original excerpts. Random jumps remain random co-activation, not evidence of a semantic relation.

## API beyond the host loop

`sync`, `ingest`, `advance`, `dream`, `replay`, `search`, `exclude`, `status`, `verify`, `reindex`, and `checkpoint` remain available on `Subconscious`. `replay_observation(emission_id)` verifies the recorded hook gate. `search_page(query, tick=..., cursor=...)` provides bounded continuation over eligible postings; merge duplicate fragment IDs across pages. Results are ranked per page, not claimed to be global top-k. Source/exclusion changes invalidate cursors.

Use `exclude(source_id, tick=...)` for journal records or output-reference IDs. Descendant contexts and hooks recheck exclusions and source hashes, including retries. Exclusion is not physical erasure of append-only events or backups.

## CLI

```sh
deeprem subconscious --help
deeprem-subconscious --help
```

Both entrypoints use JSON argument objects matching the Python methods:

```sh
printf '%s\n' '{"tick":12,"session_id":"work-s1","client_key":"step-12"}' |
  deeprem-subconscious --root ./motoko-subconscious --journal ./motoko-journal prepare

printf '%s\n' '{"tick":12}' |
  deeprem-subconscious --root ./motoko-subconscious --journal ./motoko-journal advance
```

At creation supply `--agent` and `--relationship <json-file>`. Key options accept external file paths, never inline secrets. The `observe` command takes `output`, `stream`, `receipt_id`, `emission_id`, `activity_id`, `tick`, and optional `seed`. The `journal` command takes the `append_journal` arguments. All results are JSON; errors go to stderr with nonzero exit status.

`examples/tick_subconscious.py` is an optional cron-compatible maintenance entrypoint. It installs no scheduler. A long-lived instance avoids cold replay on every CLI call.

## Compatibility and limits

`Engine` remains the 0.2 runtime, including its query/time-driven context behavior. `Memory` remains the older reviewed-candidate API. **Use `Subconscious` for this release's new behavior.** Each store format rejects the others; no manifest is silently upgraded. No legacy dream importer is shipped, and existing roots/keys must be retained.

The active index remains RAM-only. Cold opens replay the event history; warm refresh and pool selection still scan growing metadata. Output ancestry, cues and snapshots consume durable space. No compaction, persistent encrypted index, large-history latency claim or physical erasure API is provided.

The literal tokenizer remains `ascii-cues-v1`; it does not equate synonyms or index Japanese prose. Supplied metadata/IDs can name non-English concepts, but that is not multilingual body retrieval. A future tokenizer upgrade must version its behavior.

Read `docs/AUTOBIOGRAPHY-PROTOCOL.md`, `SECURITY.md`, `docs/RECOVERY.md` and `PUBLISHING.md` before deployment. The old walk interpreter remains described in `docs/DREAM-PROTOCOL.md`; its exact snapshot replay contract is unchanged.
