Metadata-Version: 2.4
Name: reminis
Version: 0.1.1
Summary: Store LLM weights in a SQLite database. Query, diff, rollback, and merge — for free.
Project-URL: Homepage, https://github.com/ronxldwilson/reminis
Project-URL: Repository, https://github.com/ronxldwilson/reminis
Author-email: Ronald Wilson <ronaldwilson00000@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: fine-tuning,gguf,llm,machine-learning,sqlite,weights
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: gguf>=0.16.0
Requires-Dist: numpy>=1.24.0
Description-Content-Type: text/markdown

# reminis

**Your model's weights are just data. Store them in a database.**

`reminis` converts any GGUF model into a SQLite database where every tensor becomes a queryable, versionable, diffable row. Convert back to GGUF when you're done. Lossless. Fast.

```bash
pip install reminis
```

## Why

The ML world treats model weights as opaque files. You save the whole thing, load the whole thing, and if something goes wrong, you retrain from scratch.

Once weights are in a database, you get — for free — everything that 40 years of database engineering has built: queries, rollback, diffs, branching, merging, audit logs, access control, replication.

## Quick Start

```bash
# Convert a GGUF model to SQLite
reminis convert model.gguf

# Inspect what's inside
reminis info model.db

# Convert back to GGUF
reminis export model.db -o model_restored.gguf
```

## Verified Results

SHA256-verified lossless round-trip across 9 model variants covering 13 quantization types:

```
Model                               Dtypes                Tensors  GGUF MB    DB MB    RT MB  Conv(s)   Exp(s)   Result
---------------------------------------------------------------------------------------------------------------------------------------
SmolLM-135M.IQ3_M                   F32,IQ3_S,IQ4_NL,Q4_K       272     86.0     86.1     84.3     1.73     0.05     PASS
SmolLM-135M.IQ4_XS                  F32,IQ4_NL,IQ4_XS,Q5_K      272     87.1     87.1     85.4     1.81     0.06     PASS
SmolLM-135M.Q2_K                    F32,IQ4_NL,Q3_K,Q8_0         272     84.1     84.2     82.4     1.64     0.05     PASS
SmolLM-135M.Q3_K_M                  F32,IQ4_NL,Q4_K,Q5_0         272     89.2     89.3     87.5     1.67     0.06     PASS
SmolLM-135M.Q4_K_M                  F32,Q4_K,Q5_0,Q6_K,Q8_0      272    100.6    100.7     98.9     1.71     0.05     PASS
SmolLM-135M.Q5_K_M                  F32,Q5_1,Q5_K,Q6_K,Q8_0      272    106.9    106.9    105.2     1.83     0.07     PASS
SmolLM-135M.Q6_K                    F32,Q6_K,Q8_0                 272    132.0    132.0    130.3     1.84     0.07     PASS
SmolLM-135M.Q8_0                    F32,Q8_0                      272    138.1    138.2    136.4     1.92     0.08     PASS
SmolLM-135M.f16                     F16,F32                       272    258.3    258.4    256.7     2.39     0.14     PASS
---------------------------------------------------------------------------------------------------------------------------------------
9/9 models passed SHA256-verified lossless round-trip
ALL TESTS PASSED - every tensor in every model matches byte-for-byte
```

Every tensor in every model was hashed with SHA256 before and after the round-trip. Zero data loss.

## What's in the Database

```
$ reminis info model.db

Database: model.db (258.4 MB)
  general.name: SmolLM 135M
  general.architecture: llama
  Metadata fields: 40
  Tensors: 272
  Parameters: 134,515,008
  Weight data: 256.6 MB

  Dtype breakdown:
    F16          211 tensors     256.5 MB
    F32           61 tensors       0.1 MB
```

Every tensor gets its own row with full metadata:

| Column | Description |
|--------|-------------|
| `name` | Tensor path (e.g. `blk.5.attn_q.weight`) |
| `shape` | Dimensions as JSON (e.g. `[576, 576]`) |
| `dtype` | Data type (`F16`, `F32`, `Q4_K`, `Q8_0`, etc.) |
| `n_elements` | Number of parameters |
| `n_bytes` | Storage size in bytes |
| `data` | Raw weight data as BLOB |

All model metadata (architecture, context length, vocab size, etc.) is stored in a `model_meta` table.

## Query Your Model

Once in SQLite, you can query weights like any database:

```sql
-- Largest tensors by parameter count
SELECT name, n_elements, n_bytes / 1024 / 1024 as mb
FROM tensors ORDER BY n_elements DESC LIMIT 5;

-- All attention weights in layer 5
SELECT name, shape, dtype FROM tensors
WHERE name LIKE 'blk.5.attn%';

-- Total size by dtype
SELECT dtype, COUNT(*) as count, SUM(n_bytes) / 1024 / 1024 as total_mb
FROM tensors GROUP BY dtype;

-- Model architecture
SELECT key, value FROM model_meta
WHERE key LIKE '%context_length%' OR key LIKE '%block_count%';
```

## Python API

```python
from reminis import gguf_to_sqlite, sqlite_to_gguf

# Convert
db_path = gguf_to_sqlite("model.gguf")

# Query with standard sqlite3
import sqlite3
conn = sqlite3.connect(db_path)
for name, n_elements in conn.execute(
    "SELECT name, n_elements FROM tensors ORDER BY n_elements DESC LIMIT 5"
):
    print(f"{name}: {n_elements:,} params")

# Export back
sqlite_to_gguf(db_path, "model_restored.gguf")
```

## Supported Formats

All GGUF tensor types are supported and verified, including:

| Type | Description | Verified |
|------|-------------|----------|
| F32, F16 | Full precision | Yes |
| Q4_K, Q5_K, Q6_K | K-quants (4/5/6 bit) | Yes |
| Q8_0 | 8-bit quantized | Yes |
| Q3_K, Q5_0, Q5_1 | Other quants | Yes |
| IQ3_S, IQ4_NL, IQ4_XS | Importance-weighted quants | Yes |

## Roadmap

- [x] Publish to PyPI
- [x] GGUF to SQLite converter (lossless, verified across 13 quant types)
- [x] SQLite to GGUF back-converter (lossless, byte-perfect)
- [x] SHA256 verification test suite
- [ ] Fine-tune tracking with edit logs
- [ ] Surgical rollback of bad training steps
- [ ] Weight diffing between model versions
- [ ] Delta-based model distribution (weight migrations)
- [ ] Model merging via SQL operations
- [ ] Inference from database-stored weights
- [ ] Unsloth integration

## License

MIT
