Metadata-Version: 2.4
Name: lxcore
Version: 0.2.0
Summary: Elite lock-free binary key-value database: append-only segments, snapshots, background compaction, LRU cache, tunable durability
License: MIT License
        
        Copyright (c) 2026 Lunax
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
Project-URL: Homepage, https://github.com/ainax/lxcore
Project-URL: Repository, https://github.com/ainax/lxcore
Project-URL: Issues, https://github.com/ainax/lxcore/issues
Keywords: database,key-value,mmap,embedded,nosql,fast
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: Database :: Database Engines/Servers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# ⬡ lxcore

**Elite embedded key-value database for Python.**  
Append-only, lock-free reads, crash-safe, segment-based, with snapshots, background compaction, and tunable durability — all in pure Python, zero dependencies.

---

## At a glance

```python
from lxcore import LxDB

with LxDB("mydb") as db:
    db.set("user:1", {"name": "Alice", "score": 42})
    db.set("user:2", {"name": "Bob",   "score": 91})

    print(db.get("user:1"))           # {'name': 'Alice', 'score': 42}
    print(db.prefix_scan("user:"))    # [(b'user:1', {...}), (b'user:2', {...})]
```

---

## Design

lxcore is built around one principle: **deterministic performance with predictable I/O**.

| Property | Detail |
|---|---|
| Storage | Append-only segment files (`seg_00000001.lx`, …) |
| Index | In-memory hash map + persistent checkpoint |
| Concurrency | Single writer, unlimited lock-free readers |
| Durability | Three modes: `unsafe`, `normal`, `strict` |
| Compaction | Background daemon + manual trigger |
| Compression | Optional per-value zlib (level 1) |
| CRC | CRC-32 on every record — detects corruption on read |
| Snapshots | Immutable read views captured at a point in time |

No locks, no SQL, no query planner, no external dependencies.

---

## Installation

```bash
pip install lxcore
```

Requires Python ≥ 3.10.

---

## Quick start

### Open / close

```python
from lxcore import LxDB

db = LxDB("mydb")        # creates mydb.lxdb/ if absent
db.close()

# context manager (preferred)
with LxDB("mydb") as db:
    ...
```

### Write

```python
db.set("key", "value")           # typed upsert (any JSON-compatible value)
db.set("key", 42)
db.set("key", {"a": [1, 2, 3]})
db.set("key", b"\xde\xad")       # raw bytes

db.create("newkey", "value")     # raises KeyError if key exists
db.write("key", b"rawbytes")     # raw upsert
```

### Read

```python
val = db.get("key")              # typed read — returns Python object
raw = db.read("key")             # raw bytes
ok  = db.exists("key")           # bool
```

### Update / Delete

```python
db.update("key", "new_value")    # raises KeyError if missing
db.delete("key")                 # returns True/False
```

### Batch operations

```python
db.batch_set({"a": 1, "b": 2, "c": [3, 4]})   # atomic group write
db.batch_create({"x": 10, "y": 20})            # all-or-nothing create
db.batch_delete(["a", "b"])                     # returns count deleted
```

---

## Namespaces

Namespaces are zero-cost logical sub-spaces — keys are stored as `<ns>\x00<key>`.

```python
users = db.ns("users")
posts = db.ns("posts")

users.set("alice", {"age": 30})
posts.set("p1",    {"title": "Hello"})

users.get("alice")   # {'age': 30}
posts.get("alice")   # None — isolated

users.keys()         # ['alice']
users.count()        # 1
users.clear()        # delete all keys in namespace
```

---

## Query

```python
# All keys starting with a prefix
db.prefix_scan("user:")                   # [(key_bytes, value), ...]

# Lexicographic range
db.range_scan("user:0050", "user:0099")

# Namespace sub-prefix
ns = db.ns("events")
ns.prefix_scan("2024:")
```

---

## Snapshots

Snapshots capture an immutable read view. Writes after the snapshot are invisible to it.

```python
snap = db.snapshot()
print(len(snap))            # key count at capture time

val = snap.read(b"key")     # raw bytes
for k, v in snap.items():   # iterate (yields key_bytes, raw_bytes)
    ...

# Snapshot is lightweight — just a dict copy + shallow segment refs.
# Release it before calling compact().
del snap
```

---

## Durability modes

```python
from lxcore import DURABILITY_UNSAFE, DURABILITY_NORMAL, DURABILITY_STRICT

db = LxDB("mydb", durability=DURABILITY_UNSAFE)   # no fsync — max throughput
db = LxDB("mydb", durability=DURABILITY_NORMAL)   # periodic fsync (default, every 5s)
db = LxDB("mydb", durability=DURABILITY_STRICT)   # fsync on every write
```

| Mode | Throughput | Durability guarantee |
|---|---|---|
| `unsafe` | ~340 K ops/s | data may be lost on power failure |
| `normal` | ~325 K ops/s | at most 5s of writes lost |
| `strict` | ~470 ops/s   | every write survives power failure |

---

## Compression

```python
db = LxDB("mydb", compress=True)   # zlib level-1, transparent on read
```

Values are only compressed if the compressed form is smaller. Incompressible values are stored raw. No change to the API.

---

## Compaction

Dead records (tombstones, overwritten values) accumulate in segments. Compaction rewrites all live records into a single segment and drops the rest.

```python
# Manual
db.compact()

# Automatic (default: runs when dead_ratio > 40%)
db = LxDB("mydb", auto_compact=True, compact_threshold=0.4, compact_interval=30.0)
```

---

## Checkpoints

On close, lxcore saves a binary index checkpoint (`checkpoint.lxidx`). On the next open, the checkpoint is loaded and only records newer than it are replayed — making startup fast even on large databases.

```python
db.checkpoint()   # force checkpoint at any time
```

---

## Metrics & introspection

```python
db.metrics()
# {
#   'key_count': 50000, 'segment_count': 3,
#   'total_bytes': 8_400_000, 'dead_bytes': 210_000, 'dead_ratio': 0.025,
#   'writes': 50000, 'reads': 12000, 'deletes': 500,
#   'ops_per_sec': 183000.0, 'cache_hit_ratio': 0.94,
#   'compact_count': 1, 'secs_since_compact': 12.4,
#   ...
# }

db.info()   # full dump including segment map, writer_id, durability mode
```

---

## LRU value cache

```python
db = LxDB("mydb", max_cache=4096)   # cache last 4096 values (default: 2048)
db = LxDB("mydb", max_cache=0)      # disable cache
```

---

## Segment configuration

```python
db = LxDB("mydb", max_segment_size=128 * 1024 * 1024)   # 128 MB per segment (default: 64 MB)
```

When a segment reaches `max_segment_size`, a new one is created automatically. Compaction collapses all segments back to one.

---

## Migration from v1

If you have a legacy single-file `.lxdb` database (lxcore < 1.0):

```python
from lxcore import migrate_v1

engine = migrate_v1("/path/to/old.lxdb", "/path/to/new")
engine.close()
```

---

## Database viewer

lxcore ships a full-featured web viewer built on Flask:

```bash
python viewer/app.py /path/to/mydb --port 5000
```

Open `http://localhost:5000` in your browser. Features:

- **Browse** — paginated key/value table with search, prefix filter, inline edit, delete
- **Segments** — visual size bars per segment file
- **Metrics** — live dashboard (key count, ops/sec, dead ratio, cache hit rate, …)
- **Raw Scanner** — low-level record-by-record view with CRC status
- **Tools** — compact, checkpoint, flush, v1 migration — all one click

---

## Stress test

```bash
python tests/stress.py             # 50 000 ops (default)
python tests/stress.py 200000      # custom scale
```

Covers 18 suites: raw CRUD, typed API, batch ops, namespaces, prefix/range scans, snapshots, compaction, checkpoint reload, compression, 3 durability modes, 8-thread concurrent reads, concurrent write+read, LRU cache, metrics, CRC crash consistency, segment rotation, v1 migration, and 1 MB value roundtrip.

---

## Architecture reference

```
mydb.lxdb/
├── seg_00000001.lx       # append-only segment (14-byte header + records)
├── seg_00000002.lx       # created when previous segment fills
└── checkpoint.lxidx      # persistent index (binary, ~23B + 18B per key)
```

### Record format (v2)

```
STATUS (1B) FLAGS (1B) SEQ (8B) KEY_LEN (2B) VAL_LEN (4B)  →  16 bytes header
KEY   (KEY_LEN bytes)
VALUE (VAL_LEN bytes)
CRC32 (4 bytes, over all of the above)
```

`FLAGS` bit-0 = value is zlib-compressed.  
`SEQ` is a monotonic uint64 writer sequence number used for checkpoint replay.

---

## Benchmark (single machine, Python 3.12)

| Operation | Throughput |
|---|---|
| batch_delete | 582 K ops/s |
| raw_delete | 390 K ops/s |
| full_scan_reload | 430 K ops/s |
| checkpoint_reload | 342 K ops/s |
| raw_create | 284 K ops/s |
| raw_read | 256 K ops/s |
| batch_set | 212 K ops/s |
| typed_set | 124 K ops/s |
| typed_get | 162 K ops/s |
| compact (50 K records) | 329 K ops/s |
| concurrent reads (8 threads) | 17 K ops/s |

---

## Changelog

### 0.2.0
- Segment-based append-only storage (replaces single file)
- Persistent index checkpoints for fast startup
- Background compactor daemon
- Snapshot isolation (immutable read views)
- LRU value cache
- Tunable durability modes (`unsafe` / `normal` / `strict`)
- Optional zlib value compression
- Prefix and range scan queries
- Sequence numbers on every record
- Runtime metrics & introspection
- v1 single-file migration utility
- Web-based database viewer (Flask)
- Fast native int/float serialization paths

### 0.1.0
- Initial release

---

## License

MIT — see `LICENSE`.
