Metadata-Version: 2.3
Name: lakefront
Version: 0.8.1
Summary: Lakehouse Observability Platform
Keywords: lakehouse,duckdb,tui,data,parquet,s3,terminal,observability
Author: Piotr Nowakowski
Author-email: Piotr Nowakowski <piotrnow@proton.me>
License: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Terminals
Requires-Dist: duckdb>=1.0.0,<2.0.0
Requires-Dist: pandas>=2.3.3
Requires-Dist: pyarrow>=23.0.1
Requires-Dist: pydantic-settings[toml]>=2.13.1
Requires-Dist: tomli-w>=1.2.0
Requires-Dist: textual<=9.0.0
Requires-Dist: typer>=0.24,<1.0.0
Requires-Dist: loguru>=0.7.3
Requires-Dist: httpx>=0.28.1
Requires-Python: >=3.10
Project-URL: Homepage, https://github.com/pitersbro/lakefront
Project-URL: Repository, https://github.com/pitersbro/lakefront
Project-URL: Bug Tracker, https://github.com/pitersbro/lakefront/issues
Description-Content-Type: text/markdown

# Lakefront

A terminal-based lakehouse observability platform for exploring and managing data sources from your command line.

![Project](docs/project-screen.svg)

---

## Quickstart

The fastest way to see Lakefront in action is the built-in demo:

```bash
uv run lakefront init
uv run lakefront demo
```

This seeds a sample project with two CSV datasets and opens it straight in the TUI.

To set up your own project instead:

```bash
# 1. Initialise ~/.lakefront
uv run lakefront init

# 2. Create a project
uv run lakefront projects create my-project -d "My first project"

# 3. Attach a data source (local Parquet or CSV)
uv run lakefront projects source add -p my-project -n orders -k local --path /data/orders.parquet

# 4. Open the TUI
uv run lakefront ui --project my-project
```

Running `lakefront ui` without `--project` opens a navigation screen listing all your projects.

---

## About

Working with lakehouse data — Parquet files on local disk or S3, DuckDB queries, materialized views — usually means jumping between tools, writing throwaway scripts, or wrestling with heavyweight UIs. Lakefront puts it all in one place: a fast TUI and CLI for data engineers who live in the terminal.

**Problems it solves:**

- **No single tool for lakehouse exploration** — Lakefront combines DuckDB-powered SQL querying, S3 source management, and dataset browsing in one cohesive interface.
- **Configuration sprawl** — profiles let you switch between environments (local dev, staging, production S3) with a single command, keeping credentials out of your scripts.
- **Context switching** — instead of firing up a Jupyter notebook or a GUI just to peek at a Parquet file, you stay in the terminal.

---

## Examples

### Initialise Lakefront

Bootstrap the `~/.lakefront` directory structure and create a default profile:

```bash
uv run lakefront init
```

### Config Management

```bash
# List all profiles
uv run lakefront config list

# Show config directories and paths
uv run lakefront config info

# Create a new profile
uv run lakefront config create --profile staging

# Inspect a profile's current settings
uv run lakefront config inspect --profile staging

# See which profile is active
uv run lakefront config get-active

# Switch to a different profile
uv run lakefront config set-active --profile staging
```

Secrets (S3 access keys etc.) can be written to the TOML profile or
set via environment variables instead:

```bash
export LAKEFRONT_S3__ACCESS_KEY=...
export LAKEFRONT_S3__SECRET_KEY=...
```

### Project Management

Projects are the top-level organisational unit in Lakefront. Each project lives in its own directory under `~/.lakefront/projects/` and can be pinned to a config profile.

```
~/.lakefront/projects/
└── my-project/
    ├── project.toml      ← metadata + pinned profile
    └── results/          ← analysis outputs
```

```bash
# List all projects
uv run lakefront projects list

# Create a new project
uv run lakefront projects create my-project -d "EDA on S3 parquet" -p staging

# Inspect a project
uv run lakefront projects inspect my-project

# Delete a project (prompts for confirmation)
uv run lakefront projects delete my-project
uv run lakefront projects delete my-project --yes
```

### Source Management

Data sources are attached to a project and point to a local path or S3 prefix.

```bash
# Add a source
uv run lakefront projects source add -p my-project -n raw -k s3 --path s3://bucket/raw/
uv run lakefront projects source add -p my-project -n local -k local --path /data/parquet/

# Remove a source
uv run lakefront projects source remove -p my-project -n raw
```

---

## TUI (Terminal User Interface)

The interactive TUI provides a rich, multi-pane interface for exploring and analyzing data without leaving the terminal.

### Project Screen

The main project workspace with a three-pane layout:

**Left Pane — Data Sources:**

- Browse all attached sources with expandable tree view
- View column names and data types inline
- Quickly navigate between sources
- **P**: Load statistical profile for the selected source in the right pane

**Center Pane — SQL Editor & Results:**

- Tabbed SQL editor for writing and managing multiple queries
- Syntax-highlighted editor with DuckDB SQL support
- **Ctrl+R**: Execute query
- **Ctrl+N**: Run query in a new results tab
- **Ctrl+S**: Save script to disk
- **Ctrl+T**: Create new editor tab
- **Ctrl+W**: Close current tab
- Results pane displays query output in scrollable tables

**Right Pane — Profiler:**

- Statistical summary for the selected source
- Row counts, column types, null rates, and value distributions
- Loaded on demand with **P** from the source pane

### Explore Screen

Deep-dive analysis with statistical profiling and AI insights:

- **Statistical Profile**: Automatic data profiling showing distribution, nulls, and cardinality
- **AI-Powered Insights**: Ask questions about your dataset using LLM integration
- **Interactive Q&A**: Type questions to get natural-language analysis and recommendations
- **Keyboard Shortcuts**:
  - **Ctrl+R** or Enter: Submit question to AI
  - **Q**: Return to project screen

### Navigation

- **Tab / Shift+Tab**: Move focus between panes
- Modal dialogs for source attachment and confirmations
- Themed UI with multiple color schemes (configurable via profile settings)

---
# Changelog

## v0.8.1 (2026-05-03)

### Fix

- move dependencies back inside [project] table

## v0.8.0 (2026-05-03)

### Feat

- add navigation screen and demo project setup

### Fix

- **tui**: make profiler on-demand via 'p' keybind
- **core**: skip configure_s3 for projects with no S3 sources
- **tui**: move NavigationScreen I/O off the main thread
- **tui**: use call_from_thread for notify calls inside thread workers

## v0.7.1 (2026-04-27)

### Fix

- toml properties not being overriden by corresponding env vars

## v0.7.0 (2026-04-26)

### Feat

- **tui**: support multiple tabs in editor and result panes

## v0.6.0 (2026-04-26)

### Feat

- **core**: read analyzer row limit setting
- **tui**: read theme setting from core section
- **core**: add core config section

## v0.5.1 (2026-04-26)

### Refactor

- **tui**: consolidate styles in app.tcss

## v0.5.0 (2026-04-24)

### Feat

- **tui**: add explore action to result pane

## v0.4.0 (2026-04-22)

### Feat

- data profile is presented on source selection
- enable ai powered data source exploration
- **cli**: add configuration profile delete and demo commands
- add anthropic section to configuration model

### Fix

- analyzer mishandles bool types
- typo on app title

### Refactor

- move data profiling and llm code to analyzer module
- move data profiling code from tui to core.analyzer
- split core.main responsibilities across submodules
- core

## v0.3.4 (2026-04-20)

### Fix

- **tui**: rebuild source list after attach and detach

## v0.3.3 (2026-04-19)

### Feat

- add commitizen for automated changelog and versioning

### Fix

- Add a timeout for S3 source path existence check
- Project init ignores sources if unable to reach them or their existence check fails

## v0.3.0 (2026-04-19)

### Feat

- Add support for attaching S3 sources to a project

## v0.2.2 (2026-04-18)

### Feat

- Local sources can be attached to and detached from a project during runtime (#4)
- **tui**: basic source navigation and query execution
- **core**: can attach and query csv, parquet and parquet datasets (#3)
- **tui**: preliminary layout
- **core**: project context registers sources for querying (#2)
- **wip**: project lifecycle management (#1)

### Refactor

- **core,cli**: moving around and renaming things
