Metadata-Version: 2.4
Name: fwforge
Version: 0.1.4
Summary: Fixed-width file parser for legacy logistics & supply chain data
Author-email: Devotall <devinmcm1233@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/Devotall/fwforge
Project-URL: Repository, https://github.com/Devotall/fwforge
Project-URL: Issues, https://github.com/Devotall/fwforge/issues
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=6.0.2
Dynamic: license-file

# FixedWidth Forge

[![PyPI version](https://img.shields.io/pypi/v/fwforge.svg)](https://pypi.org/project/fwforge/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/fwforge)](https://pypi.org/project/fwforge/)
[![GitHub](https://img.shields.io/github/license/Devotall/fwforge)](https://github.com/Devotall/fwforge)

**Parse legacy fixed-width files from logistics, warehouses, ERP systems, and mainframes — instantly.**

Turn messy carrier reports, shipment manifests, and COBOL exports into clean CSV or JSON with a simple YAML schema.

## Why Logistics & Supply Chain?
Legacy formats like **fixed-width flat files**, **EDI**, and proprietary mainframes are the glue of the supply chain. `fwforge` provides the bridge to modern data pipelines without the enterprise bloat.

## Features
- **Schema-Driven**: Define column layouts in YAML. Supports `start+length` or `start+end` positions.
- **Ultra-Fast**: Built on Python principles for rapid parsing of multi-gigabyte flat files.
- **Batch Processing**: Process entire directories of manifest exports with one command.
- **Inference Engine**: Use `--infer` to automatically generate a baseline schema from a sample data file.
- **Clean Output**: Transform legacy data to clean CSV or JSON with built-in type casting.

## Installation (Free / Open Source)

```bash
pip install fwforge
```

Install from source (latest dev):

```bash
pip install git+https://github.com/Devotall/fwforge.git
```

**Looking for the commercial Pro binary?**  
Visit our [store](https://buy.lemonsqueezy.com/fwforge-pro) — Lemon Squeezy now handles distribution and delivery of the standalone high-performance fwforge-pro binary (multi-threading, audit logs, enterprise features, etc.). One-time purchase or annual license (no PyPI, no subscriptions for the binary itself).

(The [fwforge-pro/](https://github.com/Devotall/fwforge/tree/main/fwforge-pro) directory in this repo contains the source/build configuration for the Pro binary.)

## Security

FW Forge is built with a security-first mindset. The codebase has undergone a proactive security review to ensure robustness and data integrity. 

- **Minimal Dependency Surface Area**: We intentionally minimize third-party dependencies to reduce supply chain risk.
- **Secure Architecture**: The project separates core processing logic from distribution and uses a clean, minimal codebase that is periodically scanned for vulnerabilities.
- **Input Sanitization**: File processing operations use standard, vetted libraries and avoid insecure shell-based operations.

For any security-related inquiries or to report a potential vulnerability, please reach out via our [GitHub Issues](https://github.com/Devotall/fwforge/issues) page.

### 1. Infer a schema
Start from scratch with a sample file:
```bash
fwforge --infer -i data.txt > my-layout.yaml
```

### 2. Parse data
Convert legacy data to CSV (using the included sample):
```bash
fwforge -i data.txt -s layout.yaml -f csv -o output.csv
cat output.csv
```

### 3. Batch process a folder
```bash
fwforge -i ./daily_manifests/ -s manifest.yaml -f json
```

## VS Code Extension (Alpha)
Preview legacy mainframe data as interactive tables directly in VS Code.

- Located in: `vscode-extension/`
- Supports context-menu "FW Forge: Preview as Table" for `.txt` and `.fwf` files (and similar).
- Visualizes raw mainframe output using your custom YAML schemas.
- Requires the `fwforge` CLI in PATH (configurable via `fwforge.cliPath` setting).
- **Funnel for Pro**: Automatically detects `fwforge-pro` binary (or falls back); shows >50MB capability nudge for free CLI users. Also tracks free single-file previews and offers upgrade after ~20 uses to address batch bypass via 1-at-a-time in the IDE.

### Installation
```bash
cd fwforge/vscode-extension
npm install -g @vscode/vsce
vsce package
code --install-extension fixedwidth-forge-0.1.0.vsix
```

The extension runs the CLI to parse and renders a clean HTML table in a webview.

## Early Adoption

**23 downloads / acquisitions** (20 in the last 30 days) 🎉

- Page views in funnel: 12 (100%)
- Acquisition: 23 (191.67% trend)
- Tracked via dashboard analytics (funnel + trend charts)

Thanks to the first 23 users who gave it a shot! This is after the initial PyPI + VS Code extension launch.

## Free vs Pro

- **Free (PyPI + open source)**: `pip install fwforge`. Great for most users. Directory batch processing is limited to 5 files (soft limit). The VS Code extension also has a soft limit on single-file previews (~20) when using the free CLI.
- **Pro (commercial binary)**: Standalone `fwforge-pro` binary (multi-threading, auto-repair, audit logs, pre-built schemas, unlimited files). Drop it in PATH and the extension picks it up automatically.

Purchase Pro at the [Lemon Squeezy store](https://buy.lemonsqueezy.com/fwforge-pro).

This follows an "invisible sales" model: the extension, CLI limits, and docs gently direct power users who hit scale to the paid version.

### Invisible Sales / Long-Tail Strategy
- Document specific formats: "How to parse SAP fixed-width to CSV", "AS/400 flat file parsing", etc. (add pages to GitHub Wiki or /docs).
- List supported formats in the VS Code Marketplace description.
- Answer GitHub issues in related logistics/EDI projects with links to fwforge.

## Example Schema (`layout.yaml`)
```yaml
name: "Freight-Manifest-v1"
columns:
  - name: "carrier_code"
    start: 0
    length: 5
    trim: true
    type: "string"
  - name: "weight"
    start: 20
    length: 10
    trim: true
    type: "float"
```

## Development

```bash
git clone https://github.com/Devotall/fwforge.git
cd fwforge
uv sync
uv run pytest -v

# Run the CLI
uv run fwforge --help
```

## Roadmap

- More robust type casting (dates, currency, custom)
- Schema validation + strict mode
- Better inference (header detection, multi-line records)
- Performance / memory improvements for GB+ files
- Standalone binary releases
- Expanded output formats (parquet, etc.)

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for release notes.
