Metadata-Version: 2.4
Name: csv-surgeon
Version: 1.1.0
Summary: Intelligent CSV repair and sanitization for broken data files
Author: Intellirim
License: MIT
Project-URL: Homepage, https://github.com/intellirim/csv-surgeon
Project-URL: Repository, https://github.com/intellirim/csv-surgeon
Keywords: csv,data-cleaning,cli-tool,data-engineering,etl,data-quality
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1.0
Requires-Dist: chardet>=5.0.0
Dynamic: license-file

# CSV Surgeon

Intelligent CSV repair and sanitization for broken data files

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Overview

CSV Surgeon is a CLI tool that automatically detects and repairs broken CSV files with encoding issues, delimiter mismatches, malformed quotes, embedded linebreaks, and structural inconsistencies. Unlike web-based tools or manual approaches, it uses statistical analysis and heuristic algorithms to infer correct file structure, normalize encodings, and reconstruct malformed records while preserving data integrity.

## Installation

```bash
pip install csv-surgeon
```

## Usage

```bash
# Repair broken CSV
csv-surgeon repair broken.csv

# Save to file with encoding
csv-surgeon repair data.csv -o cleaned.csv --encoding utf-8

# Analyze without repair
csv-surgeon analyze messy.csv

# Redact PII
csv-surgeon repair input.csv --delimiter , --sanitize-pii
```

## Features

- **Multi-pass encoding detection**: Uses chardet with fallback chain (UTF-8, UTF-16, Latin-1, CP1252) and confidence scoring
- **Statistical delimiter inference**: Tests 4 delimiter types (comma, semicolon, tab, pipe) using coefficient of variation
- **Context-aware quote repair**: Detects and repairs unclosed quotes using delimiter-aware heuristics
- **Embedded linebreak reconstruction**: Merges records split across multiple lines using column count consistency
- **Header auto-detection**: Distinguishes headers from data using type consistency analysis
- **PII sanitization**: Detects and redacts emails, phone numbers (3 formats), and SSNs using regex patterns
- **Detailed reporting**: Provides confidence scores (0.0-1.0) for encoding and delimiter detection
- **Error handling**: Gracefully handles permission errors, empty files, and binary files

## How It Works

Uses chardet for encoding detection, statistical analysis for delimiter inference (coefficient of variation), state machine for quote repair, and column count mode for record reconstruction.

## Why CSV Surgeon?

Unlike RepairMyCSV (web-only), csvkit (complex), OpenRefine (GUI), or Excel (manual), CSV Surgeon combines automatic detection, CLI automation, and intelligent algorithms in one simple tool.

## License

MIT License - Copyright (c) 2026 Intellirim
