Metadata-Version: 2.4
Name: pyextractit
Version: 0.1.0
Summary: A utility to extract recursively from single zip/tar/tar.gz file till matching the target file name patter (regexp). then rename the file or files with specified prefix. This is quite useful when your data hides in the deep
Project-URL: Homepage, https://github.com/fxyzbtc/pyextractit
Project-URL: Repository, https://github.com/fxyzbtc/pyextractit
Project-URL: Documentation, https://deepwiki.com/fxyzbtc/pyextractit
Project-URL: Issues, https://github.com/fxyzbtc/pyextractit/issues
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Utilities
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: loguru>=0.7.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: typer>=0.9.0
Requires-Dist: tenacity>=8.0.0
Dynamic: license-file

# PyExtractIt

A utility to recursively extract from deeply nested compressed archives until finding files matching the target filename pattern (regexp). Files are renamed with sequential numbering (prefix_filename_sn1, prefix_filename_sn2, etc.). This is especially useful when your target data is buried deep within multiple layers of compressed archives.

## Features

- 🔄 **Deep recursive extraction**: Automatically extracts nested archives regardless of depth to find target files
- 🎯 **Filename pattern matching**: Uses regular expressions to match filenames (not paths) 
- 📦 **Multiple formats**: Supports ZIP, TAR, TAR.GZ, TAR.BZ2, TAR.XZ formats
- 🏷️ **Sequential file renaming**: Renames matched files with customizable prefix and sequential numbering (prefix_filename_sn1, prefix_filename_sn2, etc.)
- 🌊 **Unlimited depth**: Extracts target files no matter how deeply nested in compressed archives
- 📁 **Structure preservation**: Option to maintain directory structure
- ⚡ **Performance**: Efficient extraction with progress indicators
- 🛡️ **Error handling**: Robust error handling and logging

## Installation

```bash
# Install from PyPI (when published)
pip install pyextractit

# Install from source
git clone https://github.com/fxyzbtc/pyextractit.git
cd pyextractit
pip install -e .
```

## Quick Start

### Command Line Usage

```bash
# Extract all .txt files from an archive (renamed as extracted_filename_sn1.txt, extracted_filename_sn2.txt, etc.)
pyextractit archive.zip ".*\.txt$" --prefix "extracted_"

# Extract config files with specific naming pattern from deeply nested archives
pyextractit nested.tar.gz "config.*\.json$" --output ./configs

# Preserve directory structure during extraction
pyextractit data.zip ".*\.csv$" --preserve-structure

# Limit archive extraction depth (but target files are still extracted regardless of depth)
pyextractit deep.zip ".*\.log$" --max-depth 3

# Disable unlimited depth for target files (apply depth limit to target extraction too)
pyextractit archive.tar.gz ".*\.txt$" --limited-depth --max-depth 5

# Overwrite existing files
pyextractit archive.tar.gz ".*\.txt$" --overwrite

# Verbose output
pyextractit archive.zip ".*\.txt$" --verbose
```

### Python API Usage

```python
from pyextractit import RecursiveExtractor, ExtractorConfig
from pathlib import Path

# Create configuration
config = ExtractorConfig(
    target_pattern=r".*\.txt$",  # Matches filename only, not full path
    prefix="extracted_",
    max_depth=50,  # For archive extraction depth
    output_dir=Path("./output"),
    preserve_structure=False,
    overwrite_existing=False,
    unlimited_depth=True  # Extract target files regardless of depth
)

# Create extractor and run
extractor = RecursiveExtractor(config)
result = extractor.extract_from_archive(Path("archive.zip"))

# Check results
if result.success:
    print(f"Found {len(result.matched_files)} matching files")
    for match in result.matched_files:
        print(f"Extracted: {match.final_path}")  # e.g., extracted_config_sn1.txt, extracted_data_sn2.csv
else:
    print(f"Extraction failed: {result.error_message}")
```

## Configuration Options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `target_pattern` | str | - | Regular expression pattern to match target filenames (not full paths) |
| `prefix` | str | `"extracted_"` | Prefix to add to matched files (files renamed as prefix_filename_sn1, prefix_filename_sn2, etc.) |
| `max_depth` | int | `50` | Maximum depth for recursive archive extraction (1-200) |
| `output_dir` | Path | `None` | Output directory (default: `./extracted`) |
| `preserve_structure` | bool | `False` | Preserve directory structure |
| `overwrite_existing` | bool | `False` | Overwrite existing files |
| `temp_dir` | Path | `None` | Custom temporary directory |
| `unlimited_depth` | bool | `True` | Extract target files regardless of depth |

## Supported Archive Formats

- `.zip` - ZIP archives
- `.tar` - TAR archives  
- `.tar.gz`, `.tgz` - Gzip-compressed TAR archives
- `.tar.bz2` - Bzip2-compressed TAR archives
- `.tar.xz` - XZ-compressed TAR archives

## Common Use Cases

### 1. Extract Configuration Files

```bash
# Find all config files in nested archives (renamed as backup_config_sn1.json, backup_config_sn2.yaml, etc.)
pyextractit app-backup.zip "config.*\.(json|yaml|yml)$" --prefix "backup_"
```

### 2. Data Mining from Deep Archives

```bash
# Extract CSV data files from deeply nested structure (unlimited depth)
pyextractit dataset.tar.gz ".*\.csv$" --max-depth 10 --preserve-structure
```

### 3. Log File Extraction

```bash
# Extract log files with date pattern (renamed as extracted_app_2024_sn1.log, extracted_system_2024_sn2.log, etc.)
pyextractit logs.zip ".*_2024.*\.log$" --output ./logs --overwrite
```

### 4. Backup File Recovery

```bash
# Find and extract specific backup files (renamed as recovered_backup_db_sn1.sql, recovered_backup_logs_sn2.sql, etc.)
pyextractit backup.tar.gz "backup_.*\.sql$" --prefix "recovered_"
```

## Exit Codes

- `0`: Success
- `1`: Error (configuration error, extraction failure, etc.)

## Logging

PyExtractIt uses structured logging with the following levels:

- **ERROR**: Critical errors that cause extraction to fail
- **WARNING**: Non-critical issues (corrupted files, permission errors)
- **INFO**: General extraction progress and results
- **DEBUG**: Detailed extraction process information

Logs are written to:
- Console (configurable verbosity)
- `pyextractit.log` file (rotated, 30 days retention)

## Examples

### Complex Nested Structure

```
archive.zip
├── data/
│   ├── config.json ✓ (matches pattern)
│   └── nested.zip
│       ├── inner_config.json ✓ (matches pattern)
│       └── deeper.tar.gz
│           └── deep_config.json ✓ (matches pattern)
└── README.txt
```

```bash
pyextractit archive.zip ".*config\.json$" --prefix "found_"
```

**Result:**
```
./extracted/
├── found_config_sn1.json
├── found_inner_config_sn2.json
└── found_deep_config_sn3.json
```

### Archive Detection

If matched files are themselves archives, PyExtractIt will indicate this:

```bash
pyextractit outer.zip "backup_.*\.zip$" --verbose
```

**Output:**
```
✅ Extraction completed successfully!
📁 Extracted to: ./extracted
🎯 Found 2 matching files
📦 Total files extracted: 15
🔍 Maximum depth reached: 2
⏱️  Time taken: 1.34 seconds

📋 Matched files:
  📦 found_backup_2024_sn1.zip (archive)
    ⚠️  This file is an archive and can be processed further
  📦 found_backup_legacy_sn2.zip (archive)
    ⚠️  This file is an archive and can be processed further
```

## Development

### Setup Development Environment

```bash
git clone https://github.com/fxyzbtc/pyextractit.git
cd pyextractit
pip install -e ".[dev]"
```

### Run Tests

```bash
# Run all tests with coverage
pytest --cov=pyextractit --cov-report=html

# Run specific test file
pytest tests/test_extractor.py -v

# Run with verbose output
pytest -v -s
```

### Code Quality

```bash
# Format code
ruff format pyextractit tests

# Lint code
ruff check pyextractit tests

# Type checking (if mypy is installed)
mypy pyextractit
```

## Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature-name`)
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass (`pytest`)
6. Submit a pull request

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Changelog

### v0.1.0 (2024-08-05)
- Initial release
- Recursive archive extraction
- Pattern-based file matching
- Multiple archive format support
- Command-line interface
- Python API
- Comprehensive test suite

## Links

- **Homepage**: https://github.com/fxyzbtc/pyextractit
- **Documentation**: https://deepwiki.com/fxyzbtc/pyextractit
- **Issues**: https://github.com/fxyzbtc/pyextractit/issues
- **PyPI**: https://pypi.org/project/pyextractit/ (when published)

## Support

If you encounter any issues or have questions:

1. Check the [documentation](https://deepwiki.com/fxyzbtc/pyextractit)
2. Search [existing issues](https://github.com/fxyzbtc/pyextractit/issues)
3. Create a [new issue](https://github.com/fxyzbtc/pyextractit/issues/new) with:
   - Python version
   - Archive type and structure
   - Command used
   - Error messages
   - Expected vs actual behavior

This repository serves as a template for creating new Python projects. It provides a basic structure and configuration to get started quickly.

## Purpose

This template aims to streamline the setup process for new Python projects by providing a standardized layout, dependency management configuration, and basic example files. It helps avoid the repetitive setup tasks involved in starting a new project.

## Using the Template
0.  **Fine-tune**
    *   Review the copilot instructions if you also use microsoft copilot
    *   The instruction is extreme personal favor
1.  **Clone or Copy:**
    *   Use this repository as a template on GitHub (click "Use this template").
    *   Alternatively, clone or download the repository and manually copy the files to your new project directory.
2.  **Rename Project:**
    *   Rename the `pyprojectname` directory to your actual project's name.
    *   Update the project name in `pyproject.toml`.
3.  **Install Dependencies:**
    ```bash
    uv pip install -e .[dev]
    ```
4.  **Start Developing:** Begin adding your project's code and tests.

## Development Guide (Using This Template)

*   **Dependencies:** Manage dependencies using `uv` and `pyproject.toml`. Add runtime dependencies under `[project.dependencies]` and development dependencies under `[project.optional-dependencies]`.
*   **Structure:** Place your library code within the main project directory (e.g., `your_project_name/`). Write tests in the `tests/` directory.
*   **Entry Points:** Configure command-line scripts or module execution (`python -m your_project_name`) in `pyproject.toml` under `[project.scripts]` or `[project.entry-points."console_scripts"]`.
*   **Testing:** Run tests using `pytest`. Ensure good test coverage.
*   **Linting/Formatting:** Use `ruff` or `black` to maintain code style.

## Links

*   **Homepage:** [https://github.com/fxyzbtc/mypytemplate]
*   **Wiki:** [https://deepwiki.com/fxyzbtc/mypytemplate]
*   **Issues:** [Link to GitHub Issues Page]
