Metadata-Version: 2.4
Name: desync-data
Version: 0.1.0
Summary: Utility tools for Desync AI users
Author: Jackson Ballow
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Requires-Dist: desync_search
Requires-Dist: beautifulsoup4
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# desync-data

Utility functions for working with `PageData` objects from the [`desync_search`](https://pypi.org/project/desync-search/) library. These tools help you clean, filter, deduplicate, extract links, and export structured data from websites crawled via Desync.

---

## 🚀 Features

- 🔍 Remove boilerplate from page content (prefix/suffix delimiters)
- 📌 Filter pages by URL substring
- 🧹 Remove duplicate pages (based on `text_content`)
- 🔗 Extract link graphs (internal navigation structure)
- 📤 Export pages to CSV, JSON, or SQLite

---

## 📦 Installation

```bash
pip install desync-tools
