Metadata-Version: 2.5
Name: crawlutron-core
Version: 0.4.0rc1
Summary: Policy-driven, self-hosted web crawler and content-extraction service built around FastAPI
License-Expression: MIT
Requires-Python: >=3.14
Requires-Dist: beautifulsoup4==4.15.0
Requires-Dist: brotli==1.2.1
Requires-Dist: camoufox==0.5.9
Requires-Dist: charset-normalizer==3.5.2
Requires-Dist: crawlutron-safe-url>=0.4.0.dev0
Requires-Dist: curl-cffi==0.17.0
Requires-Dist: fastapi==0.162.0
Requires-Dist: h2==4.4.1
Requires-Dist: httpx==0.28.1
Requires-Dist: lxml==6.1.3
Requires-Dist: markdownify==1.2.3
Requires-Dist: markitdown[docx,pdf,pptx,xlsx]==0.1.9
Requires-Dist: pdfplumber==0.11.10
Requires-Dist: pydantic==2.14.2
Requires-Dist: soupsieve==4.0.0
Requires-Dist: trafilatura==2.3.1
Requires-Dist: yoyo-migrations==9.0.0
Description-Content-Type: text/markdown

# crawlutron-core

The crawler engine: URL admission, fetch and transport, extraction, orchestration,
storage, and the `admission` resource-adaptive admission kernel.

This is a library, not a service. To run the HTTP service, install
`crawlutron-rest`; to run it in a container, see the `crawlutron` image.

```python
from crawlutron.admission import AdmissionGate
```

The package ships `py.typed`: its annotations are part of the contract.

## License

MIT. See the repository's `LICENSE`.