Metadata-Version: 2.5
Name: sinapsis_langchain_readers
Version: 1.0.0
Summary: Package that provides support for Langchain community data loaders.
Project-URL: Homepage, https://sinapsis.tech
Project-URL: Documentation, https://docs.sinapsis.tech/docs
Project-URL: Tutorials, https://docs.sinapsis.tech/tutorials
Project-URL: Repository, https://github.com/SinapsisHub/sinapsis-hub.git
Author-email: Ariel Ruiz-Garcia <ariel@sinapsis.tech>
Maintainer-email: sinapsis <dev@sinapsis.tech>
License-Expression: MIT OR Apache-2.0
License-File: LICENSE-APACHE
License-File: LICENSE-MIT
Requires-Python: >=3.13
Requires-Dist: langchain-community>=0.3.5
Requires-Dist: sinapsis<2,>=1
Provides-Extra: all
Requires-Dist: amazon-textract-caller>=0.2.4; extra == 'all'
Requires-Dist: apify-client>=1.8.1; extra == 'all'
Requires-Dist: arxiv>=2.1.3; extra == 'all'
Requires-Dist: assemblyai>=0.35.1; extra == 'all'
Requires-Dist: atlassian-python-api>=3.41.16; extra == 'all'
Requires-Dist: azure-ai-generative>=1.0.0b11; extra == 'all'
Requires-Dist: azure-storage-blob>=12.23.1; extra == 'all'
Requires-Dist: azureml-fsspec>=1.3.1; extra == 'all'
Requires-Dist: beautifulsoup4>=4.12.3; extra == 'all'
Requires-Dist: bibtexparser>=1.4.2; extra == 'all'
Requires-Dist: boto3>=1.35.57; extra == 'all'
Requires-Dist: browserbase>=1.0.0; extra == 'all'
Requires-Dist: bs4>=0.0.2; extra == 'all'
Requires-Dist: dgml-utils>=0.3.0; extra == 'all'
Requires-Dist: dropbox>=12.0.2; extra == 'all'
Requires-Dist: fauna>=2.3.0; extra == 'all'
Requires-Dist: firecrawl-py>=1.4.0; extra == 'all'
Requires-Dist: geopandas>=1.0.1; extra == 'all'
Requires-Dist: gitpython>=3.1.43; extra == 'all'
Requires-Dist: google-api-python-client>=2.151.0; extra == 'all'
Requires-Dist: google-auth-httplib2>=0.2.0; extra == 'all'
Requires-Dist: google-auth-oauthlib>=1.2.1; extra == 'all'
Requires-Dist: google-cloud-speech>=2.33.0; extra == 'all'
Requires-Dist: html2text>=2024.2.26; extra == 'all'
Requires-Dist: jq>=1.8.0; extra == 'all'
Requires-Dist: langchain-community>=0.3.5; extra == 'all'
Requires-Dist: langchain-google-alloydb-pg>=0.8.0; extra == 'all'
Requires-Dist: langchain-google-bigtable>=0.4.1; extra == 'all'
Requires-Dist: langchain-google-community[gcs]>=2.0.2; extra == 'all'
Requires-Dist: langchain-openai>=0.2.6; extra == 'all'
Requires-Dist: langchain-unstructured>=0.1.5; extra == 'all'
Requires-Dist: langchain-yt-dlp==0.0.7; extra == 'all'
Requires-Dist: langchain>=0.3.7; extra == 'all'
Requires-Dist: librosa>=0.11.0; extra == 'all'
Requires-Dist: llvmlite>=0.44.0; extra == 'all'
Requires-Dist: lxml>=4.30; extra == 'all'
Requires-Dist: mastodon-py>=1.8.1; extra == 'all'
Requires-Dist: motor>=3.7.1; extra == 'all'
Requires-Dist: msoffcrypto-tool>=6.0.0; extra == 'all'
Requires-Dist: nest-asyncio>=1.6.0; extra == 'all'
Requires-Dist: o365>=2.1.2; extra == 'all'
Requires-Dist: pandas>=2.2.3; extra == 'all'
Requires-Dist: pdfplumber>=0.11.7; extra == 'all'
Requires-Dist: playwright>=1.48.0; extra == 'all'
Requires-Dist: pyairtable>=2.3.5; extra == 'all'
Requires-Dist: pydub>=0.25.1; extra == 'all'
Requires-Dist: pymupdf>=1.24.13; extra == 'all'
Requires-Dist: pyodps>=0.12.1; extra == 'all'
Requires-Dist: pyowm>=3.3.0; extra == 'all'
Requires-Dist: pypdf>=5.1.0; extra == 'all'
Requires-Dist: python-dotenv>=1.0.1; extra == 'all'
Requires-Dist: python-magic>=0.4.27; extra == 'all'
Requires-Dist: sodapy>=2.2.0; extra == 'all'
Requires-Dist: telethon>=1.40.0; extra == 'all'
Requires-Dist: tweepy>=4.14.0; extra == 'all'
Requires-Dist: unstructured-client>=0.25.9; extra == 'all'
Requires-Dist: unstructured[pdf]>=0.16.5; extra == 'all'
Requires-Dist: wikipedia>=1.4.0; extra == 'all'
Requires-Dist: youtube-transcript-api>=1.2.2; extra == 'all'
Requires-Dist: yt-dlp>=2024.11.4; extra == 'all'
Provides-Extra: langchain-cloud-readers
Requires-Dist: amazon-textract-caller>=0.2.4; extra == 'langchain-cloud-readers'
Requires-Dist: assemblyai>=0.35.1; extra == 'langchain-cloud-readers'
Requires-Dist: atlassian-python-api>=3.41.16; extra == 'langchain-cloud-readers'
Requires-Dist: azure-ai-generative>=1.0.0b11; extra == 'langchain-cloud-readers'
Requires-Dist: azure-storage-blob>=12.23.1; extra == 'langchain-cloud-readers'
Requires-Dist: azureml-fsspec>=1.3.1; extra == 'langchain-cloud-readers'
Requires-Dist: boto3>=1.35.57; extra == 'langchain-cloud-readers'
Requires-Dist: dropbox>=12.0.2; extra == 'langchain-cloud-readers'
Requires-Dist: google-api-python-client>=2.151.0; extra == 'langchain-cloud-readers'
Requires-Dist: google-auth-httplib2>=0.2.0; extra == 'langchain-cloud-readers'
Requires-Dist: google-auth-oauthlib>=1.2.1; extra == 'langchain-cloud-readers'
Requires-Dist: google-cloud-speech>=2.33.0; extra == 'langchain-cloud-readers'
Requires-Dist: html2text>=2024.2.26; extra == 'langchain-cloud-readers'
Requires-Dist: langchain-community>=0.3.5; extra == 'langchain-cloud-readers'
Requires-Dist: langchain-google-bigtable>=0.4.1; extra == 'langchain-cloud-readers'
Requires-Dist: langchain-google-community[gcs]>=2.0.2; extra == 'langchain-cloud-readers'
Requires-Dist: langchain-openai>=0.2.6; extra == 'langchain-cloud-readers'
Requires-Dist: o365>=2.1.2; extra == 'langchain-cloud-readers'
Requires-Dist: playwright>=1.48.0; extra == 'langchain-cloud-readers'
Requires-Dist: pyodps>=0.12.1; extra == 'langchain-cloud-readers'
Requires-Dist: python-dotenv>=1.0.1; extra == 'langchain-cloud-readers'
Provides-Extra: langchain-common-readers
Requires-Dist: bs4>=0.0.2; extra == 'langchain-common-readers'
Requires-Dist: jq>=1.8.0; extra == 'langchain-common-readers'
Provides-Extra: langchain-database-readers
Requires-Dist: fauna>=2.3.0; extra == 'langchain-database-readers'
Requires-Dist: langchain-google-alloydb-pg>=0.8.0; extra == 'langchain-database-readers'
Requires-Dist: langchain>=0.3.7; extra == 'langchain-database-readers'
Requires-Dist: motor>=3.7.1; extra == 'langchain-database-readers'
Requires-Dist: pyowm>=3.3.0; extra == 'langchain-database-readers'
Provides-Extra: langchain-pdfs-readers
Requires-Dist: bibtexparser>=1.4.2; extra == 'langchain-pdfs-readers'
Requires-Dist: pdfplumber>=0.11.7; extra == 'langchain-pdfs-readers'
Requires-Dist: pymupdf>=1.24.13; extra == 'langchain-pdfs-readers'
Requires-Dist: pypdf>=5.1.0; extra == 'langchain-pdfs-readers'
Provides-Extra: langchain-productivity-tools
Requires-Dist: gitpython>=3.1.43; extra == 'langchain-productivity-tools'
Provides-Extra: langchain-productivity-tools-readers
Requires-Dist: lxml>=4.30; extra == 'langchain-productivity-tools-readers'
Provides-Extra: langchain-social-readers
Requires-Dist: langchain-yt-dlp==0.0.7; extra == 'langchain-social-readers'
Requires-Dist: mastodon-py>=1.8.1; extra == 'langchain-social-readers'
Requires-Dist: nest-asyncio>=1.6.0; extra == 'langchain-social-readers'
Requires-Dist: pandas>=2.2.3; extra == 'langchain-social-readers'
Requires-Dist: telethon>=1.40.0; extra == 'langchain-social-readers'
Requires-Dist: tweepy>=4.14.0; extra == 'langchain-social-readers'
Requires-Dist: youtube-transcript-api>=1.2.2; extra == 'langchain-social-readers'
Provides-Extra: langchain-unstructured-readers
Requires-Dist: langchain-unstructured>=0.1.5; extra == 'langchain-unstructured-readers'
Requires-Dist: msoffcrypto-tool>=6.0.0; extra == 'langchain-unstructured-readers'
Requires-Dist: python-magic>=0.4.27; extra == 'langchain-unstructured-readers'
Requires-Dist: unstructured-client>=0.25.9; extra == 'langchain-unstructured-readers'
Requires-Dist: unstructured[pdf]>=0.16.5; extra == 'langchain-unstructured-readers'
Provides-Extra: langchain-webpages-readers
Requires-Dist: apify-client>=1.8.1; extra == 'langchain-webpages-readers'
Requires-Dist: arxiv>=2.1.3; extra == 'langchain-webpages-readers'
Requires-Dist: beautifulsoup4>=4.12.3; extra == 'langchain-webpages-readers'
Requires-Dist: browserbase>=1.0.0; extra == 'langchain-webpages-readers'
Requires-Dist: dgml-utils>=0.3.0; extra == 'langchain-webpages-readers'
Requires-Dist: firecrawl-py>=1.4.0; extra == 'langchain-webpages-readers'
Requires-Dist: geopandas>=1.0.1; extra == 'langchain-webpages-readers'
Requires-Dist: librosa>=0.11.0; extra == 'langchain-webpages-readers'
Requires-Dist: llvmlite>=0.44.0; extra == 'langchain-webpages-readers'
Requires-Dist: pyairtable>=2.3.5; extra == 'langchain-webpages-readers'
Requires-Dist: pydub>=0.25.1; extra == 'langchain-webpages-readers'
Requires-Dist: pymupdf>=1.24.13; extra == 'langchain-webpages-readers'
Requires-Dist: sodapy>=2.2.0; extra == 'langchain-webpages-readers'
Requires-Dist: yt-dlp>=2024.11.4; extra == 'langchain-webpages-readers'
Provides-Extra: langchain-wepages-readers
Requires-Dist: youtube-transcript-api>=1.0.0; extra == 'langchain-wepages-readers'
Provides-Extra: langchain-wikipedia-readers
Requires-Dist: wikipedia>=1.4.0; extra == 'langchain-wikipedia-readers'
Description-Content-Type: text/markdown

<h1 align="center">
<br>
<a href="https://sinapsis.tech/">
  <img
    src="https://github.com/SinapsisHub/brand-resources/blob/main/public/logo/sinapsis-logo-horizontal-dark-brand.png?raw=true"
    alt="Sinapsis"
    width="300">
</a>
<br>
Sinapsis Langchain Readers
<br>
</h1>

<h3 align="center">Load documents and external content through LangChain connectors.</h3>

## Overview

**`sinapsis_langchain_readers`** — Package that provides support for Langchain community data loaders.

A retrieval pipeline must first convert many kinds of sources into documents. This integration isolates source-specific loading from downstream chunking, indexing and embedding.

> [!WARNING]
> **Upstream Deprecation — LangChain Community**
>
> LangChain officially sunset the `langchain-community` package in May 2026,
> moving toward independently maintained, provider-specific integration packages.
>
> `sinapsis-langchain-readers` currently relies on `langchain-community`
> for its document loader implementations.
>
> Existing Templates may continue to work with compatible dependency versions.
> However, because the upstream package is no longer maintained, these
> integrations may not receive new features, compatibility updates, or bug fixes.
>
> Future versions of this Sinapsis integration may transition to dedicated
> LangChain provider packages where suitable alternatives are available.
>
> **This notice concerns the upstream dependency, not a deprecation of
> `sinapsis-langchain-readers` itself.**

### Where it fits

Choose the extra matching the source you intend to read—web, Wikipedia, PDFs, databases, cloud storage or productivity systems. Provider authentication and parser limitations remain source-specific.

## Installation

Requires **Python 3.13 or later**, as declared in this package’s manifest. Install the base distribution using your preferred package manager:

```bash
uv pip install sinapsis_langchain_readers
# or
pip install sinapsis_langchain_readers
```

## Optional features and dependencies

The base installation includes the dependencies required by the package. The following **extras** are declared in this package’s `pyproject.toml`; install only those needed by your workflow. They are standard Python optional dependencies, not separate execution profiles.

| Extra | Adds / when to select it |
|---|---|
| `langchain-webpages-readers` | LangChain connectors for webpages readers. **Dependencies:** `apify-client`, `arxiv`, `beautifulsoup4`, `browserbase`, `dgml-utils` and 9 more |
| `langchain-wikipedia-readers` | LangChain connectors for wikipedia readers. **Dependencies:** `wikipedia` |
| `langchain-pdfs-readers` | LangChain connectors for pdfs readers. **Dependencies:** `bibtexparser`, `pdfplumber`, `pymupdf`, `pypdf` |
| `langchain-unstructured-readers` | LangChain connectors for unstructured readers. **Dependencies:** `langchain-unstructured`, `unstructured[pdf]`, `unstructured-client`, `python-magic`, `msoffcrypto-tool` |
| `langchain-cloud-readers` | LangChain connectors for cloud readers. **Dependencies:** `amazon-textract-caller`, `assemblyai`, `atlassian-python-api`, `azure-ai-generative`, `azure-storage-blob` and 16 more |
| `langchain-social-readers` | LangChain connectors for social readers. **Dependencies:** `langchain-yt-dlp`, `mastodon-py`, `nest-asyncio`, `pandas`, `telethon` and 2 more |
| `langchain-productivity-tools-readers` | LangChain connectors for productivity tools readers. **Dependencies:** `lxml` |
| `langchain-common-readers` | LangChain connectors for common readers. **Dependencies:** `bs4`, `jq` |
| `langchain-database-readers` | LangChain connectors for database readers. **Dependencies:** `fauna`, `langchain`, `langchain-google-alloydb-pg`, `motor`, `pyowm` |
| `langchain-productivity-tools` | LangChain connectors for productivity tools. **Dependencies:** `gitpython` |
| `all` | Bundle declared by this package; its manifest includes `langchain-productivity-tools`, `langchain-database-readers`, `langchain-common-readers`, `langchain-productivity-tools-readers`, `langchain-social-readers`, `langchain-cloud-readers`, `langchain-unstructured-readers`, `langchain-pdfs-readers`, `langchain-wikipedia-readers`, `langchain-webpages-readers`. It does not include `langchain-wepages-readers`. |
| `langchain-wepages-readers` | LangChain connectors for wepages readers. **Dependencies:** `youtube-transcript-api` |

For example:

```bash
uv pip install "sinapsis_langchain_readers[langchain-webpages-readers]"
```

The manifest also defines development-only dependency groups: `dev` (`ruff`, `pre-commit`). These support source development and testing and are **not** runtime extras.

<h2 id="features">🚀 Features</h2>

<h3> Templates Supported</h3>

The **Sinapsis Langchain** module provides wrapper templates for **LangChain's community data loaders**, making them seamlessly usable within Sinapsis.
> [!NOTE]
> Each loader template supports one attribute:
> - **`add_document_as_text_packet`** (`bool`, default: `False`): Whether to add the loaded document as a text packet.
> Other attributes can be dynamically assigned through the class initialization dictionary (`class init attributes`).


For example, for ***WikipediaLoaderWrapper*** use ```sinx templates example-config WikipediaLoaderWrapper``` to produce the following example config:

```yaml
agent:
  name: agent to load Wikipedia documents using WikipediaLoaderWrapper template
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: WikipediaLoaderWrapper
  class_name: WikipediaLoaderWrapper
  template_input: InputTemplate
  attributes:
    add_document_as_text_packet: false
    wikipedialoader_init:
        query: the query for wikipedia
        lang: en
        load_max_docs: 5000
        load_all_available_meta: False
        doc_content_chars_max: 4000,
```

A complete list of available document loader classes in LangChain can be found at:
[LangChain Community Document Loaders](https://python.langchain.com/api_reference/community/document_loaders.html#langchain-community-document-loaders)

<details>
<summary><strong><span style="font-size: 1.25em;">🚫 Excluded Loaders</span></strong></summary>

Some base classes or loaders that required additional configuration have been excluded and support for this will be included in future releases.

- **Blob**
- **BlobLoader**
- **OracleTextSplitter**
- **OracleDocLoader**
- **TrelloLoaderExecute**
- **TwitterTweetLoader**
- **TrelloLoader**
- **GoogleApiYoutubeLoader**
- **GoogleApiClient**
- **DiscordChatLoader**
- **AssemblyAIAudioTranscriptLoader**
- **ArcGISLoader**

For all other supported loaders, refer to the LangChain API reference linked above.
</details>

## Example Agent

The following is adapted directly from the package’s shipped [`one_drive_loader.yml`](src/sinapsis_langchain_readers/configs/one_drive_loader.yml) configuration. Save it as `my_sinapsis_langchain_readers_agent.yaml`. The example may require model checkpoints, input files, additional Template packages, or provider credentials as described in its configuration.

```yaml
agent:
  name: my_sinapsis_langchain_readers_agent
  description: "One Drive Loader example"

templates:

- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: LangChainOneDriveLoader
  class_name: LangChainOneDriveLoader
  template_input: InputTemplate
  attributes:
    add_document_as_text_packet: false
    folder_id: folder_id
    drive_id: drive_id
    auth_with_token: false
    token_path: /path/to/token.txt
```

```bash
sinx jobs run my_sinapsis_langchain_readers_agent.yaml
```

More configurations: [`one_drive_loader.yml`](src/sinapsis_langchain_readers/configs/one_drive_loader.yml), [`wikipedia_loader.yml`](src/sinapsis_langchain_readers/configs/wikipedia_loader.yml), [`google_drive_loader.yml`](src/sinapsis_langchain_readers/configs/google_drive_loader.yml), [`test_config.yaml`](src/sinapsis_langchain_readers/configs/test_config.yaml).

## Documentation

- [Sinapsis documentation](https://docs.sinapsis.tech/)
- [Tutorials and examples](https://docs.sinapsis.tech/docs/latest/tutorials)
- [Sinapsis website](https://sinapsis.tech/)

## License

Sinapsis open-source components are available under your choice of the [MIT License](LICENSE-MIT) or the [Apache License 2.0](LICENSE-APACHE), unless otherwise noted.
