Metadata-Version: 2.5
Name: flashdata-haystack
Version: 0.2.1
Summary: Google and YouTube data components for Haystack pipelines and agents
Project-URL: Homepage, https://flashdata.dev/?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=combined
Project-URL: Documentation, https://github.com/flashdatadev/flashdata-mcp/tree/main/packages/haystack
Project-URL: Source, https://github.com/flashdatadev/flashdata-mcp
Project-URL: Issues, https://github.com/flashdatadev/flashdata-mcp/issues
Author: FlashData
License-Expression: MIT
License-File: LICENSE
Keywords: agent,flashdata,google-search,haystack,youtube-transcript
Requires-Python: >=3.11
Requires-Dist: haystack-ai<4,>=3.4
Requires-Dist: httpx<1,>=0.28.1
Requires-Dist: jsonschema<5,>=4.23
Requires-Dist: pydantic<3,>=2.7.4
Description-Content-Type: text/markdown

# FlashData for Haystack

Build research pipelines and agent tools with six Google sources and nine YouTube sources.
This package exposes 17 components for 15 capabilities: YouTube delivery has separate submit,
status and result components. It does not include a model, embeddings, or a vector store.

Requires Python 3.11+ and Haystack 3.4. Install from the versioned source release:

```bash
pip install "git+https://github.com/flashdatadev/flashdata-mcp.git@haystack-v0.2.1#subdirectory=packages/haystack"
```

The package is also available on [PyPI](https://pypi.org/project/flashdata-haystack/0.2.1/). Installation is free under MIT; FlashData API calls use
paid Credits. Configure `FLASHDATA_API_KEY` through your environment or secret manager after
creating a key in your FlashData account. Use the linked source guides for current rates and limits.

```python
from haystack import Pipeline
from flashdata_haystack import FlashDataGoogleSearch, as_tool

pipe = Pipeline()
pipe.add_component("search", FlashDataGoogleSearch())
result = pipe.run({"search": {"query": "retrieval augmented generation evaluation", "num": 5}})
print(result["search"]["result"])

# Constructing a tool does not call an API or a model.
tool = as_tool(FlashDataGoogleSearch())
# Add it to an existing Haystack Agent; the agent decides when to search.
```

The full API JSON is available as `result`. Realtime components also return `documents`, one
Haystack Document per top-level API result. Document content is complete JSON, with source and
result index metadata. Search snippets are not fetched webpage content; no media is automatically
downloaded. Each example also shows how to format results into context without a model call.

| Capability | Guide and runnable pipeline | Source contract |
| --- | --- | --- |
| Google Search | [Web evidence for an agent](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/google-search-flashdata.md) | [Inputs, outputs, pricing](https://flashdata.dev/docs/sources/google_search?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=google_search-task) |
| Google News | [News briefing](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/google-news-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/google_news?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=google_news-task) |
| Google Images | [Image shortlist](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/google-images-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/google_images?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=google_images-task) |
| Google Shopping | [Offer comparison](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/google-shopping-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/google_shopping?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=google_shopping-task) |
| Google Videos | [Video references](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/google-videos-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/google_videos?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=google_videos-task) |
| Google Places | [Local business shortlist](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/google-places-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/google_places?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=google_places-task) |
| YouTube Search | [Tutorial discovery](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-search-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_search?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_search-task) |
| YouTube Extended Search | [Larger research inventory](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-search-extended-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_search_extended?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_search_extended-task) |
| YouTube Metadata | [Video inspection](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-metadata-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_metadata?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_metadata-task) |
| YouTube Transcript | [Timestamped evidence](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-transcript-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_transcript?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_transcript-task) |
| YouTube Captions | [Uploader caption review](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-captions-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_captions?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_captions-task) |
| YouTube License Metadata | [License-label inspection](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-trainability-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_trainability?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_trainability-task) |
| YouTube Channel | [Channel briefing](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-channel-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_channel?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_channel-task) |
| YouTube Autocomplete | [Topic discovery](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-suggestions-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_suggestions?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_suggestions-task) |
| YouTube Delivery | [Saved job and storage delivery](https://github.com/flashdatadev/flashdata-mcp/blob/haystack-v0.2.1/packages/haystack/integration-pages/youtube-download-flashdata.md) | [Source guide](https://flashdata.dev/docs/sources/youtube_download?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=youtube_download-task) |

These are source documentation and proposed directory pages. They are not proof of 15 approved
Haystack directory entries. Actual IDs, URLs and review state are tracked separately in `entries.json`.

Save pipelines with `pipe.dumps()` and load reviewed YAML using
`Pipeline.loads(saved, allowed_modules=["flashdata_haystack"])`. API keys use Haystack `Secret`;
only environment variable names are serialized. Token secrets deliberately refuse serialization.
Key and storage values never appear in tool input schemas.

YouTube delivery is asynchronous and needs customer storage. `FlashDataYouTubeDownloadSubmit`
defaults to `allow_submit=False`; the operator must explicitly enable a paid storage write and set
`FLASHDATA_DOWNLOAD_STORAGE`. Save the receipt and use `FlashDataYouTubeDownloadStatus` or
`FlashDataYouTubeDownloadResult` to read that job. There is no automatic retry or polling.

Requests use a 120-second network timeout (15-second connect timeout) and a 20 MiB response limit.
After an uncertain submission, inspect Jobs before resubmitting to avoid duplicate charges.
API errors are reported as `FlashDataError` without raw credentials or response bodies.
Model, storage and API costs are separate. See [pricing](https://flashdata.dev/pricing?utm_source=haystack&utm_medium=integration&utm_campaign=intent_gtm_2026q4&utm_content=combined) and
[privacy](https://flashdata.dev/legal/privacy).

Development:

```bash
uv sync --group dev
uv run python scripts/sync_contract.py
uv run python scripts/generate.py
uv run ruff check --fix .
uv run ruff format .
uv run ruff check .
HAYSTACK_TELEMETRY_ENABLED=false uv run pytest
uv run python -m build
uv run twine check dist/*
```

The HTTP, input and storage contracts are vendored from the reviewed Python CLI implementation.
`scripts/sync_contract.py` reproduces them and records source hashes; only the User-Agent differs.
Generated component classes have explicit input sockets and fixed sources. Tests use HTTP fixtures
and block network access; they do not prove production API, model or customer-storage execution.

## Subtitle languages

For transcript and captions, `language` accepts one code or a list of 1–10 distinct
codes, such as `["en", "es"]`. Omit it to prefer `en-orig`, then `en`, returning one
track. Explicit strings keep the single-track response. Lists, including one-element
lists, return `tracks` and per-language `errors` inside each result. Keep available
tracks and report the failed languages; if none succeed, `tracks` is empty.
The `auto` choice applies to every language and never switches caption source.
