Metadata-Version: 2.4
Name: sphinx-unified-search
Version: 0.1.3
Summary: Merge search indexes from multiple Sphinx documentation projects.
Author: Canonical
License: Apache-2.0
Project-URL: Homepage, https://github.com/canonical/sphinx-unified-search
Classifier: Framework :: Sphinx
Classifier: Framework :: Sphinx :: Extension
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Sphinx>=7.0
Requires-Dist: requests>=2.31
Requires-Dist: psutil>=5.9
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Requires-Dist: sphinx-autobuild>=2024.2.4; extra == "test"
Dynamic: license-file

# sphinx-unified-search

![Status: Work in Progress][status-badge]
![Feedback Welcome][feedback-badge]
![Config may change][config-badge]
![PyPI release][release-badge]

> **⚠️ Work in progress.** This project is under active development.
> Configuration options and behavior may change without notice. It has not
> been fully tested for production use, but early testers and feedback are
> very welcome. Please [open an issue][issues] if you hit problems or have
> suggestions.

## Cross-project search for Sphinx documentation

`sphinx-unified-search` is a Sphinx extension that lets you search across
multiple Sphinx documentation sites from a single search box. It works by
merging remote `searchindex.js` files into your project's local search index
at build time.

The standard [Sphinx] search UI is preserved: no new search box, no external
service, and no per-query cost.

## Features

- Works with existing Sphinx themes
- No JavaScript customization required
- No external search service required
- Preserves the standard Sphinx search UI
- Remote search results link directly to their original documentation pages,
  keeping traffic on the source sites
- Supports any number of remote Sphinx projects
- Supports token-authenticated fetching for private/internal projects

## Who might find this useful

- **Upstream / downstream content**- if your (product and) docs build on top of
  an upstream project maintained elsewhere, your users can search the upstream
  or related (Sphinx) docs without leaving your site.
- **Multi-product teams** - separate Sphinx sites that share an audience, where
  users shouldn’t have to know which exact site to search. Or products that are
  part of a larger ecosystem.
- **Internal platform teams** - private doc sets that (may) need to be
  searchable alongside public ones without setting up new infrastructure.

---

## How to use the extension

Installing and configuring `sphinx-unified-search` is straightforward. If you
also want to include a private [Read the Docs (RTD)] project in the unified
search results, complete the extra setup below first.

### Configure access to private documentation projects

If you want to include a private RTD project in the unified search, you must
create an HTTP header token that can be used to download its `searchindex.js`.

For each private project:

1. Open the project's Read the Docs settings.
1. Go to "Sharing" and "Add share".
1. Select "HTTP header token" for the "Access type", and a suitable expiry
   date.
1. Select "Allow access to all versions?" if you intend to use `searchindex.js`
   for all versions of the docs set. Otherwise, unselect that option and add the
   versions that will be used by `sphinx-unified-search`.

Copy the generated token and store it as a secret environment variable in the
RTD project that is _building_ the unified search.

For example, if "PROJECT B" wants to the `searchindex.js` from "PROJECT A" to
its search results, generate the HTTP header token in PROJECT A, and add that as
an environment variable in PROJECT B
(e.g. `PROJECTA_DOCS_TOKEN=<generated HTTP header token>`).

### Install the extension

Add `sphinx-unified-search` to the `requirements.txt` file in your Sphinx
project:

```text
sphinx-unified-search
```

### Enable the extension

Add `sphinx_unified_search` to the `extensions` list in `conf.py`:

```python
extensions = [
    "sphinx_unified_search",
]
```

### Configure projects to index for unified search

Configure the list of documentation projects whose search indexes should be
included in the unified search.

For each project, specify:

- `name`: A unique identifier for the project. For search results from
  projects other than the host project, this value is appended to the page
  title in the search results.
- `base_url`: The base URL of the documentation, used as the destination when
  a user clicks a search result.
- `searchindex_url`: The URL of the project's `searchindex.js`.

For private projects, also specify:

- `auth_token_env`: The name of the environment variable containing the RTD HTTP
  header token.
- `auth_scheme`: The authorization scheme to use in the HTTP `Authorization`
  header. For RTD HTTP header tokens, set this to `"Token"`.

```python
unified_search_projects = [
    # Public project - no auth needed, plain HTTP fetch
    {
        "name": "Landscape",
        "base_url": "https://ubuntu.com/landscape/docs/",
        "searchindex_url": "https://ubuntu.com/landscape/docs/searchindex.js",
    },

    # Private project - token-gated fetch
    {
        "name": "ProjectA",
        "base_url": "https://documentation.ubuntu.com/projecta/",
        "searchindex_url": "https://documentation.ubuntu.com/projecta/searchindex.js",
        "auth_token_env": "PROJECTA_DOCS_TOKEN",
        "auth_scheme": "Token",
    },
]
```

### Build the docs locally

When building the documentation locally, you must set the environment variables
referenced by the `auth_token_env` entries in `unified_search_projects`.

For example, if your remote project uses this config:

```python
{
    "auth_token_env": "PROJECTA_DOCS_TOKEN",
    ...
}
```

Export the corresponding token before building:

```shell
export PROJECTA_DOCS_TOKEN="<token>"
```

Repeat this for each private project included in `unified_search_projects` set.

Once the required environment variables are set, build or serve the
documentation as usual with `make html` or `make run`.

### Filter search results by project

Add project filter chips to the search results page so visitors can narrow
the results to one or more projects:

```python
unified_search_project_filter = True

# Optional: label for this project's own results in the filter.
# Defaults to Sphinx's `project` setting.
unified_search_local_project_name = "My Docs"
```

When the search has finished, a "Filter by project" row appears above the
results, with one chip per project that has results and the number of results
next to it. Select one or more chips to show only those projects; select
"All" (or deselect every chip) to show everything again. The filter only hides
rows: results are not removed, and previews are still loaded for all of them.
The selection is stored in the URL (`?q=term&project=ProjectA`), so it survives
a reload and can be shared. The filter is off by default, and isn't shown when
results come from only one project.

### Fetch remote indexes once per live-preview session

By default, every build downloads each remote `searchindex.js` again. With
[sphinx-autobuild] (`make run`), that means a fresh download of every remote
index on each rebuild, which slows down the edit-preview loop.

Set the following in `conf.py` to download each remote index only once per
`sphinx-autobuild` session and reuse it for later rebuilds:

```python
unified_search_cache_remote_indexes = True
```

The cache is scoped to the running `sphinx-autobuild` process and stored under
the Sphinx doctree directory, so it is discarded when you restart the preview.
Regular builds (`make html`, Read the Docs, CI) are unaffected and always fetch
fresh indexes. Only successfully parsed downloads are cached, and the cache key
covers the index URL and its resolved authentication, so a changed token or
header forces a new download. Tokens themselves are never written to disk.

## Some things to note

A few behaviors and limitations to be aware of before you publish a unified
search, especially if you include private projects.

### Private docs: result previews

On the search results page, each result shows a short preview of the matching
content, fetched from the remote project’s page. This request is made
client-side in the visitor’s browser, so it can’t use the build-time token, and
it is subject to the browser’s same-origin policy.

The preview loads only if the remote page is same-origin or allows cross-origin
requests via CORS headers. Otherwise, the extension shows a placeholder message
instead. The search results and their links work normally in all cases.

### Beware of headings

If you add a private project as a remote of a public docs set, page titles and
headings from the private project will appear in the public search results, even
though the page _content_ itself won't load. Don't include private projects
whose titles or headings contain confidential information.

### Build the docs in CI (GitHub Actions)

Adding a secret to a repository doesn't automatically expose it to every
workflow run. Each job that needs it must reference it explicitly.

This matters most when the docs build runs in a reusable workflow (for example, `uses: ./.github/workflows/build-docs.yml`). Secrets aren't inherited by the
called workflow by default. A build that works locally with an exported token
can therefore fail in CI, or silently skip the private index.

To fix this, pass the secret through in the calling workflow:

```yaml
jobs:
  build-docs:
    uses: ./.github/workflows/build-docs.yml
    secrets:
      PROJECTA_DOCS_TOKEN: ${{ secrets.PROJECTA_DOCS_TOKEN }}
    # or, to pass all secrets: `secrets: inherit`
```

The reusable workflow must also declare the secret under
`on.workflow_call.secrets`, and the build step should expose it as an
environment variable named in `auth_token_env`.


[status-badge]: https://img.shields.io/badge/status-work%20in%20progress-yellow
[feedback-badge]: https://img.shields.io/badge/feedback-welcome-blue
[config-badge]: https://img.shields.io/badge/config-subject%20to%20change-orange
[release-badge]: https://img.shields.io/pypi/v/sphinx-unified-search
[issues]: https://github.com/canonical/sphinx-unified-search/issues
[Sphinx]: https://www.sphinx-doc.org/en/master/
[Read the Docs (RTD)]: https://about.readthedocs.com/
[sphinx-autobuild]: https://github.com/sphinx-doc/sphinx-autobuild
