Metadata-Version: 2.5
Name: findxpand
Version: 0.1.1
Summary: Server-side SEO remediation at your origin. One deploy; fixes arrive as a pushed manifest.
Project-URL: Homepage, https://nextoria.ae/findxpand
Author: Nextoria
License: MIT
Keywords: asgi,cli,django,fastapi,flask,middleware,seo,wsgi
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# findxpand

Server-side SEO remediation at your origin, for WSGI (Django, Flask, Pyramid)
and ASGI (FastAPI, Starlette, Quart). No dependencies.

```bash
uvx findxpand init          # or: pipx run findxpand init
```

`init` reads the project you run it in — framework, server, package manager,
the start command in your Procfile or Dockerfile — and prints the three things
you need: the install line for *your* package manager, a generated token, and
the one prefix that attaches the middleware. It writes nothing but `.env`, and
only after asking.

## Installing

Whichever of these your project already uses. `init` picks the right one from
the lockfile it finds and prints only that line.

| You have | Install |
|---|---|
| `uv.lock` | `uv add findxpand` |
| `poetry.lock` or `[tool.poetry]` | `poetry add findxpand` |
| `Pipfile.lock` | `pipenv install findxpand` |
| `requirements.txt` | `echo 'findxpand' >> requirements.txt`, then `pip install -r requirements.txt` |
| none of them | `pip install findxpand` |

The second column was a *Today* column naming `./findxpand-0.1.1.tar.gz`, for
the fortnight before PyPI had the package. It is gone rather than kept beside
the real one: two install lines per row is a reader choosing between them, and
the local file is now only for an air-gapped build, which is a support
conversation rather than a README.

The requirements route edits the file as well as installing, deliberately: an
install that is not written into `requirements.txt` survives exactly until the
next image rebuild.

**The distribution is `findxpand`; the import package is
`findxpand_middleware`.** They differ on purpose and the second one is not
changing — `sitecustomize`, `FINDXPAND_APP`, every entry point and every test
name it, and none of that is visible to you. `beautifulsoup4` imports as `bs4`
for the same reason.

## The whole install

```bash
findxpand-run gunicorn myproject.wsgi:application     # or uvicorn, or manage.py
```

One prefix on the command you already run. Nothing in your application changes,
there is no middleware to place, and `FINDXPAND_TOKEN` in the same environment
is the only other thing needed.

`findxpand-run` puts a bootstrap on `PYTHONPATH` and wraps the application at
the moment your server resolves it — gunicorn's `import_app`, uvicorn's
`Config.load`, Django's `get_wsgi_application`. Anything it does not recognise
still works: set `FINDXPAND_APP` and point your server at ours.

```bash
FINDXPAND_APP=myproject.wsgi:application \
  gunicorn findxpand_middleware.auto:application
```

### Why there is no `MIDDLEWARE` line

Because there is no position that works. Measured 31 Aug 2026:

| stack | mounted before compression | mounted after it |
|---|---|---|
| Express + `compression` | `skip-encoded` — page unmodified, `/status` reporting healthy, the fix silently never lands | 2,069 bytes of plain HTML on the wire under `content-encoding: gzip` |
| Django + `GZipMiddleware` | *not available* — outermost is the only place a WSGI wrapper goes, and outermost is after the compression | — |

So the step is deleted rather than reworded. `findxpand-run` wraps the
application outside your stack and owns the encoding negotiation on the pages it
rewrites: a path in the manifest is taken from your app as `identity` and
compressed on the way out; a path the manifest does not name is not touched at
all, and your own compression still runs on the rest of the site exactly as
before.

## Wrapping it yourself

Still supported, and what `findxpand-run` does for you:

```python
# WSGI
from findxpand_middleware.wsgi import Findxpand
application = Findxpand(application,
                        token=os.environ["FINDXPAND_TOKEN"],
                        cache_file="/var/tmp/findxpand.json")

# ASGI — same arguments
from findxpand_middleware.asgi import Findxpand
app = Findxpand(app, token=os.environ["FINDXPAND_TOKEN"],
                cache_file="/var/tmp/findxpand.json")
```

That is the only deploy. Approved fixes arrive afterwards as a manifest pushed
to `/__findxpand/manifest`, and no further release is needed.

> **Pick a cache path that survives a restart.** `/var/tmp` is fine on a VM and
> wrong on a container or a serverless platform, where it is discarded and every
> restart drops every rule until the next push. Use a mounted volume, and if the
> platform has no durable disk at all, say so — the manifest-in-the-repository
> mode exists for exactly that and does not need one.

## Environment

| Variable | Meaning |
|---|---|
| `FINDXPAND_TOKEN` | Shared secret. Required; the admin endpoints 401 without it, and `findxpand-run` starts your command unattached and says so if it is missing. |
| `FINDXPAND_CACHE_FILE` | Where a pushed manifest is persisted. Without it, every restart drops every rule until the next push. |
| `FINDXPAND_MANIFEST_FILE` | A manifest committed to your repository, for the no-inbound-endpoint mode. Read at boot; a push overrides it. |
| `FINDXPAND_ENABLED` | `false` leaves it wrapped and stops it touching responses. |
| `FINDXPAND_MAX_BYTES` | Largest HTML body to buffer. Default 2 MB; larger responses pass through. |
| `FINDXPAND_REQUIRE_SIGNATURE` | Verify the signature on every push. Default on. See below. |
| `FINDXPAND_RECOMPRESS` | Re-encode the pages we rewrite. Default on; off only for an intermediary that has taken over the negotiation. |
| `FINDXPAND_APP` | The application to wrap, when your server is not one `findxpand-run` recognises. |

The same names are the keyword arguments if you wrap the callable yourself:
`token`, `cache_file`, `manifest_file`, `enabled`, `max_bytes`,
`require_signature`, `recompress`.

### What it does not touch

Wrapping the callable puts it in front of every request, and it steps out of the
way of almost all of them. A path with no rule in the manifest is passed straight
to your application before anything is buffered — your JSON APIs, file downloads
and webhooks are never held. Beyond that it only rewrites `200` responses whose
content type is `text/html` and which nothing has already compressed.

## Checking it landed

```bash
findxpand doctor --url https://your-site
findxpand doctor --port 8000          # or: this machine, which is where it starts
```

Three exit codes and no fourth:

```text
0  it works, and we confirmed it
1  it is broken, and we confirmed that
2  we could not examine it - which is never reported as health
```

The third is the one that matters. Doctor reads `/__findxpand/status` with your
token and then **fetches one page the manifest actually names**, looking for
`x-findxpand-origin-mw` on the response. That second request is the whole point.
A middleware that is installed, answering, and attached to a process no request
reaches reports `degraded: false` and means it: from inside there is nothing to
complain about, and `considered = 0` is indistinguishable from a quiet morning.
Nothing in the process can close that, so doctor closes it from outside.

Three real runs against the same install, one minute apart:

```text
$ findxpand doctor --url https://example.ae --no-probe
findxpand doctor https://example.ae/ - could not examine

  - /status answered: 1 page rule(s), 0 redirect(s), manifest v4 via push;
    considered 0, applied 0.
  - it answers, and no response has ever reached a rewrite decision
    (considered = 0). Either no traffic has arrived since this process
    started, or the process answering /status is not the one serving the
    site. Nothing has been examined, so nothing is claimed.

  exit 2 - could not examine; nothing above confirmed the middleware is
  rewriting pages.

$ findxpand doctor --url https://example.ae
findxpand doctor https://example.ae/ - working

  - in the request path - /about came back marked "1". The rewrite is
    landing. The counters above were read before this fetch and do not
    include it, which is why they are still zero: the marker is the
    evidence, not the counter.

  exit 0 - working.

$ findxpand doctor --url https://example.ae      # mounted behind compression
findxpand doctor https://example.ae/ - broken

  - /about arrived already compressed. The middleware is behind something
    that compresses and needs to be in front; findxpand-run puts it there.

  exit 1 - broken.
```

The third one is why the probe exists: that install had **zero** in every
counter and `degraded: false`, because nothing had asked it for a page yet. The
fetch is what found it.

Two verdicts are worth knowing in advance. A **401** is exit 1, not 2 — the
middleware is there and holding a different secret from the one we push
manifests with, so nothing we approve can land until it matches. A **404**, or
anything else that is not our JSON, is exit 2 — it is equally what the wrong URL
looks like, and the URL the middleware answers on is often not the public
hostname.

`--no-probe` reads `/status` only and then says so rather than passing:
unmeasured is exit 2. `--json` gives the same findings with the exit code and a
machine-readable verdict name in the body. `findxpand status --url ...` prints
the middleware's own `/status` untouched. `--timeout` is milliseconds, and
`--dir` is the project to read `.env` from — the same flags, spelled the same
way, as the Node package's CLI.

Doctor also names any field in a pushed manifest this package would not apply,
because a field silently dropped at this end reads, from our side, exactly like
a fix that landed.

## What it does to a response

Only when the path has a rule, the status is 200, the content type is HTML, and
nothing has already compressed it. Everything else passes through, marked in
`x-findxpand-origin-mw`: `1` rewritten, `pass` unchanged, `skip-encoded`
already compressed, `skip-large` over `max_bytes`, `error` the rewrite raised
and the original bytes were served.

### It tells you when it is doing nothing

`/__findxpand/status` reports `counters`, `applied_age`, `degraded` and
`degraded_reason` as well as the rule count. This is the failure worth
monitoring: mounted after compression, or mounted where no rule matches, the
middleware serves every page untouched and every other signal looks healthy.

```json
{
  "degraded": true,
  "degraded_reason": "12 response(s) arrived already compressed and could not be rewritten - the middleware is mounted after compression, and needs to be mounted before it",
  "counters": { "considered": 12, "applied": 0, "skip_encoded": 12, "error": 0 }
}
```

Alert on `degraded`. `applied_age` is seconds since a rewrite last landed, or
`null` if one never has.

## Where this route does not work

A host that serves the response itself — Vercel, Netlify, GitHub Pages, S3,
Akamai NetStorage — has no process of yours to install into and nothing to put a
container in front of. `findxpand init` recognises the first two from the
checkout and refuses with exit 1, naming the two routes that do work: the
Cloudflare edge worker if you are behind Cloudflare, and a pull request against
your repository otherwise.

## Pushes are signed, not merely authenticated

Every manifest push carries `x-findxpand-signature` and `x-findxpand-timestamp`:
HMAC-SHA256 over `timestamp.body`, keyed by your token. Unsigned, altered or
stale pushes are refused with **401**.

The bearer token proves who sent a push and says nothing about what was in it.
Anything holding it — a logging proxy, a mirrored request, an old CI secret —
could otherwise replay a captured manifest indefinitely, or swap the manifest
inside one. The timestamp is inside the digest, so a captured push expires; the
window is 300 seconds, which means the clock on that host has to be roughly
right.

There is one supported reason to turn this off, and it is an intermediary that
rewrites or re-encodes request bodies, since that invalidates any signature over
them.

## Why it never calls us

It cannot. There is no outbound request anywhere in this package — the CLI's
`doctor` and `status` are the only two commands that open a socket at all, and
both of them talk to *your* origin. If Findxpand is unavailable, your site
serves the last manifest it received, indefinitely.

## Tests

```bash
python -m unittest discover -s tests
```

Standard library only, like the package. The conformance suite runs
`../conformance/cases.json` — the same fixture the Node middleware runs, so the
two cannot drift apart.
