genropy-asgi for Dummies

Quick Start

What you need first

• Python ≥ 3.11.
• A working GenroPy environment — ~/.gnr/environment.xml pointing at your GenroPy tree, the same file gnrwsgiserve needs.
• An existing site — genropy-asgi serves a site, it does not create one.
• psycopg2, if the site is on PostgreSQL (install GenroPy with its pgsql extra, or psycopg2-binary).

Install

# both from git, in this order — see the warning below
pip install git+https://github.com/genropy/genro-asgi.git
pip install git+https://github.com/genropy/genropy-asgi.git

# from checkouts, for development
pip install -e '<path>/genro-asgi'
pip install -e '<path>/genropy-asgi[dev]'

GenroPy itself is a runtime requirement: the worker builds a GnrWsgiSite and imports gnr.* when it runs, never at build time. Install it in the same virtualenv (with its pgsql extra for a PostgreSQL site).

Serve your site

gnrasgiserve mysite
# → http://127.0.0.1:8000/index  (the built-in defaults)

That is the drop-in. Common options:

gnrasgiserve mysite -p 9000                 # another port
gnrasgiserve mysite -H 0.0.0.0 -p 9000      # reachable from outside
gnrasgiserve mysite --nodebug               # debug off
gnrasgiserve mysite --workers 2             # a pool of 2 worker subprocesses

The two macOS variables you do NOT need

You will find PGGSSENCMODE=disable and OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES presented as mandatory in older notes about this stack. They are not, and the reason is structural: both defend against fork(), and nothing here forks. A worker is a brand-new interpreter —

subprocess.Popen([self.executable, "-m", WORKER_ENTRY_MODULE], env=env, start_new_session=True)

— and os.fork appears nowhere in genro-asgi or genropy-asgi. The child does not inherit a half-open libpq connection, so there is nothing to segfault. Verified by serving a two-worker pool with both variables removed from the environment: workers announced, /index and /metrics at 200, no crash.

Check it is really up

curl -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/index    # → 200
curl -s http://127.0.0.1:8000/metrics                                # → the site counters

genropy_site_counters{counter="users"} 2
genropy_site_counters{counter="pages"} 2
genropy_site_counters{counter="connections"} 2

Those three counters are the whole site as the front sees it, aggregated across every worker. They are Prometheus exposition format, so any scraper reads them as-is.

Try It: the test_invoice_pg bench

This is the environment the bridge was actually exercised on — a real GenroPy site with a real PostgreSQL database and 332 user accounts, 32 of which share a known password so a load harness (or a curious developer with several browser profiles) can log in as many different people. Everything below is checked out from repositories you already have, except the database itself.

1. The site: it ships inside GenroPy

The instance lives in the genropy repository and is tracked there — nothing to write by hand:

genropy/projects/test_invoice/
├── instances/
│   ├── test_invoice/        # the SQLite variant
│   └── test_invoice_pg/     # the PostgreSQL one — this is the bench
│       ├── instanceconfig.xml
│       ├── root.py
│       └── site/
└── packages/                # invc, adm, sys …

instanceconfig.xml points at a PostgreSQL database named test_invoice_pg on localhost, and it is the only one of those files the repository actually tracks.

2. The database: ask for a dump

The populated database is not in any repository — 35 MB of data, 332 rows in adm.adm_user. It travels as a gzipped plain-SQL pg_dump (~1.4 MB), which loads on any PostgreSQL version:

createdb test_invoice_pg
gunzip -c test_invoice_pg.sql.gz | psql -q -d test_invoice_pg

The data is the bench database as it stands, so the 32 accounts already carry the shared password — you can log in straight after loading, without touching the fixtures in the next step.

3. The accounts: 32 users, one password

The accounts come with the instance, but with passwords nobody knows. The genropy-asgi repository carries the fixture that makes them usable, in benchmarks/test_users.zip. The handed-over dump already has it applied — run this only if you rebuilt the database from somewhere else:

cd benchmarks
unzip -o test_users.zip
psql -d test_invoice_pg -f set_pwd_a.sql

They are fictional accounts on a throwaway database: invented names, @testinvoice.com addresses, and the hash of one letter as a password. Fine for a bench, obviously not for anything reachable from outside your machine.

4. Run it — single process

gnrasgiserve test_invoice_pg -p 8098 --nodebug

Open http://127.0.0.1:8098/index and log in as any name from usernames.txt with password a. It is the same site you would get from gnrwsgiserve — that is the point.

5. Run it — a pool of two workers

gnrasgiserve test_invoice_pg -p 8098 --workers 2 --nodebug

Now the front supervises two worker subprocesses and they announce themselves over a unix-domain socket. In the log you will see one line per worker:

INFO: W:b7f498d8…: serving on uds:/…/gnrhub_…/hub.sock (pid 17782)
INFO: W:811bb8c2…: serving on uds:/…/gnrhub_…/hub.sock (pid 17781)

The site behaves identically. What changed is that your first visit got a sticky_cid cookie, and every later request of yours goes back to the same worker. Log in from two different browser profiles and you may well be served by two different processes — the fastest way to feel what the pool does.

6. Watch what is happening

The counters, always available:

curl -s http://127.0.0.1:8098/metrics

The live monitor is served by genro-asgi at /_server/monitor/, and every one of its routes is gated SERVER_ADMIN. The built-in recipe declares no administrator, so out of the box the monitor answers 401. To open it, launch with a config that declares one — and, because the user store encrypts at rest, a storage key as well:

# monitor_config.py
import os
from genro_bag.resolvers import EnvResolver
from genro_storage import StorageManager
from genro_asgi.config import AsgiConfigBuilder
from genropy_asgi.spa import GenropySpaApplication

SITE = os.environ.get("GNR_ASGI_PATH") or "test_invoice_pg"
PORT = int(os.environ.get("GNR_ASGI_PORT") or 8098)

class ServerConfiguration(AsgiConfigBuilder):
    def main(self, root):
        cfg = root.configuration()
        cfg.server(host="127.0.0.1", port=PORT)
        cfg.middleware()
        cfg.storage(app=StorageManager, storage_key=EnvResolver("GENRO_STORAGE_KEY"))
        auth = cfg.authentication()
        auth.admin_password(EnvResolver("GNR_ADMIN_PASSWORD"))
        cfg.applications().application(
            code="site",
            app_class=GenropySpaApplication,
            source=SITE,
            debug=False,
            workers=2,
            local_worker=False,
        )

# a Fernet key for the at-rest encryption of the user store
export GENRO_STORAGE_KEY=$(python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")
export GNR_ADMIN_PASSWORD=<choose one>

gnrasgiserve test_invoice_pg --config monitor_config.py -p 8098

Then open http://127.0.0.1:8098/_server/login_page in the browser, sign in as admin with that password, and go to /_server/monitor/. From a script it is a form post:

curl -s -c jar.txt -d 'identity=admin&password=<pwd>' \
     http://127.0.0.1:8098/_server/login
curl -s -b jar.txt http://127.0.0.1:8098/_server/monitor/snapshot

7. The load harness, if you want numbers

benchmarks/ holds a stdlib-only harness: a faithful replay of a recorded browser session, driven at rising concurrency against a running instance. Nothing to install, and its README.md carries the recipe that works, including the traps. Run every script from inside that directory — they open their data files by relative name.

Single vs. Pool

Two shapes of the same application, chosen at launch. Here is exactly how they differ.

One process is the front and holds one worker. Right for development, for a handful of concurrent users, and as the exact drop-in for gnrwsgiserve: one thing to launch, one log to read, a debugger that sees everything. Note what it is not: it is not a special "simple mode" with its own code. It is the pool machinery with a worker that happens to live in the same process, so the protocol it exercises is the real one.

The front supervises N worker subprocesses, each hosting the same site with its own in-process register and its own thread pool. Because a synchronous GenroPy site saturates one process at a few busy users, running it in several processes multiplies throughput on the cores you already have. Each user is pinned to one worker, so the per-user and per-page state living in that worker's register stays coherent and local.

How a request finds its worker

• A request arrives carrying the sticky_cid cookie → the front hands it to the worker that owns that connection.
• No cookie yet (a fresh visitor) → the front sends it to the reception, the group's first routable worker, which mints the cookie on the first connection. Guests already live there, so a login happens where the guest is.
• On login the reception either keeps the user, or hands them to a less occupied worker — see Scaling.

What is coherent, and when

Per-user and per-page state lives in one worker's register and is immediately coherent there: there is no distributed lock, because there is nothing distributed to lock.

Global state — the legacy globalStore() — rides the framework's store rail and is eventually coherent: a write on one worker reaches the others after one channel round-trip, and a worker joining later is seeded when it announces. That suits what global state is really used for (cache-invalidation stamps, feature flags). If you write a value on worker 1 and read it on worker 2 in the same instant, you may read the old one. That is by design, not a defect.

Datachanges across workers

A record edited by one user must reach every page subscribed to that table — possibly on another worker. The queue is local to the worker that owns the page: a page drains its own pending list in its own process, never over the network. When a change has to cross, the front forwards it to the destination worker's /datachange_in endpoint, where it lands on the local queue like any other. Over-notifying a worker that does not subscribe is harmless — it is dropped at the fan-out.

How the Pool Scales

The front starts with the configured number of workers and grows the pool on measured pressure, not on head counts. There is no per-user cap: an idle user costs about a megabyte and no CPU, so counting users answers the wrong question. Every worker reports its occupancy — a number in 0..1 built from three components, cpu, executor and memory — and the front decides from that.

Placement: reception first

Every visitor is born on the reception — the first active worker of the pool, where the guests live, so a login happens where the guest already is. On login the reception keeps the user while its own occupancy is under reception_threshold (default 0.5); over that, the user goes to the least occupied of the other workers still under admission_threshold (default 0.8).

The login itself never ships anything: the worker announces the re-label and the user's slice stays where it is, so the request that carried the login keeps finding its pages to the end. The move, when there is one, is executed after that decision — not in the middle of it.

Scale-up: only when the group cannot place well

A new worker is spawned when the pool has nowhere good to put the next user — no non-reception worker under admission_threshold (or, in a pool of one, the reception itself over its keep-threshold). One hot worker with idle capacity elsewhere never triggers a spawn. A spawn already in flight is waited for, never stacked on top of another.

Scale-down: compaction on a capacity ledger

The front keeps a ledger: C, what the pool may take (the reception up to its threshold, a whole gate each for the others), against O, what it currently holds. A worker is taken out only while C - O, read without that worker, stays above compaction_margin (default 1.5). Its users are drained onto the survivors and it retires. A drain that does not empty retires nothing: a worker still holding state is never killed. min_workers is the floor (default 1) and the reception is never compacted, so guests always have a home.

Reading the margin without the condemned worker is what gives the hysteresis: scale-up and scale-down cannot chase each other.

The knobs

The one timing caveat

A fresh worker takes a few seconds to boot a full GnrWsgiSite. Under a genuine burst — logins arriving faster than a worker can announce — the extra logins pile onto the last worker until the new one is ready, then routing resumes. This is deliberate: the front never stacks a second spawn while one is in flight, because doing so under a load test is how you end up with a dozen workers and no memory.

Watching it from outside

Two windows, and it is worth knowing exactly what each one shows:

The Daemon Is Gone

A GenroPy site talks to its register for every piece of shared state: which connections and pages exist, who is logged in, the datachanges pending for a browser, the page/user/connection/global stores. It reaches all of it through site.register, one command at a time.

Historically that register was a daemon — a separate process on a wire, Pyro4 first, later the genro-nodaemon TCP daemon. Every command was serialised onto that wire, and the daemon was one more thing to launch, supervise and debug.

genropy-asgi removes the wire. GenropyRegisterClient answers every site.register command from the hosting application's own registries, surface and stores — in the same process, no serialisation, no connection at construction.

How the legacy still finds it

The package declares a gnr.web:daemon entry point. The legacy gnr.web.daemon switcher imports the module named there and installs it as gnr.web.daemon, so every legacy import resolves to genropy-asgi's own module — with nothing behind it. The site builds its register directly at site.register, with no connection to open.

# pyproject.toml — this is the whole wiring
[project.entry-points."gnr.web"]
daemon = "genropy_asgi.siteregister"

The switch model, in one paragraph

GenroPy pushes datachanges to browsers: a record edited by one user must reach every page subscribed to that table, wherever it lives. Each worker owns the queue of the pages it holds; a page drains its own pending list in its own process. When a change must cross to a page on another worker, the front forwards it to that worker's /datachange_in endpoint, where it lands on the local queue like any other. No page ever pulls over the network, and no central process holds the queues.

One Server, Many Apps

The legacy site is not the only thing the server can host: it is a multi-app ASGI server. There are two ways to grow native ASGI surface next to the site, and both stay in one process, on one port, on one origin, against the same GenroPy database.

• Beside — declare separate applications, each on its own path prefix (/api/, /mcp/…).
• In place — add native routes to the site application itself, shadowing individual paths one at a time. This is the incremental-migration path.

Beside: a REST / OpenAPI surface on the same database

GenropyProxyOpenApiApplication hosts a GnrApp behind an OpenAPI application: your routing class exposes plain methods as REST, and the mixin closes the database connection on the right (executor) thread after each call. Point it at the same instance the site serves and you have a second surface on the same data.

# in the config recipe, beside the site
from genropy_asgi.proxy import GenropyProxyOpenApiApplication
from genropy_asgi.spa import GenropySpaApplication
from myproject.api import MyApi

apps = cfg.applications()
apps.application(code="site", mount="", app_class=GenropySpaApplication,
                 source=SITE, workers=2, local_worker=False)
apps.application(code="api", mount="api",           # → /api/
                 app_class=GenropyProxyOpenApiApplication,
                 instance=SITE,                    # the same GenroPy instance
                 routing_class=MyApi(),
                 docs="swagger")                   # Swagger UI under _meta/docs

The same class, exposed to AI agents

One routing class can serve both a REST client and an MCP client: McpOpenApiApplication points the MCP engine at the same router, so there is no second implementation to keep in sync.

from genro_asgi import McpOpenApiApplication

apps.application(code="mcp", mount="mcp",
                 app_class=McpOpenApiApplication,
                 routing_class=MyApi(), api_name="tools")

In place: replace one site path at a time

The apps above live under their own prefixes. The in-place pattern instead adds native routes to the site application, on paths the site already owns, and each one shadows the legacy handler for exactly its own path. The native surface grows while the site keeps serving everything not yet moved: no second deployment, no cut-over day.

from genro_routes import route
from genropy_asgi.spa import GenropySpaApplication

class MySite(GenropySpaApplication):

    @route(media_type="application/json")
    def sys_health(self):        # /sys/health is native from now on…
        return {"status": "ok"}
        # …/sys/customer, /sys/order, … still render on the legacy site.

Then point app_class at MySite in the recipe. This is the same seam the built-in /metrics route uses — it is a @route on the site application, nothing more.

It works because the host demultiplexes in two stages: the first path segment picks an internal root, then the full path is resolved in the application's own router. A structural miss inside a claimed root falls through to the legacy site — so claiming a root does not claim its whole subtree, and a single native route shadows only its own path.

Why "same origin" is the whole point

Because every application lives under one host and port, the browser sees one origin: no CORS to configure, and the legacy session cookie is sent to every path. A legacy GenroPy page can therefore reach a new ASGI endpoint directly — a fetch("/api/…") from page code, or an <iframe src="/live/…"> embedding a modern view inside the classic UI — with the user's session already authenticated.

Config & CLI

Every gnrasgiserve option

The environment variables the built-in recipe reads

The CLI writes these before the server is built, which is why a custom config that reads GNR_ASGI_PATH still serves the instance named on the command line.

When you need a config file

--workers starts a pool on the framework's default thresholds. Write a config to tune them, to bound the pool, or to declare more applications. A configuration is a ServerConfiguration subclassing genro-asgi's AsgiConfigBuilder, and its main() writes one document: the listener, the middleware, then the applications.

# pool_config.py — the shipped example lives in examples/multiworker_config.py
import os
from genro_asgi.config import AsgiConfigBuilder
from genropy_asgi.spa import GenropySpaApplication

SITE = os.environ.get("GNR_ASGI_PATH") or "mysite"
PORT = int(os.environ.get("GNR_ASGI_PORT") or 8081)

class ServerConfiguration(AsgiConfigBuilder):
    def main(self, root):
        cfg = root.configuration()
        cfg.server(host="127.0.0.1", port=PORT)
        cfg.middleware()
        cfg.applications().application(
            code="site",
            mount="",                       # the site owns its absolute URLs
            app_class=GenropySpaApplication,
            source=SITE,
            debug=False,
            workers=2,                     # initial pool size
            local_worker=False,           # True (with workers=0) is the single
            min_workers=1,                 # compaction floor
            max_workers=None,              # None = unbounded scale-up
            reception_threshold=0.5,
            admission_threshold=0.8,
            compaction_margin=1.5,
            memory_limit_mb=None,          # None = derived from host RAM
        )

Launch it either way — the config brings the shape, the CLI instance wins:

gnrasgiserve mysite --config pool_config.py -p 8081
python -m genro_asgi serve pool_config.py

Cheat Sheet

I want to…

The moving parts

Gotchas

Not there yet

So you do not go looking for what is not built:

• Worker groups — several versions of the site behind one front, each on its own interpreter, users routed by avatar. Designed, not in the code: there is one pool.
• The placement view over HTTP — which user sits on which worker. The front knows; nothing publishes it yet, and the monitor shows the generic panel.
• A reloader — the core server has none, and --reload says so out loud.

The orchestration layer inside genro-asgi is being rebuilt as this is written: the machine described here is the one that serves traffic today, and it keeps serving it while the new one is assembled beside it.