Metadata-Version: 2.4
Name: xtransit
Version: 0.1.0a1
Summary: Collect GTFS transit feeds, retain their history in PostgreSQL, and explore routes and vehicles on a map.
Project-URL: Documentation, https://github.com/ysenarath/xtransit#readme
Project-URL: Issues, https://github.com/ysenarath/xtransit/issues
Project-URL: Source, https://github.com/ysenarath/xtransit
Author-email: Yasas Senarath <12231659+ysenarath@users.noreply.github.com>
License-Expression: MIT AND BSD-2-Clause
License-File: LICENSE.txt
License-File: src/xtransit/web/static/vendor/LICENSE
Keywords: gtfs,gtfs-realtime,maps,postgresql,transit,transportation
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: Implementation :: CPython
Requires-Python: >=3.9
Requires-Dist: asyncpg<1,>=0.30.0
Requires-Dist: docker>=7.2.0
Requires-Dist: httpx<1,>=0.27
Requires-Dist: kafka-python>=3.0.11
Requires-Dist: psycopg2-binary>=2.9.12
Requires-Dist: pydantic>=2.13.5
Requires-Dist: sqlalchemy[asyncio]>=2.0.52
Provides-Extra: data
Requires-Dist: gtfs-realtime-bindings<3,>=1.0.0; extra == 'data'
Provides-Extra: web
Requires-Dist: fastapi<1,>=0.115; extra == 'web'
Requires-Dist: uvicorn<1,>=0.30; extra == 'web'
Description-Content-Type: text/markdown

# xtransit

Collect public transit data, keep its history, and explore it on a map.

xtransit brings together **GTFS schedules** (routes, stops, and timetables) and
**GTFS Realtime feeds** (vehicle positions, arrival updates, and alerts). It finds
feeds in the MobilityDatabase catalog, saves them to PostgreSQL, and provides a
small web app for viewing routes and following vehicles across multiple networks.

This is an early alpha. It is useful for local exploration and building a transit
dataset; commands and database schemas may change between releases.

## What you can do

- Download public schedules and realtime snapshots for selected networks or the U.S.
- Keep collecting arrivals and vehicle positions, with recovery after interrupted imports.
- View routes, stops, current vehicle positions, and recorded trails on a map.
- Select several networks at once, including networks that overlap the visible map.
- Query the collected history with SQL or Python.

## Get started

You need **Python 3.9+**, **curl**, and a running **Docker** daemon or a
**Podman Docker-compatible socket**. PostgreSQL runs in a container that xtransit
creates and reuses. The download-only commands do not need a database.

Install the first alpha in a virtual environment:

```sh
python3 -m venv .venv
source .venv/bin/activate
python -m pip install "xtransit[data,web]==0.1.0a1"
```

The commands here use a macOS/Linux shell. Run the following steps from the same
directory; downloads and capture files are saved under `data/` there.

### 1. Download a small network

Start with Arlington Transit in Virginia. This downloads its schedule, arrival
updates, and vehicle positions:

```sh
xtransit --feed-id mdb-485 --feed-id mdb-1372 --feed-id mdb-1790
```

These are catalog feed IDs, not route numbers. To find other feeds, run
`xtransit --catalog-only` and browse `data/us/catalog.csv`.

### 2. Start the database and import the downloads

```sh
xtransit-db
xtransit-ingest import-files
```

The database command creates or reuses a local PostgreSQL instance and checks the
connection. Importing the same files again is safe.

### 3. Open the map and keep collecting

```sh
xtransit-web --poll-feed mdb-1790 --poll-feed mdb-1372
```

Open **http://127.0.0.1:8000**. Leave the terminal running to keep collecting new
positions and arrival updates. Press **Ctrl+C** to stop the web app and its pollers;
the database and collected history remain available.

The app polls these feeds every 30 seconds and refreshes the map every 15 seconds.
Vehicles appear when the agency is publishing positions; an empty map can mean
there is no current service. Check `xtransit-ingest status` if imports failed.

## Explore the map

Open **Your networks** to search for and select multiple networks. Click a route
to see its path, or a vehicle to view its latest observation and a 1-, 6-, or
24-hour trail. **Network stops** adds stops from the selected networks.

To select networks geographically, pan or zoom and click **Select networks in
view**. A network only needs to overlap the map; it does not need to fit entirely
inside it. **In map area only** filters the network picker as you move the map.
Coverage comes from catalog bounding boxes, so a match does not guarantee a route
passes through every part of the visible area.

**Fresh only** shows positions observed within the past 90 seconds. Older
positions appear in amber. Trails contain the observations you have collected;
they do not reconstruct trips from before collection began.

The API documentation is available at **http://127.0.0.1:8000/docs**.

## Common commands

| Task | Command |
| --- | --- |
| List available U.S. feeds | `xtransit --catalog-only` |
| Download one schedule | `xtransit --kind schedule --feed-id mdb-485` |
| Refresh that schedule | `xtransit --kind schedule --feed-id mdb-485 --refresh` |
| Import saved downloads | `xtransit-ingest import-files` |
| Check imports and errors | `xtransit-ingest status` |
| Recover pending captures | `xtransit-ingest replay` |
| Open the map without polling | `xtransit-web` |

To keep collecting separately from the web app:

```sh
xtransit-ingest poll --feed-id mdb-1372 --feed-id mdb-1790 --cycles 0
```

Without `--cycles 0`, polling runs once. Use `--help` on the collector, importer,
or web command for more options.

## Collect more networks

Repeat `--feed-id` to add feeds. Include a network's schedule and its corresponding
realtime feeds so the map can connect vehicle observations to routes.

For a nationwide collection, the following downloads public feeds, imports them,
and then continuously polls the available realtime feeds:

```sh
xtransit
xtransit-ingest import-files
xtransit-ingest --workers 8 poll --cycles 0
```

**Start small if you are trying the project for the first time.** Nationwide
schedules can occupy tens of gigabytes after import, and realtime history keeps
growing. This version retains history and capture files indefinitely; it does
not automatically archive or delete old data.

## Data coverage and reliability

xtransit uses the public [MobilityDatabase catalog](https://mobilitydatabase.org/faq).
U.S. collection covers active catalog entries marked `US`, not every transit
operator. Feeds requiring agency credentials are reported rather than fetched;
public schedule mirrors may still be available.

Realtime means the **latest observation the agency published and we successfully
collected**. It is not a continuous GPS stream. Agency delays, outages, and the
polling interval all affect freshness.

The importer saves captured responses before processing them, deduplicates repeated
data, and updates the current snapshot only after an import succeeds. Failed
imports can be replayed; invalid feeds are retained for inspection. Polling cannot
recover observations that an agency replaced between requests. Back up the database
and capture files if you need to protect against disk loss.

## Where the data lives

| Location | Contents |
| --- | --- |
| `data/us/` | Feed catalog, downloaded schedules, snapshots, and run reports |
| `data/ingest-spool/` | Durable captures used to recover interrupted imports |
| `~/.xtransit/postgres.json` | Local database settings and generated credentials |
| `~/.xtransit/postgresql/` | Persistent PostgreSQL data |

Set `XT_HOME` before starting xtransit to change the database storage location.
The default database port is `127.0.0.1:55432`.

See the [operations guide](https://github.com/ysenarath/xtransit/blob/main/docs/operations.md)
for SQL examples, database configuration, polling behavior, and recovery details.

## Development

```sh
git clone https://github.com/ysenarath/xtransit.git
cd xtransit
uv sync --extra data --extra web
uv run --extra data --extra web python -m unittest discover -s tests -v
node --test tests/web_networks.test.mjs
```

Node.js 18+ is needed only for the frontend tests. To run the database integration
tests too, start Docker or Podman and use:

```sh
XTRANSIT_TEST_DATABASE=1 uv run --extra data --extra web python -m unittest discover -s tests -v
```

Bug reports and contributions are welcome through
[GitHub issues](https://github.com/ysenarath/xtransit/issues).

## License and attribution

xtransit's code is [MIT licensed](https://github.com/ysenarath/xtransit/blob/main/LICENSE.txt).
Bundled Leaflet assets retain their BSD 2-Clause license. Transit feeds keep their
providers' own licenses; see the catalog for each feed's terms.

Map tiles come from OpenStreetMap and need an internet connection. Follow the
[OpenStreetMap tile usage policy](https://operations.osmfoundation.org/policies/tiles/)
when deploying a public viewer.
