Metadata-Version: 2.4
Name: meerschaum
Version: 3.4.0
Summary: Sync Time-Series Pipes with Meerschaum
Author-email: Bennett Meares <bennett.meares@gmail.com>
Maintainer-email: Bennett Meares <bennett.meares@gmail.com>
License-Expression: Apache-2.0
Project-URL: Documentation, https://docs.meerschaum.io
Project-URL: Changelog, https://meerschaum.io/news/changelog
Project-URL: GitHub, https://github.com/bmeares/Meerschaum
Project-URL: Homepage, https://meerschaum.io
Project-URL: Donate, https://github.com/sponsors/bmeares
Project-URL: Discord, https://discord.gg/8U8qMUjvcc
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: MacOS
Classifier: Programming Language :: SQL
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Database
Classifier: Natural Language :: English
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Provides-Extra: required
Provides-Extra: minimal
Provides-Extra: formatting
Requires-Dist: pprintpp>=0.4.0; extra == "formatting"
Requires-Dist: asciitree>=0.3.3; extra == "formatting"
Requires-Dist: typing-extensions>=4.7.1; extra == "formatting"
Requires-Dist: pygments>=2.7.2; extra == "formatting"
Requires-Dist: colorama>=0.4.3; extra == "formatting"
Requires-Dist: rich>=13.4.2; extra == "formatting"
Requires-Dist: more-termcolor>=1.1.3; extra == "formatting"
Requires-Dist: humanfriendly>=10.0.0; extra == "formatting"
Requires-Dist: tabulate>=0.9.0; extra == "formatting"
Provides-Extra: core
Requires-Dist: wheel>=0.34.2; extra == "core"
Requires-Dist: setuptools>=63.3.0; extra == "core"
Requires-Dist: PyYAML>=5.3.1; extra == "core"
Requires-Dist: pip>=22.0.4; extra == "core"
Requires-Dist: update-checker>=0.18.0; extra == "core"
Requires-Dist: semver>=3.0.2; extra == "core"
Requires-Dist: pathspec>=0.9.0; extra == "core"
Requires-Dist: python-dateutil>=2.7.5; extra == "core"
Requires-Dist: requests>=2.32.5; extra == "core"
Requires-Dist: certifi>=2024.8.30; extra == "core"
Requires-Dist: idna>=3.10.0; extra == "core"
Requires-Dist: binaryornot>=0.4.4; extra == "core"
Requires-Dist: pyvim>=3.0.2; extra == "core"
Requires-Dist: ptpython>=3.0.27; extra == "core"
Requires-Dist: aiofiles>=0.6.0; extra == "core"
Requires-Dist: packaging>=21.3.0; extra == "core"
Requires-Dist: prompt-toolkit>=3.0.39; extra == "core"
Requires-Dist: more-itertools>=8.7.0; extra == "core"
Requires-Dist: fasteners>=0.19.0; extra == "core"
Requires-Dist: virtualenv>=20.1.0; extra == "core"
Requires-Dist: attrs>=24.2.0; extra == "core"
Requires-Dist: uv>=0.2.11; extra == "core"
Requires-Dist: pydantic>=2.11.7; extra == "core"
Requires-Dist: annotated-types>=0.7.0; extra == "core"
Provides-Extra: jobs
Requires-Dist: dill>=0.4.0; extra == "jobs"
Requires-Dist: python-daemon>=3.1.2; extra == "jobs"
Requires-Dist: watchfiles>=1.1.0; extra == "jobs"
Requires-Dist: psutil>=7.0.0; extra == "jobs"
Provides-Extra: drivers
Requires-Dist: cryptography>=38.0.1; extra == "drivers"
Requires-Dist: psycopg[binary]>=3.2.9; extra == "drivers"
Requires-Dist: psycopg2-binary>=2.9.11; extra == "drivers"
Requires-Dist: PyMySQL>=0.9.0; extra == "drivers"
Requires-Dist: aiomysql>=0.0.21; extra == "drivers"
Requires-Dist: sqlalchemy-cockroachdb>=2.0.0; extra == "drivers"
Requires-Dist: duckdb>=1.0.0; extra == "drivers"
Requires-Dist: duckdb-engine>=0.13.0; extra == "drivers"
Provides-Extra: drivers-extras
Requires-Dist: pyodbc>=4.0.30; extra == "drivers-extras"
Requires-Dist: oracledb>=2.5.0; extra == "drivers-extras"
Provides-Extra: cli
Requires-Dist: pgcli>=3.1.0; extra == "cli"
Requires-Dist: mycli>=1.23.2; extra == "cli"
Requires-Dist: litecli>=1.5.0; extra == "cli"
Requires-Dist: mssql-cli>=1.0.0; extra == "cli"
Requires-Dist: gadwall>=0.2.0; extra == "cli"
Provides-Extra: gis
Requires-Dist: pyproj>=3.7.1; extra == "gis"
Requires-Dist: geopandas>=1.0.1; extra == "gis"
Requires-Dist: shapely>=2.0.7; extra == "gis"
Provides-Extra: stack
Requires-Dist: docker-compose>=1.29.2; extra == "stack"
Provides-Extra: build
Requires-Dist: cx_Freeze>=7.0.0; extra == "build"
Requires-Dist: pyinstaller>6.6.0; extra == "build"
Provides-Extra: dev-tools
Requires-Dist: twine>=3.2.0; extra == "dev-tools"
Requires-Dist: tuna>=0.5.3; extra == "dev-tools"
Requires-Dist: snakeviz>=2.1.0; extra == "dev-tools"
Requires-Dist: mypy>=0.812.0; extra == "dev-tools"
Requires-Dist: pytest>=6.2.2; extra == "dev-tools"
Requires-Dist: pytest-xdist>=3.2.1; extra == "dev-tools"
Requires-Dist: heartrate>=0.2.1; extra == "dev-tools"
Requires-Dist: build>=1.2.1; extra == "dev-tools"
Requires-Dist: attrs>=24.2.0; extra == "dev-tools"
Provides-Extra: setup
Provides-Extra: docs
Requires-Dist: zensical>=0.0.30; extra == "docs"
Provides-Extra: gui
Requires-Dist: pywebview>=3.6.3; extra == "gui"
Requires-Dist: pycparser>=2.21.0; extra == "gui"
Provides-Extra: extras
Requires-Dist: cmd2>=1.4.0; extra == "extras"
Requires-Dist: ruamel.yaml>=0.16.12; extra == "extras"
Requires-Dist: modin[ray]>=0.8.3; extra == "extras"
Requires-Dist: nanoid>=2.0.0; extra == "extras"
Requires-Dist: importlib-metadata>=4.12.0; extra == "extras"
Provides-Extra: sql
Requires-Dist: numpy>=2.3.1; extra == "sql"
Requires-Dist: pandas[parquet]>=3.0.0; extra == "sql"
Requires-Dist: pyarrow>=20.0.0; extra == "sql"
Requires-Dist: dask[complete]>=2024.12.1; extra == "sql"
Requires-Dist: partd>=1.4.2; extra == "sql"
Requires-Dist: pytz; extra == "sql"
Requires-Dist: joblib>=1.5.1; extra == "sql"
Requires-Dist: SQLAlchemy>=2.0.41; extra == "sql"
Requires-Dist: GeoAlchemy2>=0.18.0; extra == "sql"
Requires-Dist: databases>=0.9.0; extra == "sql"
Requires-Dist: aiosqlite>=0.21.0; extra == "sql"
Requires-Dist: asyncpg>=0.30.0; extra == "sql"
Requires-Dist: cryptography>=38.0.1; extra == "sql"
Requires-Dist: psycopg[binary]>=3.2.9; extra == "sql"
Requires-Dist: psycopg2-binary>=2.9.11; extra == "sql"
Requires-Dist: PyMySQL>=0.9.0; extra == "sql"
Requires-Dist: aiomysql>=0.0.21; extra == "sql"
Requires-Dist: sqlalchemy-cockroachdb>=2.0.0; extra == "sql"
Requires-Dist: duckdb>=1.0.0; extra == "sql"
Requires-Dist: duckdb-engine>=0.13.0; extra == "sql"
Requires-Dist: wheel>=0.34.2; extra == "sql"
Requires-Dist: setuptools>=63.3.0; extra == "sql"
Requires-Dist: PyYAML>=5.3.1; extra == "sql"
Requires-Dist: pip>=22.0.4; extra == "sql"
Requires-Dist: update-checker>=0.18.0; extra == "sql"
Requires-Dist: semver>=3.0.2; extra == "sql"
Requires-Dist: pathspec>=0.9.0; extra == "sql"
Requires-Dist: python-dateutil>=2.7.5; extra == "sql"
Requires-Dist: requests>=2.32.5; extra == "sql"
Requires-Dist: certifi>=2024.8.30; extra == "sql"
Requires-Dist: idna>=3.10.0; extra == "sql"
Requires-Dist: binaryornot>=0.4.4; extra == "sql"
Requires-Dist: pyvim>=3.0.2; extra == "sql"
Requires-Dist: ptpython>=3.0.27; extra == "sql"
Requires-Dist: aiofiles>=0.6.0; extra == "sql"
Requires-Dist: packaging>=21.3.0; extra == "sql"
Requires-Dist: prompt-toolkit>=3.0.39; extra == "sql"
Requires-Dist: more-itertools>=8.7.0; extra == "sql"
Requires-Dist: fasteners>=0.19.0; extra == "sql"
Requires-Dist: virtualenv>=20.1.0; extra == "sql"
Requires-Dist: attrs>=24.2.0; extra == "sql"
Requires-Dist: uv>=0.2.11; extra == "sql"
Requires-Dist: pydantic>=2.11.7; extra == "sql"
Requires-Dist: annotated-types>=0.7.0; extra == "sql"
Requires-Dist: pyproj>=3.7.1; extra == "sql"
Requires-Dist: geopandas>=1.0.1; extra == "sql"
Requires-Dist: shapely>=2.0.7; extra == "sql"
Provides-Extra: dash
Requires-Dist: Flask-Compress>=1.17.0; extra == "dash"
Requires-Dist: dash>=4.1.0; extra == "dash"
Requires-Dist: dash-bootstrap-components>=2.0.4; extra == "dash"
Requires-Dist: dash-ace>=0.2.1; extra == "dash"
Requires-Dist: dash-extensions>=2.0.5; extra == "dash"
Requires-Dist: dash-daq>=0.6.0; extra == "dash"
Requires-Dist: terminado>=0.18.1; extra == "dash"
Requires-Dist: tornado>=6.5.1; extra == "dash"
Provides-Extra: api
Requires-Dist: uvicorn[standard]>=0.35.0; extra == "api"
Requires-Dist: gunicorn>=23.0.0; extra == "api"
Requires-Dist: python-dotenv>=1.1.1; extra == "api"
Requires-Dist: websockets>=15.0.1; extra == "api"
Requires-Dist: fastapi>=0.128.0; extra == "api"
Requires-Dist: fastapi-login>=1.10.3; extra == "api"
Requires-Dist: python-multipart>=0.0.20; extra == "api"
Requires-Dist: httpx>=0.28.1; extra == "api"
Requires-Dist: httpcore>=1.0.9; extra == "api"
Requires-Dist: valkey>=6.1.0; extra == "api"
Requires-Dist: python-jose>=3.5.0; extra == "api"
Requires-Dist: numpy>=2.3.1; extra == "api"
Requires-Dist: pandas[parquet]>=3.0.0; extra == "api"
Requires-Dist: pyarrow>=20.0.0; extra == "api"
Requires-Dist: dask[complete]>=2024.12.1; extra == "api"
Requires-Dist: partd>=1.4.2; extra == "api"
Requires-Dist: pytz; extra == "api"
Requires-Dist: joblib>=1.5.1; extra == "api"
Requires-Dist: SQLAlchemy>=2.0.41; extra == "api"
Requires-Dist: GeoAlchemy2>=0.18.0; extra == "api"
Requires-Dist: databases>=0.9.0; extra == "api"
Requires-Dist: aiosqlite>=0.21.0; extra == "api"
Requires-Dist: asyncpg>=0.30.0; extra == "api"
Requires-Dist: cryptography>=38.0.1; extra == "api"
Requires-Dist: psycopg[binary]>=3.2.9; extra == "api"
Requires-Dist: psycopg2-binary>=2.9.11; extra == "api"
Requires-Dist: PyMySQL>=0.9.0; extra == "api"
Requires-Dist: aiomysql>=0.0.21; extra == "api"
Requires-Dist: sqlalchemy-cockroachdb>=2.0.0; extra == "api"
Requires-Dist: duckdb>=1.0.0; extra == "api"
Requires-Dist: duckdb-engine>=0.13.0; extra == "api"
Requires-Dist: wheel>=0.34.2; extra == "api"
Requires-Dist: setuptools>=63.3.0; extra == "api"
Requires-Dist: PyYAML>=5.3.1; extra == "api"
Requires-Dist: pip>=22.0.4; extra == "api"
Requires-Dist: update-checker>=0.18.0; extra == "api"
Requires-Dist: semver>=3.0.2; extra == "api"
Requires-Dist: pathspec>=0.9.0; extra == "api"
Requires-Dist: python-dateutil>=2.7.5; extra == "api"
Requires-Dist: requests>=2.32.5; extra == "api"
Requires-Dist: certifi>=2024.8.30; extra == "api"
Requires-Dist: idna>=3.10.0; extra == "api"
Requires-Dist: binaryornot>=0.4.4; extra == "api"
Requires-Dist: pyvim>=3.0.2; extra == "api"
Requires-Dist: ptpython>=3.0.27; extra == "api"
Requires-Dist: aiofiles>=0.6.0; extra == "api"
Requires-Dist: packaging>=21.3.0; extra == "api"
Requires-Dist: prompt-toolkit>=3.0.39; extra == "api"
Requires-Dist: more-itertools>=8.7.0; extra == "api"
Requires-Dist: fasteners>=0.19.0; extra == "api"
Requires-Dist: virtualenv>=20.1.0; extra == "api"
Requires-Dist: attrs>=24.2.0; extra == "api"
Requires-Dist: uv>=0.2.11; extra == "api"
Requires-Dist: pydantic>=2.11.7; extra == "api"
Requires-Dist: annotated-types>=0.7.0; extra == "api"
Requires-Dist: pyproj>=3.7.1; extra == "api"
Requires-Dist: geopandas>=1.0.1; extra == "api"
Requires-Dist: shapely>=2.0.7; extra == "api"
Requires-Dist: pprintpp>=0.4.0; extra == "api"
Requires-Dist: asciitree>=0.3.3; extra == "api"
Requires-Dist: typing-extensions>=4.7.1; extra == "api"
Requires-Dist: pygments>=2.7.2; extra == "api"
Requires-Dist: colorama>=0.4.3; extra == "api"
Requires-Dist: rich>=13.4.2; extra == "api"
Requires-Dist: more-termcolor>=1.1.3; extra == "api"
Requires-Dist: humanfriendly>=10.0.0; extra == "api"
Requires-Dist: tabulate>=0.9.0; extra == "api"
Requires-Dist: Flask-Compress>=1.17.0; extra == "api"
Requires-Dist: dash>=4.1.0; extra == "api"
Requires-Dist: dash-bootstrap-components>=2.0.4; extra == "api"
Requires-Dist: dash-ace>=0.2.1; extra == "api"
Requires-Dist: dash-extensions>=2.0.5; extra == "api"
Requires-Dist: dash-daq>=0.6.0; extra == "api"
Requires-Dist: terminado>=0.18.1; extra == "api"
Requires-Dist: tornado>=6.5.1; extra == "api"
Requires-Dist: dill>=0.4.0; extra == "api"
Requires-Dist: python-daemon>=3.1.2; extra == "api"
Requires-Dist: watchfiles>=1.1.0; extra == "api"
Requires-Dist: psutil>=7.0.0; extra == "api"
Provides-Extra: full
Requires-Dist: pprintpp>=0.4.0; extra == "full"
Requires-Dist: asciitree>=0.3.3; extra == "full"
Requires-Dist: typing-extensions>=4.7.1; extra == "full"
Requires-Dist: pygments>=2.7.2; extra == "full"
Requires-Dist: colorama>=0.4.3; extra == "full"
Requires-Dist: rich>=13.4.2; extra == "full"
Requires-Dist: more-termcolor>=1.1.3; extra == "full"
Requires-Dist: humanfriendly>=10.0.0; extra == "full"
Requires-Dist: tabulate>=0.9.0; extra == "full"
Requires-Dist: wheel>=0.34.2; extra == "full"
Requires-Dist: setuptools>=63.3.0; extra == "full"
Requires-Dist: PyYAML>=5.3.1; extra == "full"
Requires-Dist: pip>=22.0.4; extra == "full"
Requires-Dist: update-checker>=0.18.0; extra == "full"
Requires-Dist: semver>=3.0.2; extra == "full"
Requires-Dist: pathspec>=0.9.0; extra == "full"
Requires-Dist: python-dateutil>=2.7.5; extra == "full"
Requires-Dist: requests>=2.32.5; extra == "full"
Requires-Dist: certifi>=2024.8.30; extra == "full"
Requires-Dist: idna>=3.10.0; extra == "full"
Requires-Dist: binaryornot>=0.4.4; extra == "full"
Requires-Dist: pyvim>=3.0.2; extra == "full"
Requires-Dist: ptpython>=3.0.27; extra == "full"
Requires-Dist: aiofiles>=0.6.0; extra == "full"
Requires-Dist: packaging>=21.3.0; extra == "full"
Requires-Dist: prompt-toolkit>=3.0.39; extra == "full"
Requires-Dist: more-itertools>=8.7.0; extra == "full"
Requires-Dist: fasteners>=0.19.0; extra == "full"
Requires-Dist: virtualenv>=20.1.0; extra == "full"
Requires-Dist: attrs>=24.2.0; extra == "full"
Requires-Dist: uv>=0.2.11; extra == "full"
Requires-Dist: pydantic>=2.11.7; extra == "full"
Requires-Dist: annotated-types>=0.7.0; extra == "full"
Requires-Dist: dill>=0.4.0; extra == "full"
Requires-Dist: python-daemon>=3.1.2; extra == "full"
Requires-Dist: watchfiles>=1.1.0; extra == "full"
Requires-Dist: psutil>=7.0.0; extra == "full"
Requires-Dist: cryptography>=38.0.1; extra == "full"
Requires-Dist: psycopg[binary]>=3.2.9; extra == "full"
Requires-Dist: psycopg2-binary>=2.9.11; extra == "full"
Requires-Dist: PyMySQL>=0.9.0; extra == "full"
Requires-Dist: aiomysql>=0.0.21; extra == "full"
Requires-Dist: sqlalchemy-cockroachdb>=2.0.0; extra == "full"
Requires-Dist: duckdb>=1.0.0; extra == "full"
Requires-Dist: duckdb-engine>=0.13.0; extra == "full"
Requires-Dist: pyproj>=3.7.1; extra == "full"
Requires-Dist: geopandas>=1.0.1; extra == "full"
Requires-Dist: shapely>=2.0.7; extra == "full"
Requires-Dist: pywebview>=3.6.3; extra == "full"
Requires-Dist: pycparser>=2.21.0; extra == "full"
Requires-Dist: numpy>=2.3.1; extra == "full"
Requires-Dist: pandas[parquet]>=3.0.0; extra == "full"
Requires-Dist: pyarrow>=20.0.0; extra == "full"
Requires-Dist: dask[complete]>=2024.12.1; extra == "full"
Requires-Dist: partd>=1.4.2; extra == "full"
Requires-Dist: pytz; extra == "full"
Requires-Dist: joblib>=1.5.1; extra == "full"
Requires-Dist: SQLAlchemy>=2.0.41; extra == "full"
Requires-Dist: GeoAlchemy2>=0.18.0; extra == "full"
Requires-Dist: databases>=0.9.0; extra == "full"
Requires-Dist: aiosqlite>=0.21.0; extra == "full"
Requires-Dist: asyncpg>=0.30.0; extra == "full"
Requires-Dist: Flask-Compress>=1.17.0; extra == "full"
Requires-Dist: dash>=4.1.0; extra == "full"
Requires-Dist: dash-bootstrap-components>=2.0.4; extra == "full"
Requires-Dist: dash-ace>=0.2.1; extra == "full"
Requires-Dist: dash-extensions>=2.0.5; extra == "full"
Requires-Dist: dash-daq>=0.6.0; extra == "full"
Requires-Dist: terminado>=0.18.1; extra == "full"
Requires-Dist: tornado>=6.5.1; extra == "full"
Requires-Dist: uvicorn[standard]>=0.35.0; extra == "full"
Requires-Dist: gunicorn>=23.0.0; extra == "full"
Requires-Dist: python-dotenv>=1.1.1; extra == "full"
Requires-Dist: websockets>=15.0.1; extra == "full"
Requires-Dist: fastapi>=0.128.0; extra == "full"
Requires-Dist: fastapi-login>=1.10.3; extra == "full"
Requires-Dist: python-multipart>=0.0.20; extra == "full"
Requires-Dist: httpx>=0.28.1; extra == "full"
Requires-Dist: httpcore>=1.0.9; extra == "full"
Requires-Dist: valkey>=6.1.0; extra == "full"
Requires-Dist: python-jose>=3.5.0; extra == "full"
Dynamic: license-file
Dynamic: provides-extra

<img src="https://meerschaum.io/assets/banner_1920x320.png" alt="Meerschaum banner" style="width: 100%"/>

| PyPI | GitHub | Info | Stats |
|---|---|---|---|
| ![PyPI]( https://img.shields.io/pypi/v/meerschaum?color=%2300cc66&label=Version ) | ![GitHub Repo stars](https://img.shields.io/github/stars/bmeares/Meerschaum?style=social) | ![License](https://img.shields.io/github/license/bmeares/Meerschaum?label=License) | ![Number of plugins]( https://img.shields.io/badge/dynamic/json?color=3098c1&label=Public%20Plugins&query=num_plugins&url=https%3A%2F%2Fapi.mrsm.io%2Finfo ) |
| ![PyPI - Python Version]( https://img.shields.io/pypi/pyversions/meerschaum?label=Python&logo=python&logoColor=%23ffffff ) | ![GitHub Sponsors](https://img.shields.io/github/sponsors/bmeares?color=eadf15&label=Sponsors) | [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/meerschaum)](https://artifacthub.io/packages/search?repo=meerschaum) | ![Number of registered users]( https://img.shields.io/badge/dynamic/json?color=3098c1&label=Registered%20Users&query=num_users&url=https%3A%2F%2Fapi.mrsm.io%2Finfo ) |

<p align="center">
<img src="https://meerschaum.io/files/images/demo.gif" alt="Meerschaum demo" height="450px">
</p>

## What is Meerschaum?

**Meerschaum is an ETL framework for time-series data.** You define **pipes** — named data streams — and Meerschaum keeps them in sync: it fetches only the new or changed rows, deduplicates and upserts them, manages the schema, and handles scheduling, serving, and storage.

Write a few lines of fetch logic; Meerschaum handles the rest of the pipeline. No more copy/pasting ETL scripts, hand-rolling incremental windows, or babysitting cron jobs. Drop it into an existing stack or stand up a full database-and-dashboard stack in minutes.

```python
import meerschaum as mrsm

pipe = mrsm.Pipe('plugin:noaa', 'weather', 'atl', instance='sql:local')
pipe.sync()  ### Pulls only what's new since the last sync.
```

<p align="center">
<img src="https://meerschaum.io/assets/screenshots/weather_pipes.png"/>
</p>

## Features

- ⚡️ **Incremental by default** — [the sync engine](https://meerschaum.io/reference/pipes/syncing/) fetches only new or changed rows and concurrently updates many streams at once. Duplicate rows are ignored; rows with existing keys are updated.
- 📊 **Built for data scientists and analysts** — integrate with [Pandas, Grafana, and friends](https://meerschaum.io/reference/data-analysis-tools/); persist DataFrames and always get the latest data. Skip pandas overhead and read rows as plain dicts with `Pipe.get_docs()`.
- 🗄️ **Production-ready, batteries included** — one-click deploy a [TimescaleDB + Grafana stack](https://meerschaum.io/reference/stack/), serve data org-wide via `FastAPI` (`uvicorn`/`gunicorn`), and secure API instances with scoped auth tokens. Supports PostGIS geometry (incl. ESRI CRS) for geospatial pipelines.
- 💼 **Jobs and scheduling** — run any command as a background [job](https://meerschaum.io/reference/background-jobs/) with `-d`. Built-in scheduler handles cron and interval schedules — no `crontab` or `systemd` setup. Execute locally, via `systemd`, or remotely on an API instance with `--executor-keys`.
- 🔌 **Easily expandable** — ingest any source with a simple [plugin](https://meerschaum.io/reference/plugins/writing-plugins/): just return a DataFrame. [Add any function as a command](https://meerschaum.io/reference/plugins/types-of-plugins/#action-plugins), define parent/child pipe relationships for composable SQL pipelines, or embed Meerschaum via its [Python API](https://docs.meerschaum.io).
- ✨ **Tailored for your experience** — a rich CLI that's surprisingly enjoyable, a web dashboard for the graphically inclined, and [connectors](https://meerschaum.io/reference/connectors/) for SQL, API, Valkey, and custom backends.
- 🧳 **Portable from the start** — `$MRSM_ROOT_DIR` emulates multiple installations and groups [instances](https://meerschaum.io/reference/connectors/#instances-and-repositories). No dependencies required (anything needed installs into a virtual environment), and it's `uv`-compatible: `uv tool install meerschaum`.

### Want to learn more?

Find a wealth of information at [meerschaum.io](https://meerschaum.io), or read up on Meerschaum in the wild:

- Interview featured in [*Console 100 - The Open Source Newsletter*](https://console.substack.com/p/console-100)
- [*A Data Scientist's Guide to Fetching COVID-19 Data in 2022*](https://towardsdatascience.com/a-data-scientists-guide-to-fetching-covid-19-data-in-2022-d952b4697) (Towards Data Science)
- [*Time-Series ETL with Meerschaum*](https://towardsdatascience.com/easy-time-series-etl-for-data-scientists-with-meerschaum-5aade339b398) (Towards Data Science)
- [*How I automatically extract my M1 Finance transactions*](https://bmeares.medium.com/how-i-automatically-extract-my-m1-finance-transactions-b43cef857bc7)

## Installation

For a more thorough setup guide, visit the [Getting Started](https://meerschaum.io/get-started/) page at [meerschaum.io](https://meerschaum.io).

### TL;DR

```bash
pip install meerschaum # or `uv tool install meerschaum[api]`
mrsm stack up -d
mrsm bootstrap pipes
```

## Usage
Visit [meerschaum.io](https://meerschaum.io) for setup, usage, and troubleshooting information. You can find technical documentation at [docs.meerschaum.io](https://docs.meerschaum.io).

### CLI
```bash
### Install the NOAA weather plugin.
mrsm install plugin noaa

### Register a new pipe to the built-in SQLite DB.
### You can instead run `bootstrap pipe` for a wizard.
### Enter 'KATL' for Atlanta when prompted.
mrsm register pipe -c plugin:noaa -m weather -l atl -i sql:local

### Pull data and create the table "plugin_noaa_weather_atl".
mrsm sync pipes -l atl -i sql:local
```

### Python API

```python
import meerschaum as mrsm

pipe = mrsm.Pipe(
    'foo', 'bar',
    instance = 'sql:local',                  ### Built-in SQLite DB.
    columns  = {'datetime': 'dt', 'id': 'id'},
)

### Sync a DataFrame (or list of dicts) — creates the table on first run.
pipe.sync([{'dt': '2024-07-01', 'id': 1, 'val': 10}])

### Duplicates are ignored; rows with existing keys are updated.
pipe.sync([{'dt': '2024-07-01', 'id': 1, 'val': 100}])
assert pipe.get_rowcount() == 1

### Read back as a DataFrame, filtered by time range and params.
df = pipe.get_data(begin='2024-01-01', end='2025-01-01', params={'id': [1]})

### Or skip pandas and read plain dicts.
docs = pipe.get_docs(params={'id': [1]})
### [{'dt': datetime(2024, 7, 1), 'id': 1, 'val': 100}]
```

For composable in-database SQL pipelines (`reference` inheritance and `{{ Pipe(...) }}` table resolution), see the [SQL pipes guide](https://meerschaum.io/reference/pipes/syncing/).

### Plugins

Ingest any source by returning rows from a `fetch` function — Meerschaum handles the rest:

```python
# ~/.config/meerschaum/plugins/example.py
__version__ = '1.0.0'
required = ['requests']

def register(pipe, **kw):
    return {'columns': {'datetime': 'dt', 'id': 'id'}}

def fetch(pipe, begin=None, end=None, **kw):
    import requests
    rows = requests.get('https://api.example.com/data').json()
    return rows  ### list of dicts or a Pandas DataFrame
```

## Support Meerschaum's Development

For consulting services and to support Meerschaum's development, please considering sponsoring me on [GitHub sponsors](https://github.com/sponsors/bmeares).

Additionally, you can always [buy me a coffee☕](https://www.buymeacoffee.com/bmeares)!

### License

Copyright 2020-2026 Bennett Meares

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
