Metadata-Version: 2.4
Name: omniload
Version: 0.0.0
Summary: omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.
Author: İrem Çağın Yurttürk, Johannes Brandis, Mustafa Ersan, Sanju Shah Gupta, Saravjeet 'Aman' Singh, Tanay Bensu Yurttürk
Author-email: Burak Karakan <burak.karakan@getbruin.com>
Maintainer-email: Andreas Motl <andreas.motl@panodata.org>
License-Expression: MIT
Project-URL: Homepage, https://github.com/panodata/omniload
Project-URL: Issues, https://github.com/panodata/omniload/issues
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Customer Service
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Manufacturing
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Telecommunications Industry
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: OS Independent
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Adaptive Technologies
Classifier: Topic :: Communications
Classifier: Topic :: Database
Classifier: Topic :: Documentation
Classifier: Topic :: Education
Classifier: Topic :: Internet
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Office/Business
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Object Brokering
Classifier: Topic :: Software Development :: Pre-processors
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Version Control
Classifier: Topic :: System :: Archiving
Classifier: Topic :: System :: Clustering
Classifier: Topic :: System :: Distributed Computing
Classifier: Topic :: System :: Logging
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: System :: Networking
Classifier: Topic :: System :: Systems Administration
Classifier: Topic :: Text Processing
Classifier: Topic :: Utilities
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.Apache-2.0
License-File: LICENSE.md
License-File: NOTICE
Requires-Dist: aiohttp<4
Requires-Dist: asana<3.3
Requires-Dist: click<8.2
Requires-Dist: clickhouse-connect<1,>=0.8
Requires-Dist: clickhouse-driver<0.3
Requires-Dist: clickhouse-sqlalchemy<0.3
Requires-Dist: confluent-kafka<2.13
Requires-Dist: couchbase<5,>=4
Requires-Dist: databricks-sql-connector<4.3
Requires-Dist: databricks-sqlalchemy<1.1
Requires-Dist: dataclasses-json<0.7
Requires-Dist: dlt<1.22
Requires-Dist: dlt-cratedb<0.2
Requires-Dist: duckdb<1.5
Requires-Dist: duckdb-engine<0.18
Requires-Dist: elasticsearch<9
Requires-Dist: facebook-business<24
Requires-Dist: flatten-json<0.2
Requires-Dist: gcsfs==2025.3.2
Requires-Dist: google-ads<29
Requires-Dist: google-analytics-data<0.19
Requires-Dist: google-api-python-client<3
Requires-Dist: google-auth<3
Requires-Dist: google-cloud-bigquery-storage<3
Requires-Dist: google-cloud-spanner<4
Requires-Dist: influxdb-client<2
Requires-Dist: mysql-connector-python<10
Requires-Dist: oracledb<4
Requires-Dist: pendulum<4,>=3
Requires-Dist: psutil<7
Requires-Dist: psycopg2-binary<3
Requires-Dist: py-machineid<1
Requires-Dist: pyairtable<3
Requires-Dist: pyarrow<25,>=18
Requires-Dist: pyathena<4
Requires-Dist: pymongo<5
Requires-Dist: pymysql<2
Requires-Dist: pyrate-limiter<3.8
Requires-Dist: python-quickbooks<0.10
Requires-Dist: requests<3
Requires-Dist: rich<14
Requires-Dist: rudder-sdk-python<2.2
Requires-Dist: s3fs==2025.3.2
Requires-Dist: simple-salesforce<2
Requires-Dist: smartsheet-python-sdk<4
Requires-Dist: snowflake-connector-python<5,>=3.18
Requires-Dist: snowflake-sqlalchemy<1.10
Requires-Dist: sqlalchemy<2
Requires-Dist: sqlalchemy-bigquery<1.13
Requires-Dist: sqlalchemy-cratedb<1
Requires-Dist: sqlalchemy-hana<2.1
Requires-Dist: sqlalchemy-redshift<0.9
Requires-Dist: sqlalchemy-spanner<2
Requires-Dist: stripe<10.8
Requires-Dist: tldextract<5.2
Requires-Dist: tqdm<5
Requires-Dist: trino[sqlalchemy]==0.336.0
Requires-Dist: typer<0.14
Requires-Dist: urllib3<2.8
Provides-Extra: develop
Requires-Dist: mypy<1.16; extra == "develop"
Requires-Dist: pkginfo<2; extra == "develop"
Requires-Dist: poethepoet<1; extra == "develop"
Requires-Dist: pyproject-fmt<3; extra == "develop"
Requires-Dist: pytest-repeat<1; extra == "develop"
Requires-Dist: ruff<0.12; extra == "develop"
Requires-Dist: sqlalchemy2-stubs==0.0.2a38; extra == "develop"
Requires-Dist: validate-pyproject<1; extra == "develop"
Provides-Extra: full
Requires-Dist: omniload[ibm-db,odbc,oracle]; extra == "full"
Provides-Extra: ibm-db
Requires-Dist: ibm-db<4; platform_machine != "aarch64" and extra == "ibm-db"
Requires-Dist: ibm-db-sa<0.5; platform_machine != "aarch64" and extra == "ibm-db"
Provides-Extra: odbc
Requires-Dist: pyodbc<5.4; extra == "odbc"
Provides-Extra: oracle
Requires-Dist: oracledb>=2.0.0; extra == "oracle"
Provides-Extra: release
Requires-Dist: build<2; extra == "release"
Requires-Dist: twine<7; extra == "release"
Provides-Extra: test
Requires-Dist: pendulum[test]<4,>=3; extra == "test"
Requires-Dist: pytest<9; extra == "test"
Requires-Dist: pytest-asyncio<2; extra == "test"
Requires-Dist: pytest-cov<5; extra == "test"
Requires-Dist: pytest-xdist[psutil]<4; extra == "test"
Requires-Dist: testcontainers[mysql,postgres]<4.9; extra == "test"
Requires-Dist: types-requests==2.32.0.20240907; extra == "test"
Requires-Dist: verlib2; extra == "test"
Dynamic: license-file

<div align="center">

# omniload
<p>Copy data between any source and any destination.</p>
<img src="https://github.com/panodata/omniload/blob/main/resources/demo.gif?raw=true" width="750" />

</div>

## About

omniload is a polyglot data loader framework based on dlt.
It allows you to load data from any source into any destination,
either using a concise CLI from your shell,
or the Python API from your own applications.

omniload provides the same efficient incremental data loading modes
inherited by [dlt]: `append`, `merge`, and `delete+insert`.

## Install
We recommend using [uv] to install or run `omniload`.

```shell
pip install uv
uvx omniload
```

Alternatively, if you'd like to install it globally:
```shell
uv pip install --system omniload
```

While installation with vanilla `pip` is possible, it's an order of magnitude slower.

## Synopsis

The next command instructs omniload to read the table `public.some_data` from
your PostgreSQL instance, and to write the data to your BigQuery warehouse
under the schema `omniload` and table `some_data`.

```shell
omniload ingest \
    --source-uri 'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \
    --source-table 'public.some_data' \
    --dest-uri 'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \
    --dest-table 'omniload.some_data'
```

## Handbook

Please visit the [full documentation][Documentation], or inspect the
list of supported [sources and destinations].

## Project

### Contribute

Contributions are very much welcome. Please visit the [Documentation]
to learn how to spin up a sandbox environment on your workstation and submit
patches, or create a [ticket][Issues] to report a bug or propose a feature.

### Status

Breaking changes should be expected until a 1.0 release, so version pinning is
strongly recommended, especially when using this software as a library.
For example:
```shell
pip install 'omniload[full]==0.0.42'
```

### License

The project is licensed under the MIT License, see the [LICENSE] file for details.
Some components are licensed under the Apache 2.0 license, see the [NOTICE] file for details.

### Acknowledgements

This project would not have been possible without the amazing work by the
authors and contributors to [SQLAlchemy], [dlt], and [ingestr], turtles all
the way down. Kudos.


[dlt]: https://github.com/dlt-hub/dlt
[Documentation]: https://github.com/panodata/omniload/blob/main/docs/getting-started/quickstart.md
[ingestr]: https://bruin-data.github.io/ingestr/
[Issues]: https://github.com/panodata/omniload/issues
[LICENSE]: https://github.com/panodata/omniload/blob/main/LICENSE.md
[NOTICE]: https://github.com/panodata/omniload/blob/main/NOTICE
[sources and destinations]: https://github.com/panodata/omniload/blob/main/docs/supported-sources/index.md
[SQLAlchemy]: https://www.sqlalchemy.org/
[uv]: https://docs.astral.sh/uv/
