Metadata-Version: 2.4
Name: data-syncmaster
Version: 0.3.2
Summary: Syncmaster REST API + scheduler + Worker
Author-email: MWS Data Bridge <onetools@mts.ru>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/MTSWebServices/syncmaster
Project-URL: Documentation, https://syncmaster.readthedocs.io/
Project-URL: Source, https://github.com/MTSWebServices/syncmaster
Project-URL: CI/CD, https://github.com/MTSWebServices/syncmaster/actions
Project-URL: Tracker, https://github.com/MTSWebServices/syncmaster/issues
Keywords: Syncmaster,REST,API,Worker,Spark,Transfer,ETL
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pydantic
Classifier: Framework :: Pydantic :: 2
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/x-rst
License-File: LICENSE.txt
Requires-Dist: pydantic~=2.12.5
Requires-Dist: pydantic-settings<2.14,>=2.12
Requires-Dist: pydantic-settings-logging~=0.1.1
Requires-Dist: sqlalchemy~=2.0.44
Requires-Dist: sqlalchemy-utils~=0.42.0
Requires-Dist: pyyaml~=6.0.3
Requires-Dist: coloredlogs~=15.0.1
Requires-Dist: python-json-logger~=4.0.0
Requires-Dist: asgi-correlation-id~=4.3.4
Requires-Dist: uuid6~=2025.0.1
Requires-Dist: celery~=5.6.0
Requires-Dist: psycopg2-binary~=2.9.11
Provides-Extra: server
Requires-Dist: fastapi~=0.129; extra == "server"
Requires-Dist: starlette-exporter~=0.23.0; extra == "server"
Requires-Dist: python-multipart~=0.0.21; extra == "server"
Requires-Dist: uvicorn~=0.41.0; extra == "server"
Requires-Dist: alembic~=1.18.1; extra == "server"
Requires-Dist: pyjwt~=2.11.0; extra == "server"
Requires-Dist: asyncpg~=0.31; extra == "server"
Requires-Dist: python-keycloak<7.2.0,>=7.0.1; extra == "server"
Requires-Dist: itsdangerous~=2.2.0; extra == "server"
Provides-Extra: scheduler
Requires-Dist: asyncpg~=0.31; extra == "scheduler"
Requires-Dist: apscheduler~=3.11.1; extra == "scheduler"
Provides-Extra: worker
Requires-Dist: onetl[files]~=0.15.0; extra == "worker"
Requires-Dist: pyspark<4.0.0; extra == "worker"
Requires-Dist: horizon-hwm-store~=1.1.4; extra == "worker"
Requires-Dist: jinja2~=3.1.6; extra == "worker"
Provides-Extra: kerberos
Requires-Dist: onetl[kerberos]~=0.15.0; extra == "kerberos"
Dynamic: license-file

.. _readme:

|Logo|

.. |Logo| image:: https://raw.githubusercontent.com/MTSWebServices/syncmaster/b6622b1b139137070f2287e288a4105c1110c1e4/docs/_static/logo_wide.svg
    :alt: Data.SyncMaster logo
    :target: https://github.com/MTSWebServices/data-syncmaster

|Repo Status| |Docker image| |PyPI| |PyPI License| |PyPI Python Version| |Documentation|
|Build Status| |Coverage| |pre-commit.ci|

.. |Repo Status| image:: https://www.repostatus.org/badges/latest/wip.svg
    :target: https://www.repostatus.org/#wip
.. |Docker image| image:: https://img.shields.io/docker/v/mtsrus/syncmaster-server?sort=semver&label=docker
    :target: https://hub.docker.com/r/mtsrus/syncmaster-server
.. |PyPI| image:: https://img.shields.io/pypi/v/data-syncmaster
    :target: https://pypi.org/project/data-syncmaster/
.. |PyPI License| image:: https://img.shields.io/pypi/l/data-syncmaster.svg
    :target: https://github.com/MTSWebServices/syncmaster/blob/develop/LICENSE.txt
.. |PyPI Python Version| image:: https://img.shields.io/pypi/pyversions/data-syncmaster.svg
    :target: https://badge.fury.io/py/data-syncmaster
.. |Documentation| image:: https://readthedocs.org/projects/syncmaster/badge/?version=stable
    :target: https://syncmaster.readthedocs.io
.. |Build Status| image:: https://github.com/MTSWebServices/syncmaster/workflows/Run%20All%20Tests/badge.svg
    :target: https://github.com/MTSWebServices/syncmaster/actions
.. |Coverage| image:: https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/
    MTSOnGithub/03e73a82ecc4709934540ce8201cc3b4/raw/syncmaster_badge.json
    :target: https://github.com/MTSWebServices/syncmaster/actions
.. |pre-commit.ci| image:: https://results.pre-commit.ci/badge/github/MTSWebServices/syncmaster/develop.svg
    :target: https://results.pre-commit.ci/latest/github/MTSWebServices/syncmaster/develop

What is Data.SyncMaster?
------------------------

Data.SyncMaster is as no-code ETL tool for transferring data between databases and file systems.
List of currently supported connections:

* Apache Hive
* Clickhouse
* Iceberg (REST Catalog + S3)
* MSSQL
* MySQL
* Oracle
* Postgres
* FTP
* FTPS
* HDFS
* S3
* Samba
* SFTP
* WebDAV

Based on `onETL <https://onetl.readthedocs.io/>`_ and `Apache Spark <https://spark.apache.org/>`_.

Goals
-----

* Make transferring data between databases and file systems as simple as possible
* Provide a lot of builtin connectors to transfer data in heterogeneous environment
* RBAC and multitenancy support

Non-goals
---------

* No streaming support for now, only batch operations
* This is not a change data capture (CDC) solution

.. documentation

Documentation
-------------

See https://syncmaster.readthedocs.io
