Metadata-Version: 2.4
Name: pumpwood-miscellaneous
Version: 1.1.7
Summary: Miscellaneous class and funcitions used in Pumpwood
License: BSD-3-Clause License
License-File: LICENSE
Author: André Andrade Baceti
Author-email: a.baceti@murabei.com
Requires-Python: >=3.6,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Dist: Flask (>=1.1.4)
Requires-Dist: Flask-SQLAlchemy (>=2.3.2)
Requires-Dist: GeoAlchemy2 (>=0.9.3)
Requires-Dist: Werkzeug (>=3.1.3)
Requires-Dist: azure-storage-blob (==12.23.1)
Requires-Dist: boto3 (==1.35.58)
Requires-Dist: google-cloud-storage (==2.18.2)
Requires-Dist: pandas (>=1.0)
Requires-Dist: pika (>=1.3.2)
Requires-Dist: python-slugify (>=6.1.1)
Project-URL: Homepage, https://github.com/Murabei-OpenSource-Codes/pumpwood-miscellaneous
Description-Content-Type: text/markdown

# Pumpwood Miscellaneous
Pumpwood miscellaneous functions and classes.

<a href="https://github.com/Murabei-OpenSource-Codes/pumpwood-miscellaneous">
    pumpwood-miscellaneous
</a>.

<p align="center" width="60%">
  <img src="doc/sitelogo-horizontal.png" /> <br>

  <a href="https://en.wikipedia.org/wiki/Cecropia">
    Pumpwood is a native brasilian tree
  </a> which has a symbiotic relation with ants (Murabei)
</p>

# Description
This is a module with miscellaneous functions and classes used in Pumpwood
System. These helps from adjusting static file provider (AWS S3,
Google Bucket, Azure blob), connect, read and publish messages in RabbitMQ,
and other functionalities.

## pumpwood_miscellaneous.database
Functions to help connect and build SQLAlchemy query strings.

### SQLAlchemyPostGres
Class to substitute SQLAlchemy/Flask database, it integrates with Pumpwood
Communication pumpJsonDump that serilize complex objects like shapely
BaseGeometry, pandas dataframes and Numpy arrays. It also requery a pre_ping
request for each query, invalidating not functional connections.

```
from flask import Flask, jsonify
from pumpwood_miscellaneous.models import FlaskPumpWoodBaseModel
from pumpwood_communication.serializers import PumpWoodJSONEncoder


db = SQLAlchemyPostGres(model_class=FlaskPumpWoodBaseModel)


def create_app(config_dict, test=False):
    """Create and return app."""
    app = Flask(__name__)
    app.json_encoder = PumpWoodJSONEncoder

    db.app = app
    db.init_app(app)
```

## pumpwood_miscellaneous.models
Class and function to help definition of flask models.

### FlaskPumpWoodBaseModel
Add a default a BigInteger field id for all models.

## pumpwood_miscellaneous.query
Convert dictionary payload to a SQLAlchemy query. Make similar query to
Django filter, exclude and order_by ORM API.

```
from models import User
from pumpwood_miscellaneous.query import SqlalchemyQueryMisc

query_result = SqlalchemyQueryMisc.sqlalchemy_kward_query(
    object_model=User,
    filter_dict={
        "name__icontains": "Jonh",
        "age__gte": 30
    },
    exclude_dict={
      "company__in": ["Murabei", "Data Random Corp"]
    },
    order_by=["name", "-age"]
).limit(50).all()
```

## pumpwood_miscellaneous.rabbitmq
Manage connections with rabbitmq.

### PumpWoodRabbitMQ
Help sending msg to RabbitMQ queues, it is possible to "lazy" set constructor
variables using init function. Send function uses PumpWoodJSONEncoder, making
it possible to send complex objects as msg payload.

```
from pumpwood_miscellaneous.rabbitmq import PumpWoodRabbitMQ

not_lazy_worker = PumpWoodRabbitMQ(
  queue="some-queue",
  username="some-user",
  password="with-a-strong-pass"
  host="1.2.3.4"
  port=5672)
not_lazy_worker.send({
    "pandas": pd.DataFrame({
        "process_data": [1, 2, 3, 4],
        "time": pd.to_datatime([
            "2021-01-01", "2021-02-01", "2021-03-01", "2021-04-01"])
      }),
    "ok": "so-ok"})

lazy_worker = PumpWoodRabbitMQ()
lazy_worker.init(
  queue="some-queue",
  username="some-user",
  password="with-a-strong-pass"
  host="1.2.3.4"
  port=5672
)
```

## pumpwood_miscellaneous.storage
Make the interaction with different storage backends using the same API. So
far Google bucket and AWS S3 were implemented.

### PumpWoodStorage
```
from pumpwood_miscellaneous.storage import PumpWoodStorage

storage_google = PumpWoodStorage(
  storage_type="google_bucket",
  base_path="base_path/for/files/"
  bucket_name="some_bucket")

file_data = storage_google.read_file("file_path/file.joblib")
self.storage_google.delete_file("file_path/file.joblib")


storage_google = PumpWoodStorage(
  storage_type="aws_s3",
  base_path="base_path/for/files/"
  bucket_name="some_s3")

file_data = storage_google.read_file("file_path/file_2.joblib")
self.storage_google.delete_file("file_path/file_2.joblib")
```

### allowed_extension
Check if file extension is in a list.

```
allowed_extension(
    filename="file.xlsx", allowed_extensions=["xls", "xlsx", "parquet"],
    exception=Exception)
```

