Metadata-Version: 2.4
Name: eo-grow
Version: 1.7.15
Summary: Earth observation framework for scaled-up processing in Python
Project-URL: Homepage, https://github.com/sentinel-hub/eo-grow
Project-URL: Documentation, https://eo-grow.readthedocs.io
Project-URL: Issues, https://github.com/sentinel-hub/eo-grow/issues
Project-URL: Source, https://github.com/sentinel-hub/eo-grow
Project-URL: Forum, https://forum.sentinel-hub.com
Author-email: Sinergise EO research team <eoresearch@sinergise.com>
License: MIT License
        
        Copyright (c) 2022 Sentinel Hub
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >=3.9
Requires-Dist: boto3<=1.29.6
Requires-Dist: click
Requires-Dist: colorlog
Requires-Dist: eo-learn[visualization]>=1.5.0
Requires-Dist: fiona>=1.8.18
Requires-Dist: fs>=2.2.0
Requires-Dist: geopandas<1,>=0.14.4
Requires-Dist: numpy
Requires-Dist: opencv-python-headless
Requires-Dist: pandas
Requires-Dist: pydantic<2.0,>=1.8.0
Requires-Dist: python-dateutil
Requires-Dist: python-rapidjson
Requires-Dist: rasterio
Requires-Dist: ray[default]
Requires-Dist: sentinelhub>=3.9.0
Requires-Dist: shapely>=1.8.0
Requires-Dist: typing-extensions>=4.5.0
Provides-Extra: dev
Requires-Dist: boto3; extra == 'dev'
Requires-Dist: boto3-stubs; extra == 'dev'
Requires-Dist: build; extra == 'dev'
Requires-Dist: deepdiff; extra == 'dev'
Requires-Dist: fs-s3fs; extra == 'dev'
Requires-Dist: joblib; extra == 'dev'
Requires-Dist: lightgbm>=3.0.0; extra == 'dev'
Requires-Dist: moto[s3]>=5.0.0; extra == 'dev'
Requires-Dist: mypy>=0.990; extra == 'dev'
Requires-Dist: numpy>=2.0.0; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pyogrio; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: pytest-order; extra == 'dev'
Requires-Dist: pytest>=4.0.0; extra == 'dev'
Requires-Dist: requests-mock; extra == 'dev'
Requires-Dist: scikit-learn; extra == 'dev'
Requires-Dist: scipy; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Requires-Dist: types-mock; extra == 'dev'
Requires-Dist: types-python-dateutil; extra == 'dev'
Requires-Dist: types-requests; extra == 'dev'
Requires-Dist: types-setuptools; extra == 'dev'
Requires-Dist: types-urllib3; extra == 'dev'
Provides-Extra: docs
Requires-Dist: autodoc-pydantic; extra == 'docs'
Requires-Dist: nbsphinx; extra == 'docs'
Requires-Dist: sphinx-mdinclude==0.5.4; extra == 'docs'
Requires-Dist: sphinx-rtd-theme==1.3.0; extra == 'docs'
Requires-Dist: sphinx==7.1.2; extra == 'docs'
Provides-Extra: ml
Requires-Dist: joblib; extra == 'ml'
Requires-Dist: lightgbm>=3.0.0; extra == 'ml'
Requires-Dist: scikit-learn; extra == 'ml'
Description-Content-Type: text/markdown

[![Package version](https://badge.fury.io/py/eo-grow.svg)](https://pypi.org/project/eo-grow)
[![Supported Python versions](https://img.shields.io/pypi/pyversions/eo-grow.svg?style=flat-square)](https://pypi.org/project/eo-grow)
[![Build Status](https://github.com/sentinel-hub/eo-grow/actions/workflows/ci_action.yml/badge.svg?branch=main)](https://github.com/sentinel-hub/eo-grow/actions)
[![Docs status](https://readthedocs.org/projects/eo-grow/badge/?version=latest)](https://eo-grow.readthedocs.io/en/latest/)
[![License](https://img.shields.io/pypi/l/eo-grow.svg)](https://github.com/sentinel-hub/eo-grow/blob/master/LICENSE)
[![Overall downloads](http://pepy.tech/badge/eo-grow)](https://pepy.tech/project/eo-grow)
[![Last month downloads](https://pepy.tech/badge/eo-grow/month)](https://pepy.tech/project/eo-grow)
[![Code coverage](https://codecov.io/gh/sentinel-hub/eo-grow/branch/main/graph/badge.svg)](https://codecov.io/gh/sentinel-hub/eo-grow)

# eo-grow
**Earth observation framework for scaled-up processing in Python.**

Analyzing Earth Observation (EO) data is complex and solutions often require custom tailored algorithms. In the EO domain most problems come with an additional challenge: *How do we apply the solution on a larger scale?*

Working with EO data is made easy by the [`eo-learn`](https://github.com/sentinel-hub/eo-learn) package, while the **`eo-grow`** package takes care of running the solutions at a large scale. In **`eo-grow`** an `EOWorkflow` based solution is wrapped in a **pipeline** object, which takes care of parametrization, logging, storage, multi-processing, EOPatch management and more. However, **pipelines** are not necessarily bound to `EOWorkflow` execution and can be used for other tasks such as training ML models.

Features of **`eo-grow`** include:
- Direct use of `EOWorkflow` procedures.
- Parametrizing workflows by using validated configuration files, making executions easy to reproduce and adjust.
- Easy use of both local and AWS S3 storage with no required code adaptation.
- Splitting large areas of interest into grids and defining collections of EOPatches.
- Workflows can be run either single-process, multi-process, or even on multiple machines (by using `ray` clusters).
- Execution reports and customizable logging.
- Options for skipping already processed data when re-running a pipeline.
- Offers a CLI interface for running pipelines, validating configuration files, and generating templates.
- A collection of basic pipelines, with methods that can be overridden to tailor to a large amount of use-cases.


## General Structure Overview

The core object of **`eo-grow`** is the `Pipeline`. Each pipeline has a `run_procedure` method, which is executed after the pipeline is set up. By default, the `run_procedure` executes an `EOWorkflow` which is built by the (user-defined) `build_workflow` method.

Each pipeline is linked to so called *managers*:
- `StorageManager` handles loading and saving of files,
- `AreaManager` defines the area of interest and how it should be split into EOPatches,
- `LoggingManager` provides control over logging.

![eo-grow-structure](docs/source/figures/eo-grow-structure.png)

Managers and pipelines usually require a large amount of parameters (setting storage paths, configuring log parameters, etc.), which are provided in `.json` configuration files. Each **`eo-grow`** object contains a special `Schema` class, which is a `pydantic` model describing the parameters of the object. Config files are then validated before execution to catch issues early. Templates for config files can be generated with the `eogrow-template` CLI command.

To make config files easier to write **`eo-grow`** uses a simple [config language](https://github.com/sentinel-hub/eo-grow/tree/main/docs/source/config-language.md) that supports importing other configs, variables, and more.


## Install

### PyPi distribution

The **`eo-grow`** package requires Python version `>= 3.8` and can be installed with

```
pip install eo-grow
```


## Command Line Interface

Running pipelines is easiest by using the CLI provided by **`eo-grow`**. For all options use the `--help` flag with each command.

- `eogrow <config>` executes the pipeline defined in the `<config>` file.
- `eogrow-validate <config>` only performs validation of the `<config>` file.
- `eogrow-test <config>` initializes the pipeline/object but does not run it. Useful for testing if managers are set correctly or for generating area-split grids.
- `eogrow-ray <cluster> <config>` executes the pipeline defined in `<config>` on the active Ray cluster defined by the `<cluster>` file.
- `eogrow-template <import path> <template>` generates a template config for the object specified by the `<import path>` and saves it to the `<template>` file (or outputs it directly if `<template>` is not provided).

## Documentation

For more information on the package visit [readthedocs](https://eo-grow.readthedocs.io/en/latest/).

Explanatory examples can be found [here](https://github.com/sentinel-hub/eo-grow/tree/main/examples).

More details on the config language used by **`eo-grow`** can be found [here](https://github.com/sentinel-hub/eo-grow/tree/main/docs/source/config-language.md).


## Questions and Issues

Feel free to ask questions about the package and its use cases at [Sentinel Hub forum](https://forum.sentinel-hub.com/) or raise an issue on [GitHub](https://github.com/sentinel-hub/eo-grow/issues).


## License

See [LICENSE](https://github.com/sentinel-hub/eo-grow/blob/main/LICENSE).

## Acknowledgements

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 101004112. The project is co-funded also by the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 101086461.
