Metadata-Version: 2.4
Name: umwelt-apy
Version: 0.2.0
Summary: Python package for easy access of the of umwelt.info API
Keywords: environment,nature conservation,engineering,environmental planning,EIA,open data,UVP
Author: AG umwelt.info
Author-email: AG umwelt.info <umwelt.info@uba.de>
License-Expression: MIT OR Apache-2.0
License-File: LICENSE-APACHE
License-File: LICENSE-MIT
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Customer Service
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Scientific/Engineering :: GIS
Requires-Dist: pandas>=2.2.3
Requires-Dist: requests>=2.32.3
Requires-Dist: zstandard>=0.23.0
Requires-Dist: pandas>=2.2.3 ; extra == 'pandas'
Requires-Dist: pyarrow>=24.0.0 ; extra == 'pandas'
Requires-Dist: polars>=1.26.0 ; extra == 'polars'
Requires-Dist: pyarrow>=24.0.0 ; extra == 'polars'
Requires-Python: >=3.11
Project-URL: API documentation, https://md.umwelt.info/swagger-ui/
Project-URL: Contact form, https://umwelt.info/en/contact
Project-URL: Project details, https://umwelt.info/en/
Project-URL: Source code, https://gitlab.opencode.de/umwelt-info/packages/python/
Provides-Extra: pandas
Provides-Extra: polars
Description-Content-Type: text/markdown

## umwelt-apy Python package

![PyPI - Version](https://img.shields.io/pypi/v/umwelt_apy)
The umwelt-apy package provides a Python-based access to the (meta-)datasets of umwelt.info, the central portal for German environmental data. The package, therefore, allows for an easy integration of those datasets into your Python-based workflows. The functionality of the package mirrors the webbased access as provided at https://umwelt.info and https://md.umwelt.info/. You can use the same queries and get the same datasets by accessing our API. The datasets are returned in either a JSON format or as Python dataframes like pandas and polars. 

For more information, the [API endpoint](https://md.umwelt.info/) provides a [SwaggerUI description of the data model](https://md.umwelt.info/swagger-ui/#/) where the [search for multiple datasets](https://md.umwelt.info/swagger-ui/#/search/text_search) is explained.  
See also the [article on our portal](https://umwelt.info/artikel/r-und-python-pakete) for further details.

## Installation
The umwelt-apy package can be installed from PyPi using: 

```bash
# basic version (requests only)
pip install umwelt-apy
```

```bash
# for using pandas
pip install "umwelt-apy[pandas]"
```

```bash
# for using pandas and polars
pip install "umwelt-apy[pandas,polars]"
```

You can also install the development version of the package from our GitLab repository using:

```bash
pip install umwelt-apy --extra-index-url https://gitlab.opencode.de/api/v4/projects/9710/packages/pypi/simple
```

## Usage 
The package provides a number of functions for querying umwelt.info and representing the resulting datasets in Python. It returns these datasets in a variety of formats, like a generator object of JSON entries, a pandas.DataFrame or a polars.DataFrame. We recommend using a polars.DataFrame, as it is the most performance-efficient way to retrieve large amounts of data due to using the column-oriented Arrow IPC format.

### Core Functions
* `fetch_by_url(url)` — This function is ideal for reproducing an earlier search of yours on our web-based interface. You can use a button in our web interface to create a url, which fetches the same datasets you found in said search. 
* `fetch_by_query(query="*", ...)` — This function sends a search query to umwelt.info and fetches the same datasets you would get from using this query at our web-based interface. The fetch_by_query function is easy to use for simple queries, however, reproducing all available filters through the query can be challenging. In that case, users may prefer the fetch_by_url() function instead.
* `fetch_by_ids(ids)` — This function fetches datasets by their unique identifier. This is ideal for when the IDs are already known; e.g., this can be useful for downloading resources. After using preview_resources you can select a subset of from the preview list using fetch_by_ids() and forward it as input to download_resources().
* `download_resources(url)` — This function downloads all the resources attached to the datasets of a respective query.


### Examples
Example 1: Fetching datasets by a direct URL
The basic function of umwelt_apy is `fetch_by_url(url="*", ...)`. An example that queries umwelt.info for 'Luftqualität' and prints the title of all resulting datasets would be

```python
from umwelt_apy import fetch_by_query

for dataset in fetch_by_url(url="https://md.umwelt.info/search?query=luftqualit%C3%A4t"):
    print(dataset['title'])
```

Example 2: Fetching by query string
For background how to build a query see https://md.umwelt.info/swagger-ui/#/search/text_search
If you want to know which facet values exist for a certain facet, you can use fetch_facet_values. Let's say you want to only fetch datasets from the data portal of Bavaria.

```python
from umwelt_apy import fetch_by_query, fetch_facet_values

organisations = fetch_facet_values("organisation")
print(organisations)
results = list(fetch_by_query(query="organisation:/Land/Bayern/open.bydata AND Ozon"))
```

First, you fetch all organisation values. Then, you use the corresponding facet value in your query, and finally, you use the refined query to only fetch "Luftqualität" entries from the Bavarian data portal.

Example 3: Download resources attached to the datasets of a given query
This is a four step process. First you retrieve the list of resources from the api (fetch_by_url or fetch_by_query),
you unnest and optionally refine it further (unnest_and_filter), you screen the preview (preview_resources) and download the resources (download_resources).
Note that the unnesting is a prerequisite for preview_resources() and download_resources().
Currently this workflow only works for output = "Pandas".
See the [tutorial](https://umwelt.info/artikel/so-laden-sie-daten-bei-umweltinfo-mit-python-und-r-herunter) for further details.
```python
url = "https://md.umwelt.info/search/all?query=(Ozon)+AND+organisation%3A%2FLand%2FBayern%2Fopen.bydata"
from umwelt_apy import fetch_by_url, unnest_and_filter, preview_resources, download_resources
import pandas
results = (
    fetch_by_url(
        api_url = url,
        columns=["source", "id", "title", "resources"],
        filter_datasets = lambda dataset: "resources" in dataset,
        output = "Pandas",
    )
    .pipe(unnest_and_filter, formats=["MicrosoftExcelSpreadsheet", "Csv"], description_regex="Ozon")
    .pipe(preview_resources)
)
```
In case the list of resources in the preview should be downloaded
```python
download_resources(results, base_dir="downloads")
```

More examples are provided in the docstrings of each function.

### Requirements
For umwelt-apy you need at least Python 3.11 or higher as well as the request package version 2.32 or higher. If you want to use Pandas or Polars dataframes, you need versions 2.2.3 or 1.26, respectively. Full details are provided in the pyproject.toml.

### Contributing
Contributions are welcome! Whether you want to fix a bug, add a new feature, or improve the documentation. You can find the source code at https://gitlab.opencode.de/umwelt-info/packages