Metadata-Version: 2.4
Name: wetlands
Version: 1.0.1
Summary: Wetlands is a lightweight library to create conda environment and execute code inside them.
Author-email: Arthur Masson <arthur.masson@inria.fr>
License-Expression: MIT
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Requires-Dist: json5>=0.12.1
Requires-Dist: packaging>=24.2
Requires-Dist: psutil>=6.1.0
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: send2trash>=1.8.3
Requires-Dist: tomli>=1.0; python_version < '3.11'
Requires-Dist: typing-extensions>=4.0; python_version < '3.11'
Provides-Extra: debug
Requires-Dist: debugpy>=1.8.17; extra == 'debug'
Requires-Dist: json5>=0.12.1; extra == 'debug'
Provides-Extra: docs
Requires-Dist: mike>=2.1.3; extra == 'docs'
Requires-Dist: mkdocs-gen-files>=0.5.0; extra == 'docs'
Requires-Dist: mkdocs-include-markdown-plugin>=7.1.5; extra == 'docs'
Requires-Dist: mkdocs-literate-nav>=0.6.2; extra == 'docs'
Requires-Dist: mkdocs-material>=9.6.11; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.29.1; extra == 'docs'
Provides-Extra: shared-memory
Requires-Dist: numpy>=1.26; extra == 'shared-memory'
Description-Content-Type: text/markdown

![](Wetland.png)

# Wetlands

[![Wetlands tests](https://github.com/arthursw/wetlands/actions/workflows/ci.yml/badge.svg?event=push&branch=main)](https://github.com/arthursw/wetlands/actions/)
[![Wetlands pypi](https://img.shields.io/pypi/v/wetlands.svg?color=%2334D058)](https://pypi.org/project/wetlands/)
[![Wetlands python versions](https://img.shields.io/pypi/pyversions/wetlands.svg?color=%2334D058)](https://pypi.org/project/wetlands/)

**Wetlands** is a lightweight Python library for managing **Conda** environments.

**Wetlands** can create Conda environments on demand, install dependencies, and execute arbitrary code within them. This makes it easy to build *plugin systems* or integrate external modules into an application without dependency conflicts, as each environment remains isolated.

For example, if your application needs to use both [Stardist](https://github.com/stardist/stardist) and [Cellpose](https://www.cellpose.org/), installing them in the same environment may not work due to conflicting dependencies. With Wetlands, you can create a dedicated environment for each library and run them both as needed from your main script.

The name ***Wetlands*** comes from the tropical *environments* where anacondas thrive.

[Appose Python](https://github.com/apposed/appose-python) is a great alternative to Wetlands. It even provides the ability to run Java environments (see [Appose Java](https://github.com/apposed/appose-java)) and share memory between the Python world and the Java world.
There are other minor differences between the two libraries. For example, Wetlands provides integrated debugging tools to attach VS Code or PyCharm to isolated environments for step-through debugging with breakpoints. See the [Debugging guide](https://arthursw.github.io/wetlands/latest/debugging/) for more information.

---

**Documentation:** https://arthursw.github.io/wetlands/latest/

**Source Code:** https://github.com/arthursw/wetlands/

---

## ✨ Features

- **Automatic Environment Management**: Create and configure environments on demand.
- **Dependency Isolation**: Install dependencies without conflicts.
- **Embedded Execution**: Run Python functions or scripts inside isolated environments, with both blocking and non-blocking (task-based) APIs.
- **Task API**: Execute code asynchronously with progress reporting, cancellation, and event-driven callbacks.
- **Parallel Execution**: Launch multiple worker processes sharing a single Conda environment and distribute work.
- **Worker Health Monitoring**: Automatic detection of dead or hung workers, with configurable inactivity timeouts and transparent replacement of failed workers.
- **Integrated Debugging**: Debug code running in isolated environments using VS Code or PyCharm with breakpoints and step-through execution.
- **Pixi & Micromamba**: Wetlands uses either a self-contained `pixi` or `micromamba` for fast and lightweight Conda environment handling.

## 📦 Installation

To install **Wetlands**, simply use `pip`:

```sh
pip install wetlands
```

## 🚀 Usage Example

If the user doesn't have pixi or micromamba installed, Wetlands will download and set it up automatically.

```python
from wetlands.environment_manager import EnvironmentManager

# Initialize the environment manager
environment_manager = EnvironmentManager()

# Create and launch an isolated Conda environment named "numpy"
env = environment_manager.create("numpy", {"pip": ["numpy==2.2.4"]})
env.launch()

# Import a module proxy and call functions in the environment
minimal_module = env.import_module("minimal_module.py")
result = minimal_module.sum([1, 2, 3])
print(f"Result: {result}")

# Or use execute() for a direct blocking call
result = env.execute("minimal_module.py", "sum", args=([1, 2, 3],))

# Clean up
env.exit()
```

with `minimal_module.py`:

```python
def sum(x):
    import numpy as np
    return int(np.sum(x))
```

### Non-blocking execution with tasks

`submit()` returns a `Task` object immediately, letting you monitor progress, cancel, or wait for the result:

```python
# Submit a function for non-blocking execution
task = env.submit("compute.py", "heavy_computation", args=(data,))

# Do other work while the task runs...
print(f"Status: {task.status}")

# Block for the result when ready
task.wait_for()
print(f"Result: {task.result}")
```

### Parallel execution with multiple workers

Launch multiple worker processes sharing the same Conda environment:

```python
env.launch(max_workers=4)

# Distribute work across workers
results = list(env.map("segment.py", "segment", images))

# Or get individual Task objects for full control
tasks = env.map_tasks("segment.py", "segment", images)
```

Workers that crash or hang are detected and replaced automatically. Set `worker_timeout` to fail tasks when a worker stops responding:

```python
env.launch(max_workers=4, worker_timeout=300)  # 5-minute inactivity timeout
```

See the `examples/` folder and the [documentation](https://arthursw.github.io/wetlands/latest/) for more detailed examples.

## 🐛 Debugging

Wetlands includes tools to debug code running in isolated environments using VS Code or PyCharm. You can set breakpoints, step through code, and inspect variables in real-time.

### Quick Debugging Example

```bash
# List all running environments and their debug ports
wetlands list

# Attach VS Code to an environment for debugging
wetlands debug -s /path/to/my/project -n my_env

# Or use PyCharm instead
wetlands debug -s /path/to/my/project -n my_env -ide pycharm

# Kill an environment when done
wetlands kill -n my_env
```

For detailed debugging instructions and workflows, see the [Debugging guide](https://arthursw.github.io/wetlands/latest/debugging/).

## 🔗 Related Projects

- [Conda](https://anaconda.org/)
- [Pixi](https://pixi.sh/)
- [Micromamba](https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html)

## 🤖 Development

Use [uv](https://docs.astral.sh/uv/) to easily manage the project.

### Check & Format

Check for code errors with `uv run ruff check` and format the code with `uv run ruff format`.

### Tests

Test wetlands with `uv` and `ipdb`: `uv run pytest --pdb --pdbcls=IPython.terminal.debugger:TerminalPdb tests`
Use `--last-failed` to only re-run the failures: `uv run pytest --pdb --pdbcls=IPython.terminal.debugger:TerminalPdb --last-failed tests`

### Build and Publish

Build with `uv build`
Publish with `uv publish dist/wetlands-VERSION_NAME*`

### Generate documentation

The Wetlands documentation is generated with [`mkdocs-material`](https://squidfunk.github.io/mkdocs-material/), [`mkdocstrings`](https://mkdocstrings.github.io/), [`mike`](https://github.com/jimporter/mike) and others.

Install the doc tools with `uv pip install  ".[docs]"`.

MkDocs includes a live preview server, so you can preview your changes as you write your documentation. The server will automatically rebuild the site upon saving. Start it with: `uv run mkdocs serve`.

[`mike`](https://github.com/jimporter/mike) is used to generate multiple versions of the docs. To create a new version, `mike deploy [version]` is used by Github Actions, just update `.github/workflows/ci.yml`.

The doc is automatically generated by [Github Actions](https://squidfunk.github.io/mkdocs-material/publishing-your-site/#with-github-actions-material-for-mkdocs) (see `.github/workflows/ci.yml`).

The script `scripts/gen_ref_pages.py` is used by mkdocs to generate the API reference automatically (see [mkdocstrings recipes](https://mkdocstrings.github.io/recipes/)).

## 📋 Todo

- Use Pixi features and environment instead of creating one workspace per environment.

## 📜 License

This project was made by the [SAIRPICO team](https://www.inria.fr/en/sairpico) at Inria in Rennes (Centre Inria de l'Université de Rennes) and is licensed under the MIT License.

The logo Wetland was made by [Dan Hetteix](https://thenounproject.com/creator/DHETTEIX/) from Noun Project (CC BY 3.0).