Metadata-Version: 2.4
Name: Jug
Version: 2.5.0
Summary: A Task Based Parallelization Framework
Author-email: Luis Pedro Coelho <luis@luispedro.org>
License: MIT
Project-URL: Homepage, https://jug.readthedocs.io
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Operating System :: POSIX
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development
Classifier: Topic :: System :: Distributed Computing
Classifier: Intended Audience :: Science/Research
Requires-Python: >=3.8
Description-Content-Type: text/x-rst
License-File: COPYING.MIT
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: numpy; extra == "dev"
Requires-Dist: polars; extra == "dev"
Requires-Dist: pyyaml; extra == "dev"
Requires-Dist: redis; extra == "dev"
Dynamic: license-file

===========================================
Jug: A Task-Based Parallelization Framework
===========================================

Jug allows you to write code that is broken up into
tasks and run different tasks on different processors.

.. image:: https://github.com/luispedro/jug/actions/workflows/python-package.yml/badge.svg
       :target: https://github.com/luispedro/jug/actions/workflows/python-package.yml

.. image:: https://zenodo.org/badge/205237.svg
   :target: https://zenodo.org/badge/latestdoi/205237

.. image:: https://img.shields.io/badge/install%20with-conda-brightgreen.svg?style=flat
    :target: https://anaconda.org/conda-forge/jug

.. image:: https://static.pepy.tech/personalized-badge/jug?period=total&units=international_system&left_color=black&right_color=blue&left_text=Downloads
   :target: https://pepy.tech/project/jug

.. image:: https://img.shields.io/badge/CITATION-doi.org%2F10.5334%2Fjors.161-green.svg
   :target: https://doi.org/10.5334/jors.161


It uses the filesystem to communicate between processes and
works correctly over NFS, so you can coordinate processes on
different machines.

Jug is a pure Python implementation and should work on any platform.

Python versions 3.8 and above are supported (will almost certainly work on
earlier versions as well, but they are not part of the [CI
suite](https://github.com/luispedro/jug/actions).

*Documentation*: `https://jug.readthedocs.io/ <https://jug.readthedocs.io/>`__

*Video*: On `vimeo <http://vimeo.com/8972696>`__ or `showmedo
<http://showmedo.com/videotutorials/video?name=9750000;fromSeriesID=975>`__

*Mailing List*: `https://groups.google.com/group/jug-users
<https://groups.google.com/group/jug-users>`__

Testimonials
------------

"I've been using jug with great success to distribute the running of a
reasonably large set of parameter combinations" - Andreas Longva


Install
-------

You can install Jug with pip::

    pip install Jug

If you want to use ``jug shell``, install IPython as well::

    pip install Jug ipython

or use, if you are using `conda <https://anaconda.org/>`__, you can install jug
from `conda-forge <https://conda-forge.github.io/>`__ using the following
commands::

    conda config --add channels conda-forge
    conda install jug

Citation
--------

If you use Jug to generate results for a scientific publication, please cite

    Coelho, L.P., (2017). Jug: Software for Parallel Reproducible Computation in
    Python. Journal of Open Research Software. 5(1), p.30.

    https://doi.org/10.5334/jors.161


Short Example
-------------

Here is a one minute example. Save the following to a file called ``primes.py``
(if you have installed jug, you can obtain a slightly longer version of this
example by running ``jug demo`` on the command line)::

    from jug import TaskGenerator
    from time import sleep

    @TaskGenerator
    def is_prime(n):
        sleep(1.)
        for j in range(2,n-1):
            if (n % j) == 0:
                return False
        return True

    primes100 = [is_prime(n) for n in range(2,101)]

This is a brute-force way to find all the prime numbers up to 100. Of course,
this is only for didactical purposes, normally you would use a better method.
Similarly, the ``sleep`` function is so that it does not run too fast. Still,
it illustrates the basic functionality of Jug for embarassingly parallel
problems.

Type ``jug status primes.py`` to get::

    Task name                  Waiting       Ready    Finished     Running
    ----------------------------------------------------------------------
    primes.is_prime                  0          99           0           0
    ......................................................................
    Total:                           0          99           0           0


This tells you that you have 99 tasks called ``primes.is_prime`` ready to run.
So run ``jug execute primes.py &``. You can even run multiple instances in the
background (if you have multiple cores, for example). After starting 4
instances and waiting a few seconds, you can check the status again (with ``jug
status primes.py``)::

    Task name                  Waiting       Ready    Finished     Running
    ----------------------------------------------------------------------
    primes.is_prime                  0          63          32           4
    ......................................................................
    Total:                           0          63          32           4


Now you have 32 tasks finished, 4 running, and 63 still ready. Eventually, they
will all finish and you can inspect the results with ``jug shell primes.py``.
This requires ``ipython`` to be installed and will give you an ``ipython``
shell. The `primes100` variable is available,
but it is an ugly list of `jug.Task` objects. To get the actual value, you call
the `value` function::

    In [1]: primes100 = value(primes100)

    In [2]: primes100[:10]
    Out[2]: [True, True, False, True, False, True, False, False, False, True]

What's New
----------

Unreleased
----------

User-visible improvements
~~~~~~~~~~~~~~~~~~~~~~~~~

* Support project-local configuration files (``.jugrc`` or ``jugrc``). Jug now
  walks up the directory tree from the current working directory (up to the git
  project root) looking for local configuration files. See `configuration` for
  details.


Version 2.4.0
-------------

*Released 8 May 2025*

User-visible improvements
~~~~~~~~~~~~~~~~~~~~~~~~~

* Adds support for lambda functions in ``Tasklets``
* Adds ``NoHash`` class to disable hashing for some arguments. This is in
  ``jug.unsafe`` as it can be used to "fool" Jug, but it can be useful when
  there are nuisance arguments that are not relevant for the task (e.g., number
  of threads)
* jug.file_store: create files with better permissions

Internal improvements
~~~~~~~~~~~~~~~~~~~~~
* Convert to ``pyproject.toml`` for building

Bugfixes
~~~~~~~~

* Better error detection for permission problems
* Bugfix when using local imports and ``jug pack``


Drops support for versions of Python older than 3.7. Technically, it should
still work, but they are too old to test in Github CI, so we will not support
them.


Version 2.3.1
-------------

*Released 5 November 2023*

- Update for Python 3.12

Version 2.3.0
-------------

*Released 25 June 2023*

- jug shell: Add ``get_filtered_tasks()``
- jug: Fix ``jug --version`` (which had been broken in the refactoring to use subcommands)
- jug shell: Fix message in jug shell when there are no dependencies (it would repeatedly print the message stating *this will only be run once*)
- jug pack: Make it much faster to invalidate elements
- file_store: ensure that the temporary directory exists
- Drops support for Python 3.4

For older version see ``ChangeLog`` file or the `full history
<https://jug.readthedocs.io/en/latest/history.html>`__.

