Title: | Easy to use object-oriented thread pool framework |
---|---|
Author: | Christopher Arndt |
Version: | 1.2.3 |
Date: | 2006/06/23 |
A thread pool is an object that maintains a pool of worker threads to perform time consuming operations in parallel. It assigns jobs to the threads by putting them in a work request queue, where they are picked up by the next available thread. This then performs the requested operation in the background and puts the results in a another queue.
The thread pool object can then collect the results from all threads from this queue as soon as they become available or after all threads have finished their work. It's also possible, to define callbacks to handle each result as it comes in.
>>> main = TreadPool(poolsize) >>> requests = makeRequests(some_callable, list_of_args, callback) >>> [main.putRequests(req) for req in requests] >>> main.wait()
See the end of the module source code for a longer, annotated usage example.
You can view the API documentation, generated by epydoc, here:
You can download the latest version of this module here:
http://chrisarndt.de/en/software/python/download/
or see the colorized source code:
http://chrisarndt.de/en/software/python/threadpool/threadpool.py.html
The documentation is also packaged in the distribution.
The basic concept and some code was taken from the book "Python in a Nutshell" by Alex Martelli, copyright O'Reilly 2003, ISBN 0-596-00188-6, from section 14.5 "Threaded Program Architecture". I wrapped the main program logic in the ThreadPool class, added the WorkRequest class and the callback system and tweaked the code here and there.
There are some other recipes in the Python Cookbook, that serve a similar purpose. This one distinguishes itself by the following characteristics:
Due to the parallel nature of threads, you have to keep some things in mind:
There are several other recipes similar to this module in the Python Cookbook: