Prev         Up         Next

Programming guidelines

There are certain guidelines and idioms which should be adhered to when using the processing package.

All platforms

Picklability:
Ensure that the arguments to the methods of proxies are picklable.
Shutting down manager:

Make sure that the manager's shutdown() method is eventually called.

shutdown() will be called automatically if the manager is garbage collected or if it is used with the with statement.

Thread safety of proxies:

Do not use a proxy from more than one thread unless you protect it with a lock.

Alternatively another copy of the proxy can be created using the copy.copy() function.

(There is never a problem with different processes using the 'same' proxy.)

Platforms without os.fork()

Platforms such as Windows which lack os.fork() have a few extra restrictions:

More picklability:

Ensure that all arguments to Process.__init__() are picklable.

Also, if you subclass Process then make sure that instances will be picklable when the start() method is called.

Global variables:

Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value in the parent process at the time that start() was called.

However, global variables which are just module level constants cause no problems.

Safe importing:

Make sure that the module containing the target of a Process instance (or the definition of a subclass of Process you are using) can be safely imported by a new Python interpreter without bad causing side effects (such a starting a new process).

For example, under Windows running the following module would recursively create new processes until you get a crash:

import processing

def foo():
    print 'hello'

p = processing.Process(target=foo)
p.start()
p.join()

Instead one should protect creation of the new process by using if __name__ == '__main__': as follows:

import processing

def foo():
    print 'hello'

if __name__ == '__main__':
    p = processing.Process(target=foo)
    p.start()
    p.join()

This allows the newly spawned Python interpreter to safely import the module and then run the module's foo() function.