Prev         Up         Next

Proxy objects

A proxy is an object which refers to a shared object which lives (presumably) in a different process. The shared object is said to be the referent of the proxy. Multiple proxy objects may have the same referent.

A proxy object has methods which invoke corresponding methods of its referent (although not every method of the referent will necessarily be available through the proxy). A proxy can usually be used in most of the same ways that the its referent can:

>>> from processing import Manager
>>> manager = Manager()
>>> l = manager.list([i*i for i in range(10)])
>>> print l
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> print repr(l)
<Proxy[list] object at 0x00DFA230>
>>> l[4]
16
>>> l[2:5]
[4, 9, 16]
>>> l == [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
True

Notice that applying str() to a proxy will return the representation of the referent, whereas applying repr() will return a representation of the proxy.

An important feature of proxy objects is that they are picklable so they can be passed between processes. Note, however, that if a proxy is sent to the corresponding manager's process then unpickling it will produce the referent itself. This means that one shared object can contain a second:

>>> a = manager.list()
>>> b = manager.list()
>>> a.append(b)         # referent of `a` now contains referent of `b`
>>> print a, b
[[]] []
>>> b.append('hello')
>>> print a, b
[['hello']] ['hello']

Some proxy methods return a proxy for an iterator. In particular list and dictionary proxies are iterables so they can be used with the for statement:

>>> a = manager.dict([(i*i, i) for i in range(10)])
>>> for key in a:
...     print '<%r,%r>' % (key, a[key]),
...
<0,0> <1,1> <4,2> <81,9> <64,8> <9,3> <16,4> <49,7> <25,5> <36,6>

Methods of BaseProxy

Proxy objects are instances of subclasses of BaseProxy. The only semi-public methods of BaseProxy are the following:

_callmethod(methodname, args=(), kwds={})

Call and return the result of a method of the proxy's referent.

If proxy is a proxy whose referent is obj then the expression

proxy._callmethod(methodname, args, kwds)

will evaluate the expression

getattr(obj, methodname)(*args, **kwds)         (*)

in the manager's process.

The returned value will be either a copy of the result of (*) or if the result is an unpicklable iterator then a proxy for the result.

If an exception is raised by (*) then then is is re-raised by _callmethod(). If some other exception is raised in the manager's process then this is converted into a RemoteError exception and is raised by _callmethod().

_getvalue()

Return a copy of the referent.

If the referent is unpicklable then this will raise an exception.

__repr__
Return a representation of the proxy object.
__str__
Return the representation of the referent.

Cleanup

A proxy object has a __del__ method so that when it gets garbage collected it deregisters itself from the manager which owns its referent.

A shared object gets deleted from the manager process when there are no longer any proxies referring to it.

Examples

An example of the usage of _callmethod():

>>> l = manager.list(range(10))
>>> l._callmethod('__getslice__', (2, 7))   # equiv to `l[2:7]`
[2, 3, 4, 5, 6]
>>> l._callmethod('__iter__')               # equiv to `iter(l)`
<Proxy[iterator] object at 0x00DFAFF0>
>>> l._callmethod('__getitem__', (20,))     # equiv to `l[20]`
Traceback (most recent call last):
...
IndexError: list index out of range

As another example, the definition of the proxy type used for iterators is the following:

class IteratorProxy(BaseProxy):
    def __iter__(self):
        return self
    def next(self):
        return self._callmethod('next')