Prev         Up         Next

Connection module

The connection module allows the sending of picklable objects between processes using sockets or (on Windows) named pipes. It also has support for digest authentication (using the hmac module from the standard library).

If the C extension _processing is not available then connections will be slower and on Windows one will not be able to use named pipes.

Classes and functions

The module defines the following functions:

Listener(address=None, family=None, backlog=1, authenticate=False, authkey=None)
Returns a wrapper for a bound socket or Windows named pipe which is 'listening' for connections.
Client(address, family=None, authenticate=False, authkey=None)

Attempts to set up a connection to the listener which is using address address.

The type of the connection is determined by family argument, but this can generally be omitted since it can usually be inferred from the format of address.

If authentication or authkey is a string then then digest authentication is used. The key used for authentication will be either authkey or currentProcess.getAuthKey() if authkey is None. If authentication fails then AuthenticationError is raised. See Authentication keys.

A Connection object is returned. (See Connection objects.)

Pipe()

Returns a pair of two connection objects (see Connection objects) representing the ends of a duplex connection. It is also available directly from processing.

For example:

from processing import Process, Pipe

def foo(conn):
    conn.send(42)

if __name__ == '__main__':
    c, d = Pipe()
    p = Process(target=foo, args=[d])
    p.start()
    print c.recv()   # prints 42
    p.join()

Note that at most one thread/process should be sending or receiving from a connection object at a given time.

The module exports two exception types:

exception AuthenticationError
Exception raised when there is an authentication error.
exception BufferTooShort

Exception raise by the recvbytes_into() method of a connection object when the supplied buffer object is too small for the message read.

If e is an instance of BufferTooShort then e.args[0] will give the message as a byte string.

Only available if _processing can be imported.

Listener objects

Instances of Listener have the following methods:

__init__(address=None, family=None, backlog=1, authenticate=False, authkey=None)
address
The address to be used by the bound socket or named pipe of the listener object.
family

The type of the socket (or named pipe) to use.

This can be one of the strings 'AF_INET' (for a TCP socket), 'AF_UNIX' (for a Unix domain socket) or 'AF_PIPE' (for a Windows named pipe). Of these only the first is guaranteed to be available.

If family is None than the family is inferred from the format of address. If address is also None then a default is chosen. This default is the family which is assumed to be the fastest available. See Address formats.

Note that if family is 'AF_UNIX' then the associated file will have only be readable/writable by the user running the current process -- use os.chmod() is you need to let other users access the socket.

backlog
If the listener object uses a socket then backlog is passed to the listen() method of the socket once it has been bound.
authenticate
If authenticate is true or authkey is not None then digest authentication is used.
authkey

If authkey is a string then it will be used as the authentication key; otherwise it must be None.

If authkey is None and authenticate is true then currentProcess.getAuthKey() is used as the authentication key.

If authkey is None and authentication is false then no authentication is done.

If authentication fails then AuthenticationError is raised. See Authentication keys.

accept()

Accept a connection on the bound socket or named pipe of the listener object. If authentication is attempted and fails then AuthenticationError is raised.

Returns a Connection object. See Connection objects.

close()

Close the bound socket or named pipe of the listener object.

This is called automatically when the listener is garbage collected.

Listener objects have the following read-only properties:

address
The address which is being used by the listener object.
last_accepted

The address from which the last accepted connection came.

If this is unavailable then None is returned.

Connection objects

A connection object represents one end of a message oriented socket or pipe connection. Connection objects have the following methods:

send(obj)

Send an object to the other end of the connection which should be read using recv().

The object must be picklable.

recv()
Return an object sent from the other end of the connection using send().
fileno()
Returns the file descriptor or handle used by the connection.
close()

Close the connection.

This is called automatically when the connection is garbage collected.

poll(timeout)
Return whether there is any data available to be read.
sendbytes(buffer)

Send byte data from an object supporting the buffer interface as a complete message.

Can be used to send strings.

recvbytes()
Return a complete message of byte data sent from the other end of the connection as a string.
recvbytes_into(buffer)

Read into buffer a complete message of byte data sent from the other end of the connection and return the number of bytes in the message.

If the buffer is too short then a BufferTooShort exception is raised and the complete message of bytes data is available as e.args[0] where e is the exception instance.

Warning

The recv() method automatically unpickles the data it receives which can be a security risk. Therefore if you are using the recv() and send() methods you should be using some form of authentication. See Authentication keys.

Transferring connection objects between processes

If the C extension processing._process is available and contains support then socket objects, connection objects and file objects can be successfully pickled in one process and unpickled in another.

Note however that on Windows there is no socket.fromfd() function. As a result on Windows an unpickled socket object is not a true socket object: only the recv(), send(), sendall(), close() and fileno() methods will work.

Address formats

Note that any string beginning with two backslashes is assumed by default to be an 'AF_PIPE' address rather than an 'AF_UNIX' address.

Authentication keys

When one uses the recv() method of a connection object, the data received is automatically unpickled. Unfortunately unpickling data from an untrusted source is a security risk. Therefore Listener and Client use the hmac module to provide digest authentication.

An authentication key is a string which can be thought of as a password: once a connection is established both ends will demand proof that the other knows the authentication key. (Demonstrating that both ends are using the same key does not involve sending the key over the connection.)

If authentication is requested but do authentication key is specified then the return value of currentProcess().getAuthKey() is used (see Process objects). This value will automatically inherited by any Process object that the current process creates. This means that (by default) all processes of a multi-process program will share a single authentication key which can be used when setting up connections between the themselves.

Suitable authentication keys can also be generated by using os.urandom().

Example

The following server code creates a listener which uses 'secret password' as an authentication key. It then waits for a connection and sends some data to the client:

from processing.connection import Listener
from array import array

address = ('localhost', 6000)     # family is deduced to be 'AF_INET'
listener = Listener(address, authkey='secret password')

conn = listener.accept()
print 'connection accepted from', listener.last_accepted

conn.send([2.25, None, 'junk', float])

conn.sendbytes('hello')

conn.sendbytes(array('i', [42, 1729]))

conn.close()
listener.close()

The following code connects to the server and receives some data from the server:

from processing.connection import Client
from array import array

address = ('localhost', 6000)
conn = Client(address, authkey='secret password')

print conn.recv()                 # => [2.25, None, 'junk', float]

print conn.recvbytes()            # => 'hello'

arr = array('i', [0, 0, 0, 0, 0])
print conn.recvbytes_into(arr)    # => 8
print arr                         # => array('i', [42, 1729, 0, 0, 0])

conn.close()