Module: cogen.core

This module holds the essential stuff.

Programming with this library should be straghtforward. A coroutine is just a generator wrapped in a operation handling class:

@coroutine
def mycoro(bla):
    yield <operation>
    yield <operation>
  • the operation instructs the scheduler what to do with the coroutine: suspend it till someting happens, add another coro in the scheduler, raise a event and so on.
  • the operations are split up in 2 modules: events and sockets
    • the operations from sockets are related to network, like reading and writing, and these are done asynchronously but your code in the coroutine will see them as a regular synchronous or blocking call.
    • the operations from events are related to signals and coroutine/scheduler management.
  • if a operation has a result associated then the yield will return that result (eg. a string or a (connection, address) tuple) otherwise it will return the operation instance.

Typical example:

import sys, os
from cogen.common import *

@coroutine
def server():
    srv = sockets.Socket()
    srv.setblocking(0)
    srv.bind(('localhost',777))
    srv.listen(10)
    while 1:
        print "Listening..."
        conn, addr = yield sockets.Accept(srv)
        print "Connection from %s:%s" % addr
        m.add(handler, conn, addr)

@coroutine
def handler(sock, addr):
    yield sockets.Write(sock, "WELCOME TO ECHO SERVER !\r\n")
    while 1:
        line = yield sockets.ReadLine(sock, 8192)
        if line.strip() == 'exit':
            yield sockets.Write(sock, "GOOD BYE")
            sock.close()
            return

        yield sockets.Write(sock, line)

m = Scheduler()
m.add(server)
m.run()

Modules

coroutines

Coroutine related boilerplate and wrappers.

events

Base events (coroutine operations) and coroutine exceptions.

pubsub

queue

Port of Queue.Queue from the python standard library.

reactors

Network polling code.

The reactor works in tandem with the socket operations. Here's the basic workflow:

  • the coroutine yields a operation
  • the scheduler runs that operation (the process method) Note: all the socket operations share the same process method.
    • if run_first is False then the operation is added in the reactor for polling (with the exception that if we have data in out internal buffers the operation is runned first)
    • if run_first is set (it's default) in the operation then in process method the reactor's run_or_add is called with the operation and coroutine

Note: run_first is a optimization hack really, first it tries to run the operation (this asumes the sockets are usualy ready) and if it raises any exceptions like EAGAIN, EWOULDBLOCK etc it adds that operation for polling (via select, epoll, kqueue etc) then the run method will be called only when select, epoll, kqueue says that the socket is ready.

schedulers

Scheduling framework.

The scheduler handles the timeouts, run the operations and does very basic management of coroutines. Most of the heavy logic is in each operation class. See: events and sockets. Most of those operations work with attributes we set in the scheduler.

cogen is multi-state. All the state related to coroutines and network is in the scheduler and it's associated reactor. That means you could run several cogen schedulers in the same process/thread/whatever.

There is just one thing that uses global objects - the threadlocal-like local object in the coroutines module. It was actually aded for the wsgiserver factory that monkey patches the threadlocal module in order to make pylons run correctly (pylons relies heavily on threadlocals).

sockets

Socket-only coroutine operations and Socket wrapper. Really - the only thing you need to know for most stuff is the Socket class.

util

Mischelaneous or common.