psycopg2.extras – Miscellaneous goodies for Psycopg 2

This module is a generic place used to hold little helper functions and classes until a better place in the distribution is found.

Connection and cursor subclasses

A few objects that change the way the results are returned by the cursor or modify the object behavior in some other way. Typically !connection subclasses are passed as connection_factory argument to connect() so that the connection will generate the matching !cursor subclass. Alternatively a !cursor subclass can be used one-off by passing it as the cursor_factory argument to the cursor() method of a regular !connection.

Dictionary-like cursor

The dict cursors allow to access to the retrieved records using an iterface similar to the Python dictionaries instead of the tuples.

>>> dict_cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
>>> dict_cur.execute("INSERT INTO test (num, data) VALUES(%s, %s)",
...                  (100, "abc'def"))
>>> dict_cur.execute("SELECT * FROM test")
>>> rec = dict_cur.fetchone()
>>> rec['id']
1
>>> rec['num']
100
>>> rec['data']
"abc'def"

The records still support indexing as the original tuple:

>>> rec[2]
"abc'def"
class psycopg2.extras.DictCursor(*args, **kwargs)

A cursor that keeps a list of column name -> index mappings.

class psycopg2.extras.DictConnection

A connection that uses DictCursor automatically.

class psycopg2.extras.DictRow(cursor)

A row object that allow by-colmun-name access to data.

Real dictionary cursor

class psycopg2.extras.RealDictCursor(*args, **kwargs)

A cursor that uses a real dict as the base type for rows.

Note that this cursor is extremely specialized and does not allow the normal access (using integer indices) to fetched data. If you need to access database rows both as a dictionary and a list, then use the generic DictCursor instead of !RealDictCursor.

class psycopg2.extras.RealDictConnection

A connection that uses RealDictCursor automatically.

class psycopg2.extras.RealDictRow(cursor)

A !dict subclass representing a data record.

namedtuple cursor

New in version 2.3.

These objects require collections.namedtuple to be found, so it is available out-of-the-box only from Python 2.6. Anyway, the namedtuple implementation is compatible with previous Python versions, so all you have to do is to download it and make it available where we expect it to be...

from somewhere import namedtuple
import collections
collections.namedtuple = namedtuple
from psycopg.extras import NamedTupleConnection
# ...
class psycopg2.extras.NamedTupleCursor

A cursor that generates results as collections.namedtuple.

!fetch*() methods will return named tuples instead of regular tuples, so their elements can be accessed both as regular numeric items as well as attributes.

>>> nt_cur = conn.cursor(cursor_factory=psycopg2.extras.NamedTupleCursor)
>>> rec = nt_cur.fetchone()
>>> rec
Record(id=1, num=100, data="abc'def")
>>> rec[1]
100
>>> rec.data
"abc'def"
class psycopg2.extras.NamedTupleConnection

A connection that uses NamedTupleCursor automatically.

Logging cursor

class psycopg2.extras.LoggingConnection

A connection that logs all queries to a file or logger object.

initialize(logobj)

Initialize the connection to log to !logobj.

The !logobj parameter can be an open file object or a Logger instance from the standard logging module.

filter(msg, curs)

Filter the query before logging it.

This is the method to overwrite to filter unwanted queries out of the log or to add some extra data to the output. The default implementation just does nothing.

class psycopg2.extras.LoggingCursor

A cursor that logs queries using its connection logging facilities.

class psycopg2.extras.MinTimeLoggingConnection

A connection that logs queries based on execution time.

This is just an example of how to sub-class LoggingConnection to provide some extra filtering for the logged queries. Both the inizialize() and filter() methods are overwritten to make sure that only queries executing for more than mintime ms are logged.

Note that this connection uses the specialized cursor MinTimeLoggingCursor.

class psycopg2.extras.MinTimeLoggingCursor

The cursor sub-class companion to MinTimeLoggingConnection.

Additional data types

Hstore data type

New in version 2.3.

The hstore data type is a key-value store embedded in PostgreSQL. It has been available for several server versions but with the release 9.0 it has been greatly improved in capacity and usefulness with the addiction of many functions. It supports GiST or GIN indexes allowing search by keys or key/value pairs as well as regular BTree indexes for equality, uniqueness etc.

Psycopg can convert Python !dict objects to and from hstore structures. Only dictionaries with string/unicode keys and values are supported. !None is also allowed as value but not as a key. Psycopg uses a more efficient hstore representation when dealing with PostgreSQL 9.0 but previous server versions are supported as well. By default the adapter/typecaster are disabled: they can be enabled using the register_hstore() function.

psycopg2.extras.register_hstore(conn_or_curs, globally=False, unicode=False, oid=None)

Register adapter and typecaster for !dict-hstore conversions.

Parameters:
  • conn_or_curs – a connection or cursor: the typecaster will be registered only on this object unless globally is set to !True
  • globally – register the adapter globally, not only on conn_or_curs
  • unicode – if !True, keys and values returned from the database will be !unicode instead of !str. The option is not available on Python 3
  • oid – the OID of the hstore type if known. If not, it will be queried on conn_or_curs

The connection or cursor passed to the function will be used to query the database and look for the OID of the hstore type (which may be different across databases). If querying is not desirable (e.g. with asynchronous connections) you may specify it in the oid parameter (it can be found using a query such as SELECT 'hstore'::regtype::oid;).

Note that, when passing a dictionary from Python to the database, both strings and unicode keys and values are supported. Dictionaries returned from the database have keys/values according to the unicode parameter.

The hstore contrib module must be already installed in the database (executing the hstore.sql script in your contrib directory). Raise ProgrammingError if the type is not found.

Changed in version 2.4: added the oid parameter. If not specified, the typecaster is installed also if hstore is not installed in the public schema.

Composite types casting

New in version 2.4.

Using register_composite() it is possible to cast a PostgreSQL composite type (e.g. created with CREATE TYPE command) into a Python named tuple, or into a regular tuple if collections.namedtuple is not found.

>>> cur.execute("CREATE TYPE card AS (value int, suit text);")
>>> psycopg2.extras.register_composite('card', cur)
<psycopg2.extras.CompositeCaster object at 0x...>

>>> cur.execute("select (8, 'hearts')::card")
>>> cur.fetchone()[0]
card(value=8, suit='hearts')

Nested composite types are handled as expected, but the type of the composite components must be registered as well.

>>> cur.execute("CREATE TYPE card_back AS (face card, back text);")
>>> psycopg2.extras.register_composite('card_back', cur)
<psycopg2.extras.CompositeCaster object at 0x...>

>>> cur.execute("select ((8, 'hearts'), 'blue')::card_back")
>>> cur.fetchone()[0]
card_back(face=card(value=8, suit='hearts'), back='blue')

Adaptation from Python tuples to composite types is automatic instead and requires no adapter registration.

psycopg2.extras.register_composite(name, conn_or_curs, globally=False)

Register a typecaster to convert a composite type into a tuple.

Parameters:
  • name – the name of a PostgreSQL composite type, e.g. created using the CREATE TYPE command
  • conn_or_curs – a connection or cursor used to find the type oid and components; the typecaster is registered in a scope limited to this object, unless globally is set to !True
  • globally – if !False (default) register the typecaster only on conn_or_curs, otherwise register it globally
Returns:

the registered CompositeCaster instance responsible for the conversion

class psycopg2.extras.CompositeCaster(name, oid, attrs)

Helps conversion of a PostgreSQL composite type into a Python object.

The class is usually created by the register_composite() function.

name

The name of the PostgreSQL type.

oid

The oid of the PostgreSQL type.

type

The type of the Python objects returned. If collections.namedtuple is available, it is a named tuple with attributes equal to the type components. Otherwise it is just the !tuple object.

attnames

List of component names of the type to be casted.

atttypes

List of component type oids of the type to be casted.

UUID data type

New in version 2.0.9.

Changed in version 2.0.13: added UUID array support.

>>> psycopg2.extras.register_uuid()
<psycopg2._psycopg.type object at 0x...>

>>> # Python UUID can be used in SQL queries
>>> import uuid
>>> my_uuid = uuid.UUID('{12345678-1234-5678-1234-567812345678}')
>>> psycopg2.extensions.adapt(my_uuid).getquoted()
"'12345678-1234-5678-1234-567812345678'::uuid"

>>> # PostgreSQL UUID are transformed into Python UUID objects.
>>> cur.execute("SELECT 'a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11'::uuid")
>>> cur.fetchone()[0]
UUID('a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11')
psycopg2.extras.register_uuid(oids=None, conn_or_curs=None)

Create the UUID type and an uuid.UUID adapter.

class psycopg2.extras.UUID_adapter(uuid)

Adapt Python’s uuid.UUID type to PostgreSQL’s uuid.

inet data type

New in version 2.0.9.

>>> psycopg2.extras.register_inet()
<psycopg2._psycopg.type object at 0x...>

>>> cur.mogrify("SELECT %s", (Inet('127.0.0.1/32'),))
"SELECT E'127.0.0.1/32'::inet"

>>> cur.execute("SELECT '192.168.0.1/24'::inet")
>>> cur.fetchone()[0].addr
'192.168.0.1/24'
psycopg2.extras.register_inet()

Create the INET type and an Inet adapter.

class psycopg2.extras.Inet(addr)

Wrap a string to allow for correct SQL-quoting of inet values.

Note that this adapter does NOT check the passed value to make sure it really is an inet-compatible address but DOES call adapt() on it to make sure it is impossible to execute an SQL-injection by passing an evil value to the initializer.

Fractional time zones

psycopg2.extras.register_tstz_w_secs(oids=None, conn_or_curs=None)

The function used to register an alternate type caster for TIMESTAMP WITH TIME ZONE to deal with historical time zones with seconds in the UTC offset.

These are now correctly handled by the default type caster, so currently the function doesn’t do anything.

New in version 2.0.9.

Changed in version 2.2.2: function is no-op: see Time zones handling.

Coroutine support

psycopg2.extras.wait_select(conn)

Wait until a connection or cursor has data available.

The function is an example of a wait callback to be registered with set_wait_callback(). This function uses !select() to wait for data available.