MongoUserDict¶
Basic Usage¶
By subclassing MongoUserDict
you can manage MongoDB documents with your own
custom classes. MongoUserDict
is itself a subclass of UserDict
and
supports all dictionary access methods.
To define your class, provide the pymongo
database object and the name of the
collection:
import mongo_objects
class Event( mongo_objects.MongoUserDict ):
db = ... # provide your pymongo database object here
collection_name = 'events'
CRUD Operations¶
mongo_objects
supports all CRUD (create, read, update, delete) operations.
Create an object by creating an instance of your MongoUserDict
subclass:
event = Event( {
... document data here ...
} )
Saving a new object will write it to the database and assign it a MongoDB ObjectId. _created and _updated timestamps will also be added to the document:
event.save()
To load multiple documents, use the familiar find()
method. Filter, template
and other arguments are passed to the underlying pymongo.find()
.
The resulting documents are returned as instances of your MongoUserDict
document
subclass:
for event in Event.find():
...
Single documents may be loaded with find_one()
:
event = Event.find_one()
load_by_id()
is a convenience method to load a document by its MongoDB ObjectId.
The method accepts either a string or BSON ObjectId:
event = Event.load_by_id( '662b0d705a4c657820625300' )
To update an document, simply update the instance of your document class with
the usual methods for modifying dictionaries and then save()
it.
For existing documents (i.e. documents that already have a MongoDB ObjectId),
save()
automatically uses pymongo.find_one_and_replace()
to update the existing document in place.
To prevent overwriting modified data, save()
will only
replace objects that haven’t
already been modified in the database.
See the save()
method reference for more details on this behavior.
save()
updates the _updated timestamp to the current UTC time for all objects
successfully saved.
event['new-key'] = 'add some content to the object'
event.save()
Deleting an object is accomplished with the delete()
method. The document is deleted from
the database and the ObjectId is removed from the in-memory object. If the object is
saved again, it will be treated as a new document.
event.delete()
Read-Only Documents¶
It is possible to use projections that return incomplete documents
that can’t be safely be saved by mongo_objects
without data loss.
mongo_objects
doesn’t attempt to determine whether a projection
is safe or not.
The find()
and find_one()
methods accept a readonly keyword argument with
three potential values:
None
(default): All documents created with a projection are marked readonly. All other documents are considered read-write.True
: The documents will be marked as readonly.False
: The documents will be considered read-write. This is a potentially dangerous choice. With great power comes great responsibility.
save()
refuses to save a readonly object and raises a MongoObjectsReadOnly
exception instead.
Document IDs¶
Once a document has been saved and an ObjectId assigned by MongoDB, the id()
method returns a string representation of the ObjectId.
load_by_id()
may be used to load a specific document by its ObjectId or
by the string returned by id()
:
# load a random document
event = Event.find_one()
# save the ObjectId for later
eventId = event.id()
... time passes ...
# reload the document from its ObjectId
event_again = Event.load_by_id( eventId )
ObjectIds are represented as lowercase hex digits, so the result of id()
is safe to use in URLs.
Object Versions¶
mongo_objects
supports an optional document schema versioning system. To enable this
functionality, provide an object_version value when defining the class:
class Event( mongo_objects.MongoUserDict ):
db = ... # provide your pymongo database object here
collection_name = 'events'
object_version = 3
The current object_version will automatically be added by save()
to each document as _objver.
By default find()
and find_one()
will then automatically adjust each query
to restrict the results to the current object_version. This guarantees that only objects
at the current object version will be returned. This is equivalent to the following:
Event.find( {
... other filters ...,
'_objver' : Event.object_version
} )
To manage this functionality, find()
and find_one()
accept an optional object_version parameter with the following meaning:
None
(default): documents will automatically be filtered to the current object_versionFalse
: no filtering for object version will be performedany other value: only documents with this value as the object version will be returned
Object versioning provides a convenient workflow for migrating database schemas
and protecting the application from inadvertently reading data in an obsolete format.
First increment the object_version of the MongoUserDict
document subclass,
then loop through all objects at the previous version and perform the migration.
For example, to update the layout of the object defined above:
# object_version is now 4
class Event( mongo_objects.MongoUserDict ):
db = ... # provide your pymongo database object here
collection_name = 'events'
object_version = 4
# loop through all objects at version 3
for event in Event.find( object_version=3 ):
... perform migration steps ...
# saving the document object automatically updates _objver
# to the current value
event.save()
Polymorphism¶
Subclass the PolymorphicMongoUserDict
class to enable mongo_objects
support
for polymorphic user document classes. Specifically, from the same MongoDB collection
find()
and find_one()
will return instances of different document classes.
Each polymorphic subclass defines a separate key which
save()
adds to the document.
When the document is read back from the database, the key is compared to a list of
potential classes and the correct instance type returned.
Note the strong recommendation to define an empty subclass_map so each set of polymorphic classes use their own namespace for subclass keys.
import mongo_objects
# create a base class for the collection
class Event( mongo_objects.PolymorphicMongoUserDict ):
db = ... # provide your pymongo database object here
collection_name = 'events'
# Recommended: define an empty subclass_map in the base class
# This creates a separate namespace for the polymorphic
# subclass keys.
# Otherwise, subclasses will share the PolymorphicMongoUserDict
# subclass namespace and risk collisions with other subclasses
# from other collections.
subclass_map = {}
.. your generally useful event methods ...
# now create subclasses for each object variation
# each subclass requires a unique key
class OnSiteEvent( Event ):
subclass_key = 'onsite'
.. your onsite-specific event methods ...
class OnlineEvent( Event ):
subclass_key = 'online'
.. your online-specific event methods ...
class HybridEvent( Event ):
subclass_key = 'hybrid'
.. your hybrid-specific event methods ...
Creating Documents¶
Create and save the objects using a subclass. save()
automatically
adds the appropriate subclass key to the document.
hybrid = HybridEvent()
hybrid.save()
# save the document ID for later
hybridId = hybrid.id()
Loading Documents¶
If you load documents using the base class, all available documents will be returned. The resulting objects will each be an instance of the correct subclass based on the subclass key.
# event is an instance of HybridEvent
event = Event.load_by_id( hybridId )
# retrieve all events as the correct type
Event.find()
If you load documents using a subclass, only those documents of that subclass type will be returned.
# Only hybrid event objects will be returned
HybridEvent.find()
If a document has a missing or invalid subclass key, an instance of
the subclass with a None
subclass key is returned.
If no such subclass is defined, MongoObjectsPolymorphicMismatch
is raised.
Proxy Support¶
MongoUserDict
seamlessly supports the subdocument proxy objects also
included in this module.
get_unique_integer()
provides per-document unique integer values.
get_unique_key()
uses these unique integers to create unique string
values suitable for subdocument proxy keys.
split_id()
separates a full subdocument ID value into its components:
a parent document ObjectId plus one or more proxy keys.
The class method load_proxy_by_id()
accepts a subdocument ID as
generated by id()
and a list of one or more proxy classes.
The parent document is loaded and the proxy objects are
instantiated. See MongoDictProxy
and MongoListProxy
for more
details.
For parent documents already loaded in memory, load_proxy_by_local_id()
accepts
the proxy ID portion generated by proxy_id()
and the related list of proxy classes
to instantiate the requested proxy objects.
Class Reference¶
- class mongo_objects.MongoUserDict(doc={}, readonly=False)¶
Access MongoDB documents through user-defined UserDict subclasses.
User classes must provide collection_name and database or override the
collection()
method to return the correct pymongo collection object.- collection_name¶
Required: override with the name of the MongoDB collection where the documents are stored
- database¶
Required: override with the pymongo database connection object
- object_version = None¶
Optional: If object_version is not``None``,
find()
andfind_one()
by default restrict queries to documents with the current object_version. This enables a type of schema versioning.
- subdoc_key_sep = 'g'¶
The character sequence used to separate the document ID from proxy subdocument keys. This may be overriden but it is the user’s responsibility to guarantee that this sequence will never appear in any ID or subdoc key. Since the default IDs and subdoc keys are hex,
g
is a safe, default separator
- __init__(doc={}, readonly=False)¶
Initialize the custom UserDict object flagging readonly objects appropriately.
- Raises:
MongoObjectsAuthFailed – if authorize_init() has been overriden and does not return a truthy value
- authorize_init()¶
Called after the document object is initialized but before it is returned to the user.
This hook is called when creating a new object and during calls to
find()
andfind_one()
.Override this method and return
False
to block creating this document.- Returns:
Whether creating the current document is authorized (default
True
)- Return type:
bool
- authorize_delete()¶
Called before the current document is deleted.
Override this method and return
False
to block deleting this document.- Returns:
Whether deleting the current document is authorized (default
True
)- Return type:
bool
- classmethod authorize_pre_read()¶
Called before a read operation is performed. This is a class method as no data has been read and no document object has been created.
- Returns:
Whether reading any documents is authorized (default
True
)- Return type:
bool
- authorize_read()¶
Called after a document has been read but before the data is returned to the user. If the return value is not truthy, the data will not be returned.
Note that
find_one()
only inspects the first document returned by the underlyingpymongo.find_one()
call. If the document returned does not pass authorization, no attempt is made to locate another matching document.- Returns:
Whether reading the current document is authorized (default
True
)- Return type:
bool
- authorize_save()¶
Called before the current document is saved.
- Returns:
Whether saving the current document is authorized (default
True
)- Return type:
bool
- classmethod collection()¶
Return the
pymongo.Collection
object from the active database for the named collectionFor complex situations users may omit the database and connection_name attributes when defining the class and instead override this method.
This method must return a
pymongo.Collection
object.
- delete()¶
Delete the current object. Remove the id so
save()
will know this is a new object if we try to re-save. Other data values are still available in this dictionary even after the data is deleted from the database.- Raises:
MongoObjectsAuthFailed – if authorize_delete() has been overriden and does not return a truthy value
- classmethod find(filter={}, projection=None, readonly=None, object_version=None, **kwargs)¶
Return matching documents as instances of this class
- Parameters:
filter (dict) – Updated with cls.object_version as appropriate and passed to
pymongo.find()
projection (dict) – Passed to
pymongo.find()
readonly (
None
or bool) –If
None
and a projection is provided, mark the objects as readonly.If
None
and no projection is given, consider the objects read-write.If
True
, mark the objects as readonly regardless of the projection.If
False
, consider the objects read-write regardless of the projection.
object_version (
None
,False
, any scalar value) –Only if cls.object_version is not
None
, implement object schema versioning.If
None
, update the filter to only return documents with the current cls.object_version valueIf
False
, don’t filter objects by cls.object_versionFor any other value, update the filter to only return documents with the given object_version
- Returns:
a generator for instances of the user-defined
MongoUserDict
subclass- Raises:
MongoObjectsAuthFailed – if authorize_pre_read() has been overriden and does not return a truthy value
- classmethod find_one(filter={}, projection=None, readonly=None, object_version=None, no_match=None, **kwargs)¶
Return a single matching document as an instance of this class or None
- Parameters:
filter (dict) – Updated with cls.object_version as appropriate and passed to
pymongo.find()
projection (dict) – Passed to
pymongo.find()
readonly (
None
or bool) – as withMongoUserDict.find()
object_version – as with
find()
no_match (
None
or any value) – Value to return if no matching document is found
- Returns:
no_match value;
None
by default- Raises:
MongoObjectsAuthFailed – if authorize_pre_read() has been overriden and does not return a truthy value
- get_unique_integer(autosave=True)¶
Provide the next unique integer for this document.
These integers are convenient for use as keys of subdocuments. Starts with 1; 0 is reserved for single proxy documents which don’t have a key.
- Parameters:
autosave (bool) – Should the document be saved after the next unique integer is issued
- Returns:
The next unique integer for this document
- Return type:
int
- get_unique_key(autosave=True)¶
Format the next unique integer as a hexidecimal string
- Parameters:
autosave (bool) – passed to
get_unique_integer()
- Returns:
The lowercase hexidecimal string representing the next unique integer for this document.
- Return type:
str
- id()¶
Convert this document’s ObjectId to a string
- Returns:
The document ObjectId
- Return type:
str
- Raises:
KeyError – if the document has not yet been saved and has not been assigned an ObjectId
- classmethod load_by_id(doc_id, **kwargs)¶
Locate a document by its ObjectId
- Parameters:
doc_id (str or bson.ObjectId) – the ObjectId for the desired document
kwargs – passed to
find_one()
- Returns:
an instance of the current class or None if not found. Invalid ObjectIds return None.
- classmethod load_proxy_by_id(id, *args, readonly=False)¶
Based on a full subdocument ID string and a list of classes, load the Mongo parent document, create any intermediate proxies and return the requested proxy object.
- Parameters:
id (str) – a full subdocument ID string separated by cls.subdoc_key_sep. It includes the ObjectId of the top-level MongoDB document and one or more subdocument keys as generated by
id()
.args – one or more user-defined proxy classes in descending order, one per subdocument key. The top-level parent
MongoUserDict
subclass is not included.readonly (bool) – passed to
load_by_id()
- Returns:
an instance of the final, rightmost proxy subdocument class from args
- load_proxy_by_local_id(id, *args, readonly=False)¶
Based on a local subdocument ID string and a list of classes, create any intermediate proxies within the current document and return the requested proxy object.
- Parameters:
id (str) – a local subdocument ID string generated by
proxy_id()
containing one or more subdocument keys separated by cls.subdoc_key_sep.args – one or more user-defined proxy classes in descending order, one per subdocument key.
readonly (bool) – passed to
find_one()
- Returns:
an instance of the final, rightmost proxy subdocument class from args
- proxy_id(*args, include_parent_doc_id=False)¶
Assemble a list of proxy IDs into a single string
- Parameters:
include_parent_doc_id (bool) – whether to include the parent document ID in the resulting ID string
- Returns:
One or more proxy IDs separated by subdoc_key_sep
- Return type:
str
- save(force=False)¶
Intelligent wrapper to insert or replace a document
A current _updated timestamp is added to all documents.
The first time a document is saved, a _created timestamp is added as well.
If the class defines a non-
None
object_version, this added as _objver to the document as well.Documents without an _id are inserted into the database; MongoDB will assign the ObjectId
If force is set, document will be saved regardless of update time or even if it exists. This is useful for upserting documents from another database.
Otherwise, only a database document whose _id and _updated timestamp match this object will be replaced. This protects against overwriting documents that have been updated elsewhere.
- Parameters:
force (bool, optional) – if
True
, upsert the new document regardless of its _updated timestamp- Raises:
MongoObjectsAuthFailed – if authorize_save() has been overriden and does not return a truthy value
- classmethod split_id(subdoc_id)¶
Split a subdocument ID into its component document ID and one or more subdocument keys.
- Parameters:
subdoc_id (str) – a full subdocument ID starting with a document ObjectId followed by one or more subdocument proxy keys separated by cls.subdoc_key_sep.
- Returns:
A list. The first element of the list is the document ObjectId. The remaining elements in the list are subdocument proxy keys.
- Return type:
list
- static utcnow()¶
MongoDB stores milliseconds, not microseconds. Drop microseconds from the standard utcnow() so comparisons can be made with database times.
- Returns:
The current time with microseconds set to 0.
- Return type:
naive
datetime.datetime
Polymorphic Class Reference¶
Polymorphic proxies are supported by PolymorphicMongoUserDict
. All the
attributes and methods of MongoUserDict
are supported with the following
overrides.
- class mongo_objects.PolymorphicMongoUserDict(doc={}, readonly=False)¶
Like MongoUserDict but supports polymorphic document objects within the same collection.
Each subclass needs to define a unique subclass_key
- subclass_map = {}¶
Map subclass_keys to subclasses
Strongly recommended: Override this with an empty dictionary in the base class of your subclass tree to create a separate namespace
- subclass_key¶
Must be unique for each subclass.
One class (usually the base class of the subclass tree) may leave this as None. That subclass will be treated as the default subclass for any documents with a missing or unknown subclass key.
- subclass_key_name = '_sckey'¶
Name of internal key added to each document to record the subclass_key
- classmethod find(filter={}, projection=None, readonly=None, object_version=None, **kwargs)¶
Return matching documents as appropriate subclass instances
- Parameters:
filter (dict) – Updated with cls.object_version as appropriate and passed to
pymongo.find()
projection (dict) – Passed to
pymongo.find()
readonly (
None
or bool) – as withMongoUserDict.find()
object_version – as with
MongoUserDict.find()
- Returns:
a generator for instances of the user-defined
PolymorphicMongoUserDict
subclasses, each correct for the document being returned- Raises:
MongoObjectsAuthFailed – if authorize_pre_read() has been overriden and does not return a truthy value
- classmethod find_one(filter={}, projection=None, readonly=None, object_version=None, no_match=None, **kwargs)¶
Return a single matching document as the appropriate subclass or None
- Parameters:
filter (dict) – Updated with cls.object_version as appropriate and passed to
pymongo.find()
projection (dict) – Passed to
pymongo.find()
readonly (
None
or bool) – as withMongoUserDict.find()
object_version – as with
MongoUserDict.find()
no_match – as with
MongoUserDict.find_one()
- Returns:
an instance of the user-defined
PolymorphicMongoUserDict
subclass correct for this document- Raises:
MongoObjectsAuthFailed – if
authorize_pre_read()
has been overriden and does not return a truthy value
- classmethod get_subclass_by_key(subclass_key)¶
Look up a subclass in the subclass_map by its subclass_key. If the subclass can’t be located, return the class with subclass_key
None
. If there is no such class, raise an exception.- Parameters:
subclass_key (str) – subclass key
- Returns:
polymorphic document subclass
- Raises:
MongoObjectsPolymorphicMismatch – if the subclass key isn’t in the subclass map and no subclass with a
None
key was registered as the default.
- classmethod get_subclass_from_doc(doc)¶
Return the correct subclass to represent this document.
- Parameters:
doc (dict) – document dictionary
- Returns:
polymorphic document subclass
- Raises:
MongoObjectsPolymorphicMismatch – if the document doesn’t have a subclass key, the subclass key isn’t in the subclass map and no subclass with a
None
key was registered as the default.
- save(**kwargs)¶
Add the subclass_key to the document and call
MongoUserDict.save()
- Parameters:
kwargs – passed to
MongoUserDict.save()