yadm 2.0.9

Yet Another Document Mapper (ODM) for MongoDB

It’s small and simple ODM for use with MongoDB.

Requirements

YADM support MongoDB version 3.x only. MongoDB 2.x is not supported.

Minimal version of python — 3.6.

Quick start

Create the models

from datetime import datetime

import pymongo

from yadm import Database
from yadm import Document, EmbeddedDocument
from yadm import fields
from yadm.serialize import to_mongo


class User(Document):
    __collection__ = 'users'

    name = fields.StringField()
    email = fields.EmailField()


class PostLogItem(EmbeddedDocument):
    op = fields.StringField(choices=['created', 'comment_added'])
    at = fields.DatetimeField()
    data = fields.MongoMapField()


class Post(Document):
    __collection__ = 'posts'

    user = fields.ReferenceField(User)
    created_at = fields.DatetimeField(auto_now=True)
    title = fields.StringField()
    body = fields.StringField()
    log = fields.ListField(fields.EmbeddedDocumentField(PostLogItem))


class Comment(Document):
    __collection__ = 'comments'

    user = fields.ReferenceField(User)
    created_at = fields.DatetimeField(auto_now=True)
    post = fields.ReferenceField(Post)
    text = fields.StringField()

All documents creates from class Document. You can use multiple inheritance.

__collection__ magic attribute setups the collection name for documents of model.

Connect to database

client = pymongo.MongoClient('mongodb://localhost:27017')
db = Database(client, 'blog')

Database object is a wrapper about pymongo or motor Database.

Create documents

User

user = User(name='Bill', email='[e-mail]')
db.insert_one(user)

Just insert document to database.

Post

post = Post()
post.user = user
post.title = 'Small post'
post.body = 'Bla-bla-bla...'
post.log = [PostLogItem(op='created', at=datetime.utcnow())]
db.insert_one(post)

You can fill documents as above.

Comment the post

comment = Comment()
comment.user = user
comment.post = post
comment.text = "RE: Bla-bla-bla..."
db.insert_one(comment)
db.update_one(post, push={
    'log': to_mongo(PostLogItem(op='comment_added',
                                at=comment.created_at,
                                data={
                                  'comment': comment.id,
                                  'user': comment.user.id,
                                }))
})

We add log item to post’s log. This is very usefull case.

Queries

find

qs = db(Post).find({'title': {'$regex': '^S'}})
assert qs.count() > 0

1. db(Post) creates the QuerySet object;

2. find method get the raw-query and return new QuerySet object with updated criteria;

3. count method make the query to database and return value.

for post in qs:
    assert post.title.startswith('S')

__iter__ method make the find-query and returns the generator of documents.

find_one

Get the first finded document.

post = db(Post).find_one({'user': user.id})

get_document

Get the document by id from primary.

user = db.get_document(User, user.id)

References

user = post.user

Get attribute with reference makes the query to referred collection. Warning: N+1 problem! We have a cache in QuerySet object and get one referred document only once for one queryset.

Lookups

comments = db(Comment).find({'post': post.id}).sort(('created_at', 1))
for comment in comments.lookup('user'):
    print(comment.user.name, comment.text)

This code create the aggregate query with $lookup statement for resolve the references.

Aggregations

agg = (db.aggregate(Comment)
       .match(user=user.id)
       .group(_id='post', count={'$sum': 1})
       .sort(count=-1))

for item in agg:
    print(item)

Or traditional MongoDB syntax:

agg = db.aggregate(Comment, pipeline=[
    {'match': {'user': user.id}},
    {'group': {'_id': 'post', 'count': {'$sum': 1}}},
    {'sort': {'count': -1}},
])

CHANGES

2.0.9 (2023-08-23)

• Add comment methods for QuerySet and Aggregation to specify comment feature of MongoDB.

2.0.8 (2021-09-23)

• Asyncio support for testing.

2.0.7 (2021-04-21)

• Some bugfixes.

2.0.5 (2019-02-25)

• Add Aggregation.hint method.

2.0.4 (2019-02-20)

• Add Database.estimated_document_count method for quickly count documents in the collection.

2.0.1 (2018-11-04)

• Add QuerySet.hint for specify index for query.

2.0.0 (2018-10-25)

• A Big Rewrite document logic:

  • Document.__raw__ now contains only data from pymongo, without any AttributeNotSet or NotLoaded;

  • Document.__changed__ is removed: all changes reflects to Document.__cache__;

  • Document.__not_loaded__ frozenset of fields whitch not loaded by projection;

  • Document.__new_document__ flag is True for document’s objects whitch created directly in your code;

  • Document.__log__ list-like container with log of document changes (unstable API at now);

  • Document.__data__ is removed as deprecated;

  • Now is not allow to set fields as classes;

  • Defaults is not lazy and creates with document instance;

• Update for minimal versions of pymongo (3.7) and motor (2.0):

  • Add Database.bulk_write;

  • Add Database.insert_one, Database.insert_many and Database.delete_one;

  • Deprecate Database.insert, Database.remove;

  • Remove Database.bulk (without deprecation period, sorry);

  • Add QuerySet.count_documents;

  • Add QuerySet.update_one and QuerySet.update_many;

  • Add QuerySet.delete_one and QuerySet.delete_many;

  • Add QuerySet.find_one_and_update, QuerySet.find_one_and_replace and QuerySet.find_one_and_delete;

  • Deprecate QuerySet.count;

  • Deprecate QuerySet.update, QuerySet.remove and QuerySet.find_and_modify;

  • Remove deprecated QuerySet.with_id;

• Simple interface for build lookups: QuerySet.lookup;

• Remove bcc argument from MoneyField;

• Add Decimal128Field.

1.5.0 (2017-12-31)

• Experimental asyncio support;

• Add ReferencesListField for lists of references.

1.4.15 (2017-12-27)

• Add projection argument to Database.get_document and Database.reload;

• Add Document.__default_projection__ attribute.

1.4.14 (2017-11-06)

• Add EnumField for save enum.Enum;

• Add EnumStateField for simple state machines based on enum.Enum.

1.4.13 (2017-10-31)

• Add QuerySet.batch_size method for setup batch size for cursor;

• Some minor fixes.

1.4.10 (2017-07-07)

• ReferenceField.from_mongo try to get document from primary

  if not found by default.

1.4.9 (2017-07-06)

• Add QuerySet.read_primary method for simple setup read_preference.Primary.

1.4.4 (2017-05-17)

• Add TimedeltaField for stores durations;

• Add SimpleEmbeddedDocumentField for simply create embedded documents.

class Doc(Document):
    embedded = SimpleEmbeddedDocumentField({
        'i': IntegerField(),
        's': StringField(),
    })

1.4.3 (2017-05-14)

• Add StaticField for static data.

1.4.2 (2017-04-09)

• Additional arguments (like write_concern) for write operations;

• create_fake save the documents with write concern “majority” by default.

1.4.0 (2017-04-05)

• Drop pymongo 2 support;

• Additional options for databases and collections;

• Add Database.get_document;

• Add TypedEmbeddedDocumentField;

• reload argument of Database.update_one must be keyword

  (may be backward incompotable).

1.3.1 (2017-02-21)

• Change raw data for Money;

1.3.0 (2017-02-19)

• Add currency support to Money:

  • Totaly rewrite Money type. Now it is not subclass of Decimal;

  • Add storage for currencies: yadm.fields.money.currency.DEFAULT_CURRENCY_STORAGE;

1.2.1 (2017-01-19)

• Add QuerySet.find_in for $in queries with specified order;

1.2.0 (2016-12-27)

• Drop MongoDB 2.X suport;

• Objects for update and remove results;

• Use Faker instead fake-factory.

1.1.4 (2016-08-20)

• Add some features to Bulk:

  • Bulk.update_one(document, **kw): method for add update one document in bulk;

  • Bulk.find(query).update(**kw): update many documents by query;

  • Bulk.find(query).upsert().update(**kw): upsert document;

  • Bulk.find(query).remove(**kw): remove documents;

1.1.3 (2016-07-23)

• Add QuerySet.ids method for get only documents id’s from queryset;

• Add Money.total_cents method and Money.from_cents classmethod;

1.1 (2016-04-26)

• Add cacheing on queryset level and use it for ReferenceField;

• Add mongo aggregation framework support;

• Add read_preference setting;

• Add exc argument to QuerySet.find_one for raise exception if not found;

• Add multi argument to QuerySet.remove;

• Deprecate QuerySet.with_id;

• Refactoring.

1.0 (2015-11-14)

• Change document structure. No more bad BaseDocument.__data__ attribute:

  • BaseDocument.__raw__: raw data from mongo;

  • BaseDocument.__cache__: cached objects, casted with fields;

  • BaseDocument.__changed__: changed objects.

• Changes api for custom fields:

  • Not more need create field descriptors for every field;

  • prepare_value called only for setattr;

  • to_mongo called only for save objects to mongo;

  • from_mongo called only for load values from BaseDocument.__raw__;

  • Remove Field.default attribute. Use Field.get_default method;

  • Add Field.get_if_not_loaded and Field.get_if_attribute_not_set method;

  • By default raise NotLoadedError if field not loaded from projection;

• Changes in ReferenceField:

  • Raise BrokenReference if link is bloken;

  • Raise NotBindingToDatabase if document not saved to database;

• smart_null keyword for Field;

• Fields in document must be instances (not classes!);

• Remove ArrayContainer and ArrayContainerField;

• Remove old MapIntKeysField and MapObjectIdKeysField. Use new MapCustomKeysField;

• Add Database.update_one method for run simple update query with specified document;

• Add QuerySet.distinct;

• serialize.from_mongo now accept not_loaded sequence with filed names who must mark as not loaded, parent and name;

• serialize.to_mongo do not call FieldDescriptor.__set__;

• Fakers! Subsystem for generate test objects;

• Tests now use pytest;

• And more, and more…