Skip to main content

More human readable JSON serializer/de-serializer for MongoEngine

Project description

Build Status Coverage Status Maintainability PyPI version Documentation Status Image

What This?

This script has MongoEngine Document json serialization more-natural.

Why this invented?

Using MongoEngine to create something (e.g. RESTful API), sometimes you might want to serialize the data from the db into JSON, but some fields are weird and not suitable for frontend/api:

  "_id": {
    "$oid": "5700c32a1cbd5856815051ce"
  "name": "Hiroaki Yamamoto",
  "registered_date": {
      "$date": 1459667811724

The points are 2 points:

  • _id might not be wanted because jslint disagrees _ character unless declaring jslint nomen:true
  • There are sub-fields such $oid and $date. These fields are known as MongoDB Extended JSON. However, considering MongoEngine is ODM and therefore it has schema-definition methods, the fields shouldn’t have the special fields. In particular problems, you might get No such property $oid of undefined error when you handle above generated data on frontend.

To solve the problems, the generated data should be like this:

  "id": "5700c32a1cbd5856815051ce",
  "name": "Hiroaki Yamamoto",
  "registered_date": 1459667811724

Making above structure can be possible by doing re-mapping, but if we do it on API’s controller object, the code might get super-dirty:

"""Dirty code."""
import mongoengine as db

class User(db.Document):
  """User class."""
  name = db.StringField(required=True, unique=True)
  registered_date = db.DateTimeField()

def get_user(self):
  """Get user."""
  models = [
      ("id" if key == "_id" else key): (
        value.pop("$oid") if "$oid" in value and isinstance(value, dict)
        else value.pop("$date") if "$date" in value and isinstance(value, dict)
        else value  #What if there are the special fields in child dict?
      for (key, value) in doc.items()
    } for doc in User.objects(pk=ObjectId("5700c32a1cbd5856815051ce"))
  return json.dumps(models, indent=2)

To give the solution of this problem, I developed this scirpt. By using this script, you will not need to make the transform like above. i.e.

"""A little-bit clean code."""

import mongoengine as db
import mongoengine_goodjson as gj

class User(gj.Document):
  """User class."""
  name = db.StringField(required=True, unique=True)
  registered_date = db.DateTimeField()

def get_user(self):
  """Get user."""
  return model_cls.objects(

How to use it

Generally you can define the document as usual, but you might want to inherits mongoengnie_goodjson.Document or mongoengnie_goodjson.EmbeddedDocument.

Here is the example:

"""Example schema."""

import mongoengine_goodjson as gj
import mongoengine as db

class Address(gj.EmbeddedDocument):
    """Address schema."""

    street = db.StringField()
    city = db.StringField()
    state = db.StringField()

class User(gj.Document):
    """User data schema."""

    name = db.StringField()
    email = db.EmailField()
    address = db.EmbeddedDocumentListField(Address)

More details… there’s the doc!

If you want to know more, there’s read the doc that you want to read. You can now read the doc with drinking a cup of coffee!!


This scirpt is coded on TDD. i.e. Writing a test that fails, and then write the actual code to pass the test. Therefore, virtualenv, nose and tox will be needed to code this script (however, using detox instead of tox might be more effective.). Please read the doc for the detail.

License (MIT License)


Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for mongoengine-goodjson, version 1.1.2
Filename, size File type Python version Upload date Hashes
Filename, size mongoengine_goodjson-1.1.2.tar.gz (33.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page