Skip to main content

REST extension for Emmett framework

Project description

Emmett-REST

Emmett-REST is a REST extension for Emmett framework.

pip version Tests Status

In a nutshell

from emmett.orm import Model, Field
from emmett_rest import REST

class Task(Model):
    title = Field.string()
    is_completed = Field.bool()

app.use_extension(REST)

tasks = app.rest_module(__name__, 'api_task', Task, url_prefix='tasks')
tasks.query_allowed_fields = ['is_completed']

Output of http://{host}/tasks?where={"is_completed": false}:

{
    "meta": {
        "object": "list",
        "total_objects": 1,
        "has_more": false
    },
    "data": [
        {
            "title": "Some task",
            "is_completed": false
        }
    ]
}

Installation

You can install Emmett-REST using pip:

pip install Emmett-REST

And add it to your Emmett application:

from emmett_rest import REST

app.use_extension(REST)

Usage

The Emmett-REST extension is intended to be used with Emmett models, and it uses application modules to build APIs over them.

Let's say, for example, that you have a task manager app with a Task model:

from emmett.orm import Model, Field

class Task(Model):
    title = Field.string()
    is_completed = Field.bool()
    spent_time = Field.int()
    created_at = Field.datetime()

Then, in order to expose REST apis for your Task model, you can use the rest_module method on your application or on any application module:

from myapp import app, Task

tasks = app.rest_module(__name__, 'api_task', Task, url_prefix='tasks')

As you can see, the usage is very similar to the Emmett application modules, but we also passed the involved model to the module initialization.

This single line is enough to have a really simple REST api over the Task model, since under the default behaviour rest modules will expose 5 different routes:

  • an index route that will respond to GET requests on /tasks path listing all the tasks in the database
  • a read route that will respond to GET requests on /tasks/<int:rid> path returning a single task corresponding to the record id of the rid variable
  • a create route that will respond to POST requests on /tasks that will create a new task in the database
  • an update route that will respond to PUT or PATCH requests on /tasks/<int:rid> that will update the task corresponding to the record id of the rid variable
  • a delete route that will respond to DELETE requests on /tasks/<int:rid> that will delete the task corresponding to the record id of the rid variable.

Listing endpoints accepts some query parameters by default, and specifically:

parameter default description
page 1 page to return
page_size 20 number of records per page to return
sort_by id field to sort records by (-field for desc)
where {} json dumped string for querying

Additional routes

Emmett-REST also provides some additional not CRUD endpoints. Let's see them specifically.

Sample route

It behaves like the index route, but gets records randomly. As a consequence, the sort_by parameter gets ignored.

Responds to GET requests on {base_path}/sample endpoint, and can be activated adding 'sample' to the enabled methods on module definition or in the extension settings.

Grouping route

Responds to GET requests on {base_path}/group/<str:field> endpoint, and can be activated adding 'group' to the enabled methods on module definition or in the extension settings.

It groups by value and count records for the given field. Results of calling http://{host}/tasks/group/is_completed looks like:

{
    "meta": {
        "object": "list",
        "total_objects": 11,
        "has_more": false
    },
    "data": [
        {
            "value": true,
            "count": 1
        },
        {
            "value": false,
            "count": 10
        }
    ]
}

Stats route

Responds to GET requests on {base_path}/stats endpoint, and can be activated adding 'stats' to the enabled methods on module definition or in the extension settings. Parse a list of fields from the comma separated fields query parameter.

It return minimum, maximum and average values of the records for the specified fields. Results of calling http://{host}/tasks/stats?fields=spent_time looks like:

{
    "spent_time": {
        "min": 0,
        "max": 3600,
        "avg": 27
    }
}

REST module parameters

The rest_module method accepts several parameters (bold ones are required) for its configuration:

parameter default description
import_name as for standard modules
name as for standard modules
model the model to use
serializer None a class to be used for serialization
parser None a class to be used for parsing
enabled_methods str list: index, create, read, update, delete the routes that should be enabled on the module
disabled_methods [] the routes that should be disabled on the module
list_envelope data the envelope to use on the index route
single_envelope False the envelope to use on all the routes except for lists endpoints
meta_envelope meta the envelope to use for meta data
groups_envelope data the envelope to use for the grouping endpoint
use_envelope_on_parse False if set to True will use the envelope specified in single_envelope option also on parsing
serialize_meta True whether to serialize meta data on lists endpoint
url_prefix None as for standard modules
hostname None as for standard modules
module_class RestModule the module class to use

REST module properties

Some of the REST module parameters needs to be configured using attributes, specifically:

attribute description
allowed_sorts list of fields that can be used with sort_by parameter
query_allowed_fields list of fields that can be used with where parameter
grouping_allowed_fields list of fields that can be used in group route
stats_allowed_fields list of fields that can be used in stats route

An example would be:

tasks.allowed_sorts = ['id', 'created_at']
tasks.query_allowed_fields = ['is_completed']
tasks.grouping_allowed_fields = ['is_completed']
tasks.stats_allowed_fields ['time_spent']

Customizing the database set

Under default behavior, any REST module will use Model.all() as the database set on every operation.

When you need to customize it, you can use the get_dbset decorator. For example, you may gonna use the Emmett auth module:

from myapp import auth

@tasks.get_dbset
def fetch_tasks():
    return auth.user.tasks

or you may have some soft-deletions strategies and want to expose just the records which are not deleted:

@tasks.get_dbset
def fetch_tasks():
    return Task.where(lambda t: t.is_deleted == False)

Customizing single row fetching

Under default behaviour, any REST module will use standard select to fetch the record on the read endpoint.

When you need to customize it, you can use the get_row decorator.

For example, you may want to add a left join to the selection:

@tasks.get_row
def fetch_row(dbset):
    return dbset.select(
        including=['many_relation'], limitby=(0, 1)
    ).first()

Customizing routed methods

You can customize every route of the REST module using its index, create, read, update and delete decorators. In the next examples we'll override the routes with the default ones, in order to show the original code behind the default routes.

from emmett import request

@tasks.index()
async def task_list(dbset):
    pagination = tasks.get_pagination()
    sort = tasks.get_sort()
    rows = dbset.select(paginate=pagination, orderby=sort)
    return tasks.serialize_many(rows, dbset, pagination)

As you can see, an index method should accept the dbset parameter, that is injected by the module. This is the default one or the one you defined with the get_dbset decorator.

@tasks.read()
async def task_single(row):
    return tasks.serialize_one(row)

The read method should accept the row parameter that is injected by the module. Under default behaviour the module won't call your method if it doesn't find the requested record, but instead will return a 404 HTTP response.

@tasks.create()
async def task_new():
    response.status = 201
    attrs = await tasks.parse_params()
    r = Task.create(**attrs)
    if r.errors:
        response.status = 422
        return tasks.error_422(r.errors)
    return tasks.serialize_one(r.id)

The create method won't need any parameters, and is responsible of creating new records in the database.

@tasks.update()
async def task_edit(dbset, rid):
    attrs = await tasks.parse_params()
    r = dbset.where(Task.id == rid).validate_and_update(**attrs)
    if r.errors:
        response.status = 422
        return tasks.error_422(r.errors)
    elif not r.updated:
        response.status = 404
        return tasks.error_404()
    row = Task.get(rid)
    return tasks.serialize_one(row)
@tasks.delete()
async def task_del(dbset, rid):
    r = dbset.where(Task.id == rid).delete()
    if not r:
        response.status = 404
        return self.error_404()
    return {}

The update and delete methods are quite similar, since they should accept the dbset parameter and the rid one, which will be the record id requested by the client.

All the decorators accept an additional pipeline parameter that you can use to add custom pipes to the routed function:

@tasks.index(pipeline=[MyCustomPipe()])
def task_index:
    # code

Customizing errors

You can define custom methods for the HTTP 400, 404 and 422 errors that will generate the JSON output using the on_400, on_404 and on_422 decorators:

@tasks.on_400
def task_400err():
    return {'error': 'this is my 400 error'}

@tasks.on_404
def task_404err():
    return {'error': 'this is my 404 error'}
    
@tasks.on_422
def task_422err(errors):
    return {'error': 422, 'validation': errors.as_dict()}

Customizing meta generation

You can define custom method for the meta generation using the appropriate meta_builder decorator:

@tasks.meta_builder
def _tasks_meta(self, dbset, pagination):
    count = dbset.count()
    page, page_size = pagination
    total_pages = math.ceil(count / page_size)
    return {
        'page': page,
        'page_prev': page - 1 if page > 1 else None,
        'page_next': page + 1 if page < total_pages else None,
        'total_pages': total_pages,
        'total_objects': count
    }

Serialization

Under the default behaviour, the REST extension will use the form_rw attribute of the involved model, and overwrite the results with the contents of the rest_rw attribute if present.

For example, with this model:

from emmett.orm import Model, Field

class Task(Model):
    title = Field.string()
    is_completed = Field.bool()
    created_at = Field.datetime()
    
    form_rw = {
        'id': False,
        'created_at': False
    }

the REST extension will serialize just the title and the is_completed fields, while with this:

from emmett.orm import Model, Field

class Task(Model):
    title = Field.string()
    is_completed = Field.bool()
    created_at = Field.datetime()
    
    form_rw = {
        'id': False,
        'created_at': False
    }
    
    rest_rw = {
        'id': True
    }

the REST extension will serialize also the id field.

Serializers

Whenever you need more control over the serialization, you can use the Serializer class of the REST extension:

from emmett_rest import Serializer

class TaskSerializer(Serializer):
    attributes = ['id', 'title']
    
tasks = app.rest_module(
    __name__, 'api_task', Task, serializer=TaskSerializer, url_prefix='tasks')

Serializers are handy when you want to add custom function to serialize something present in your rows. For instance, let's say you have a very simple tagging system:

from emmett.orm import belongs_to, has_many

class Task(Model):
    has_many({'tags': 'TaskTag'})

class TaskTag(Model):
    belongs_to('task')
    name = Field.string()

and you want to serialize the tags as an embedded list in your task. Then you just have to add a tags method to your serializer:

class TaskSerializer(Serializer):
    attributes = ['id', 'title']
    
    def tags(self, row):
        return row.tags().column('name')

This is the complete list of rules that the extension will take over serializers:

  • attributes is read as first step
  • the form_rw and rest_rw attributes of the model are used to fill attributes list when this is empty
  • the fields in the include list will be added to attributes
  • the fields in the exclude list will be removed from attributes
  • every method defined in the serializer not starting with _ will be called over serialization and its return value will be added to the JSON object in a key named as the method

You can also use different serialization for the list route and the other ones:

from emmett_rest import Serializer, serialize

class TaskSerializer(Serializer):
    attributes = ['id', 'title']
    
class TaskDetailSerializer(TaskSerializer):
    include = ['is_completed']
    
tasks = app.module(
    __name__, 'api_task', Task, 
    serializer=TaskDetailSerializer, url_prefix='tasks')

@tasks.index()
def task_list(dbset):
    rows = dbset.select(paginate=tasks.get_pagination())
    return serialize(rows, TaskSerializer)

Note: under default behaviour the serialize method will use the serializer passed to the module.

Parsing input

Opposite to the serialization, you will have input parsing to parse JSON requests and perform operations on the records.

Under the default behaviour, the REST extension will use the form_rw attribute of the involved model, and overwrite the results with the contents of the rest_rw attribute if present.

For example, with this model:

from emmett.orm import Model, Field

class Task(Model):
    title = Field.string()
    is_completed = Field.bool()
    created_at = Field.datetime()
    
    form_rw = {
        'id': False,
        'created_at': False
    }

the REST extension will parse the input to allow just the title and the is_completed fields, while with this:

from emmett.orm import Model, Field

class Task(Model):
    title = Field.string()
    is_completed = Field.bool()
    created_at = Field.datetime()
    
    form_rw = {
        'id': False,
        'created_at': False
    }
    
    rest_rw = {
        'id': (True, False)
        'created_at': True
    }

the REST extension will allow also the created_at field.

Parsers

Very similarly to the Serializer class, the extension provides also a Parser one:

from emmett_rest import Parser

class TaskParser(Parser):
    attributes = ['title']
    
tasks = app.rest_module(
    __name__, app, 'api_task', Task, parser=TaskParser, url_prefix='tasks')

As for serializers, you can define attributes, include and exclude lists in a parser, and add custom methods that will parse the params:

class TaskParser(Parser):
    attributes = ['title']
    
    def created_at(self, params):
        # some code

and you also have the envelope attribute at your disposal in case you expect to have enveloped bodies over POST, PUT and PATCH requests:

class TaskParser(Parser):
    envelope = "task"

The Parser class also offers some decorators you might need in your code: parse_value and processor.

While the first one might be handy in conditions where you need to edit a single attribute:

class TaskParser(Parser):
    _completed_map = {"yes": True, "no": False}

    @Parser.parse_value("is_completed")
    def parse_completed(self, value):
        return self._completed_map[value]

the latter gives you access to all the parameters and the parsed dictionary, so you can deeply customise the parsing flow:

class TaskParser(Parser):
    @Parser.processor()
    def custom_parsing(self, params, obj):
        obj.status = "completed" if params.is_completed else "running"

Pagination

REST modules perform pagination over the listing routes under the default behaviour. This is performed with the paginate option during the select and the call to the get_pagination method.

You can customize the name of the query params or the default page sizes with the extension configuration, or you can override the method completely with subclassing.

Callbacks

Unless overridden, the default create, update and delete methods invoke callbacks you can attach to the module using the approriate decorator. Here is the complete list:

callback arguments description
before_create [sdict] called before the record insertion
before_update [int, sdict] called before the record gets update
after_parse_params [sdict] called after params are loaded from the request body
after_create [Row] called after the record insertion
after_update [Row] called after the record gets updated
after_delete [int] called after the record gets deleted

For example, you might need to notify some changes:

@tasks.after_create
def _notify_task_creation(row):
    my_publishing_system.notify(
        f"Task {row['title']} was added"
    )

Query language

The where query parameter allow, within the fields specified in query_allowed_fields, to query records in the listing routes using a JSON object.

The query language is inspired to the MongoDB query language, and provides the following operators:

operator argument type description
$and List[Dict[str, Any]] logical AND
$or List[Dict[str, Any]] logical OR
$not Dict[str, Any] logical NOT
$eq Any matches specified value
$ne Any inverse of $eq
$in List[Any] matches any of the values in list
$nin List[Any] inverse of $in
$lt Union[int, float, str] matches values less than specified value
$gt Union[int, float, str] matches values greater than specified value
$lte Union[int, float, str] matches values less than or equal to specified value
$gte Union[int, float, str] matches values greater than or equal to specified value
$exists bool matches not null or null values
$regex str matches specified regex expression
$iregex str case insensitive $regex

The JSON condition always have fields' names as keys (except for $and, $or, $not) and operators as values, where $eq is the default one:

{
    "is_completed": false,
    "priority": {"$gte": 5}
}

Customizing REST modules

Extension options

This is the list of all the configuration variables available on the extension for customization – the default values are set:

app.config.REST.default_module_class = RESTModule
app.config.REST.default_serializer = Serializer
app.config.REST.default_parser = Parser
app.config.REST.page_param = 'page'
app.config.REST.pagesize_param = 'page_size'
app.config.REST.sort_param = 'sort_by'
app.config.REST.query_param = 'where'
app.config.REST.min_pagesize = 10
app.config.REST.max_pagesize = 25
app.config.REST.default_pagesize = 20
app.config.REST.default_sort = 'id'
app.config.REST.base_path = '/'
app.config.REST.base_id_path = '/<int:rid>'
app.config.REST.list_envelope = 'data'
app.config.REST.single_envelope = False
app.config.REST.groups_envelope = 'data'
app.config.REST.use_envelope_on_parse = False
app.config.REST.serialize_meta = True
app.config.REST.meta_envelope = 'meta'
app.config.REST.default_enabled_methods = [
    'index',
    'create',
    'read',
    'update',
    'delete'
]
app.config.REST.default_disabled_methods = []

This configuration will be used by all the REST modules you create, unless overridden.

Subclassing

Under the default behavior, every REST module will use the RESTModule class. You can create as many subclasses from this one when you need to apply the same behaviour to several modules:

from emmett_rest import RESTModule

class MyRESTModule(RESTModule):
    def init(self):
        self.disabled_methods = ['delete']
        self.index_pipeline.append(MyCustomPipe())
        self.list_envelope = 'objects'
        self.single_envelope = self.model.__name__.lower()
        
    def _get_dbset(self):
        return self.model.where(lambda m: m.user == session.user.id)
        
    async def _index(self, dbset):
        rows = dbset.select(paginate=self.get_pagination())
        rv = self.serialize_many(rows)
        rv['meta'] = {'total': dbset.count()}
        return rv
        
tasks = app.rest_module(
    __name__, app, 'api_task', Task, url_prefix='tasks', 
    module_class=MyRESTModule)
tags = app.rest_module(
    __name__, app, 'api_tag', Tag, url_prefix='tags',
    module_class=MyRESTModule)

As you can see, we defined a subclass of the RESTModule one and used the init method to customize the class initialization for our needs. We strongly recommend to use this method and avoid overriding the __init__ of the class unless you really know what you're doing.

Using the init method, we disabled the delete route over the module, added a custom pipe over the index route and configured the envelope rules.

Here is a list of variables you may want to change inside the init method:

  • model
  • serializer
  • parser
  • enabled_methods
  • disabled_methods
  • list_envelope
  • single_envelope
  • meta_envelope
  • groups_envelope
  • use_envelope_on_parsing

Also, this is the complete list of the pipeline variables and their default values:

def init(self):
    self.index_pipeline = [SetFetcher(self), self._json_query_pipe]
    self.create_pipeline = []
    self.read_pipeline = [SetFetcher(self), RecordFetcher(self)]
    self.update_pipeline = [SetFetcher(self)]
    self.delete_pipeline = [SetFetcher(self)]
    self.group_pipeline = [
        self._group_field_pipe,
        SetFetcher(self),
        self._json_query_pipe
    ]
    self.stats_pipeline = [
        self._stats_field_pipe,
        SetFetcher(self),
        self._json_query_pipe
    ]
    self.sample_pipeline = [SetFetcher(self), self._json_query_pipe]

We've also overridden the methods for the database set retrieval and the index route. As you can see, these methods are starting with the _ since are the default ones and you can still override them with decorators. This is the complete list of methods you may want to override instead of using decorators:

  • _get_dbset
  • _get_row
  • _index
  • _create
  • _read
  • _update
  • _delete
  • _group
  • _stats
  • _sample
  • _build_meta
  • build_error_400
  • build_error_404
  • build_error_422

There are some other methods you may need to override, like the get_pagination one or the serialization ones. Please, check the source code of the RESTModule class for further needs.

License

Emmett-REST is released under BSD license. Check the LICENSE file for more details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Emmett-REST-1.2.0.tar.gz (30.8 kB view hashes)

Uploaded Source

Built Distribution

Emmett_REST-1.2.0-py3-none-any.whl (24.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page