Skip to main content

Kinto client

Project description

https://img.shields.io/travis/Kinto/kinto-http.py.svg https://img.shields.io/pypi/v/kinto-http.svg https://coveralls.io/repos/Kinto/kinto-http.py/badge.svg?branch=master

Kinto is a service that allows users to store and synchronize arbitrary data, attached to a user account. Its primary interface is HTTP.

kinto-http is a Python library that eases the interactions with a Kinto server instance. A project with related goals is also available for JavaScript.

Installation

Use pip:

$ pip install kinto-http

Usage

  • The first version of this API doesn’t cache any access nor provides any refresh mechanism. If you want to be sure you have the latest data available, issue another call.

Here is an overview of what the API provides:

from kinto_http import Client

client = Client(server_url="http://localhost:8888/v1",
                auth=('alexis', 'p4ssw0rd'))

records = client.get_records(bucket='default', collection='todos')
for i, record in enumerate(records):
    record['title'] = 'Todo {}'.format(i)

for record in records:
    client.update_record(record)

Creating a client

The passed auth parameter is a requests authentication policy, allowing authenticating using whatever scheme fits you best.

By default, Kinto supports Firefox Accounts and Basic authentication policies.

from kinto_http import Client
credentials = ('alexis', 'p4ssw0rd')

client = Client(server_url='http://localhost:8888/v1',
                auth=credentials)

It is also possible to pass the bucket and the collection to the client at creation time, so that this value will be used by default.

auth = ("token", "secret")
client = Client(bucket="payments", collection="receipts", auth=auth)

After creating a client, you can also replicate an existing one and overwrite some key arguments.

client2 = client.clone(collection="orders")

Using a Bearer token to authenticate

import requests

class BearerTokenAuth(requests.auth.AuthBase):
    def __init__(self, token):
        self.token = token

    def __call__(self, r):
        r.headers['Authorization'] = 'Bearer ' + self.token
        return r

auth = BearerTokenAuth("a67fjnewgre5")
client = Client(bucket="main", collection="tippytop", auth=auth)

Using FxA from a script with the email/password

from fxa.plugins.requests import FxABearerTokenAuth

auth = FxABearerTokenAuth(
    email, passwd,
    scopes=['kinto'],
    client_id="<FXA-CLIENT-ID>",
    account_server_url='https://api.accounts.firefox.com/v1',
    oauth_server_url='https://oauth.accounts.firefox.com/v1',
)
client = Client(bucket="payments", collection="receipts", auth=auth)

Getting server information

You can use the server_info method to get the server information:

from kinto_http import Client

client = Client(server_url='http://localhost:8888/v1')
info = client.server_info()
assert 'schema' in info['capabilities'], "Server doesn't support schema validation."

Handling buckets

All operations are rooted in a bucket. It makes little sense for one application to handle multiple buckets at once (but it is possible). If no specific bucket name is provided, the “default” bucket is used.

from kinto_http import Client
from kinto_http.patch_type import BasicPatch, MergePatch, JSONPatch
credentials = ('alexis', 'p4ssw0rd')

client = Client(server_url='http://localhost:8888/v1',
                auth=credentials)

# To create a bucket.
client.create_bucket(id='payments')

# To get an existing bucket
bucket = client.get_bucket(id='payments')

# Or retrieve all readable buckets.
buckets = client.get_buckets()

# To create or replace an existing bucket.
client.update_bucket(id='payments', data={'description': 'My payments data.'})

# Or modify some fields in an existing bucket.
# The Kinto server supports different types of patches, which can be used from kinto_http.patch_type.
client.patch_bucket(id='payments', changes=BasicPatch({'status': 'updated'}))

# It is also possible to manipulate bucket permissions (see later)
client.patch_bucket(id='payments', changes=BasicPatch(permissions={}))

# Or delete a bucket and everything under.
client.delete_bucket(id='payment')

# Or even every writable buckets.
client.delete_buckets()

Groups

A group associates a name to a list of principals. It is useful in order to handle permissions.

# To create a group.
client.create_group(id='receipts', bucket='payments', data={'members': ['blah', 'foo']})

# Or get an existing one.
group = client.get_group(id='receipts', bucket='payments')

# Or retrieve all groups in the bucket.
groups = client.get_groups(bucket='payments')

# To create or replace an existing bucket.
client.update_group(id='receipts', bucket='payments', 'data'={'members':['foo']})

# Or modify some fields in an existing group.
# This uses the server's support for JSON patch, but any patch_type is accepted.
client.patch_group(id='receipts', bucket='payments',
    changes=JSONPatch([{'op': 'add', 'path': '/data/members/0', 'value': 'ldap:user@corp.com'}]))

# To delete an existing group.
client.delete_group(id='receipts', bucket='payments')

# Or all groups in a bucket.
client.delete_groups(bucket='payments')

Collections

A collection is where records are stored.

# To create a collection.
client.create_collection(id='receipts', bucket='payments')

# Or get an existing one.
collection = client.get_collection(id='receipts', bucket='payments')

# Or retrieve all of them inside a bucket.
collections = client.get_collections(bucket='payments')

# To create or replace an exiting collection.
client.update_collection(id='receipts', bucket='payments', data={'description':'bleeh'})

# Or modify some fields of an existing collection.
client.patch_collection(id='receipts', bucket='payments', changes=MergePatch({'status':'updated'}))

# To delete an existing collection.
client.delete_collection(id='receipts', bucket='payments')

# Or every collections in a bucket.
client.delete_collections(bucket='payments')

Records

Records can be retrieved from and saved to collections.

A record is a dict with the “permissions” and “data” keys.

 # You can pass a python dictionary to create the record.
 client.create_record(data={'status': 'done', title: 'Todo #1'},
                      collection='todos', bucket='default')

 # You can use id to specify the record id when creating it.
 client.create_record(id='todo2', data={'status': 'doing', 'title': 'Todo #2'},
                      collection='todos', bucket='default')

 # Or get an existing one by its id.
 record = client.get_record(id='todo2', collection='todos', bucket='default')

 # Or retrieve all records.
 records = client.get_records(collection='todos', bucket='default')

# Or retrieve records page by page.
for page in client.get_paginated_records(collection='todos', bucket='default'):
   # Do something with each page
   print(page)

 # Or retrieve records timestamp.
 records_timestamp = client.get_records_timestamp(collection='todos', bucket='default')

 # To replace a record using a previously fetched record
 client.update_record(data=record, collection='todos', bucket='default')

 # Or create or replace it by its id.
 client.update_record(data={'status': 'unknown'}, id='todo2', collection='todos', bucket='default')

 # Or modify some fields in an existing record.
 client.patch_record(changes=MergePatch({'assignee': 'bob'}), id='todo2', collection='todos', bucket='default')

 # To delete an existing record.
 client.delete_record(id='89881454-e4e9-4ef0-99a9-404d95900352',
                      collection='todos')

 # Or every records of a collection.
 client.delete_records(collection='todos')

Permissions

By default, authors will get read and write access to the manipulated objects. It is possible to change this behavior by passing a dict to the permissions parameter.

client.create_record(
    data={'foo': 'bar'},
    permissions={'read': ['group:groupid']},
    collection='todos')

Buckets, collections and records have permissions which can be edited. For instance to give access to “leplatrem” to a specific record, you would do:

record = client.get_record(1234, collection='todos', bucket='alexis')
record['permissions']['write'].append('leplatrem')
client.update_record(record)

# During creation, it is possible to pass the permissions dict.
client.create_record(data={'foo': 'bar'}, permissions={})

Get or create

In some cases, you might want to create a bucket, collection, group or record only if it doesn’t exist already. To do so, you can pass the if_not_exists=True to the create_* methods:

client.create_bucket(id='bucket', if_not_exists=True)

Delete

In some cases, you might want to delete a bucket, collection, group or record only if it exists already. To do so, you can pass the if_exists=True to the delete_* methods:

client.delete_bucket(id='bucket', if_exists=True)

Overwriting existing objects

Most of the methods take a safe argument, which defaults to True. If set to True and a last_modified field is present in the passed data, or if the if_match parameter is specified then a check will be added to the requests to ensure the record wasn’t modified on the server side in the meantime.

Batching operations

Rather than issuing a request for each and every operation, it is possible to batch the requests. The client will then issue as little requests as possible.

It is possible to do batch requests using a Python context manager (with):

with client.batch() as batch:
    for idx in range(0, 100):
        batch.update_record(data={'id': idx})

Reading data from batch operations is achieved by using the results() method available after a batch context is closed.

with client.batch() as batch:
    batch.get_record('r1')
    batch.get_record('r2')
    batch.get_record('r3')

r1, r2, r3 = batch.results()

Besides the results() method, a batch object shares all the same methods as another client.

Retry on error

When the server is throttled (under heavy load or maintenance) it can return error responses.

The client can hence retry to send the same request until it succeeds. To enable this, specify the number of retries on the client:

client = Client(server_url='http://localhost:8888/v1',
                auth=credentials,
                retry=10)

The Kinto protocol lets the server define the duration in seconds between retries. It is possible (but not recommended) to force this value in the clients:

client = Client(server_url='http://localhost:8888/v1',
                auth=credentials,
                retry=10,
                retry_after=5)

Pagination

When the server responses are paginated, the client will download every pages and merge them transparently.

However, it is possible to specify a limit for the number of items to be retrieved in one page:

records = client.get_records(_limit=10)

In order to retrieve every available pages with a limited number of items in each of them, you can specify the number of pages:

records = client.get_records(_limit=10, pages=float('inf'))  # Infinity

Generating endpoint paths

You may want to generate some endpoint paths, you can use the get_endpoint utility to do so:

client = Client(server_url='http://localhost:8888/v1',
                auth=('token', 'your-token'),
                bucket="payments",
                collection="receipts")
print(client.get_endpoint("record",
                          id="c6894b2c-1856-11e6-9415-3c970ede22b0"))

# '/buckets/payments/collections/receipts/records/c6894b2c-1856-11e6-9415-3c970ede22b0'

Handling datetime and date objects

In addition to the data types supported by JSON, kinto-http.py also supports native Python date and datetime objects.

In case a payload contain a date or a datetime object, kinto-http.py will encode it as an ISO formatted string.

Please note that this transformation is only one-way. While reading a record, if a string contains a ISO formated string, kinto-http.py will not convert it to a native Python date or datetime object.

If you know that a field will be a datetime, you might consider encoding it yourself to be more explicit about it being a string for Kinto.

Command-line scripts

In order to have common arguments and options for scripts, some utilities are provided to ease configuration and initialization of client from command-line arguments.

import argparse
import logging

from kinto_http import cli_utils

logger = logging.getLogger(__name__)

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Download records")
    cli_utils.set_parser_server_options(parser)

    args = parser.parse_args()

    cli_utils.setup_logger(logger, args)

    logger.debug("Instantiate Kinto client.")
    client = cli_utils.create_client_from_args(args)

    logger.info("Fetch records.")
    records = client.get_records()
    logger.warn("{} records.".format(len(records)))

The script now accepts basic options:

$ python example.py --help

usage: example.py [-h] [-s SERVER] [-a AUTH] [-b BUCKET] [-c COLLECTION] [-v]
                  [-q] [-D]

Download records

optional arguments:
  -h, --help            show this help message and exit
  -s SERVER, --server SERVER
                        The location of the remote server (with prefix)
  -a AUTH, --auth AUTH  BasicAuth token:my-secret
  -b BUCKET, --bucket BUCKET
                        Bucket name.
  -c COLLECTION, --collection COLLECTION
                        Collection name.
  --retry RETRY         Number of retries when a request fails
  --retry-after RETRY_AFTER
                        Delay in seconds between retries when requests fail
                        (default: provided by server)
  -v, --verbose         Show all messages.
  -q, --quiet           Show only critical errors.
  -D, --debug           Show all messages, including debug messages.

Run tests

In one terminal, run a Kinto server:

$ make runkinto

In another, run the tests against it:

$ make tests

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kinto-http-10.2.0.tar.gz (45.9 kB view details)

Uploaded Source

Built Distribution

kinto_http-10.2.0-py2-none-any.whl (47.1 kB view details)

Uploaded Python 2

File details

Details for the file kinto-http-10.2.0.tar.gz.

File metadata

  • Download URL: kinto-http-10.2.0.tar.gz
  • Upload date:
  • Size: 45.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.8.1 pkginfo/1.4.1 requests/2.18.4 setuptools/36.5.0 requests-toolbelt/0.7.0 clint/0.5.1 CPython/2.7.15+ Linux/4.18.0-12-generic

File hashes

Hashes for kinto-http-10.2.0.tar.gz
Algorithm Hash digest
SHA256 e8343772a4e16bbb832ec36600e98b16a3a6ee02bd5a0142897eae180be4fcb0
MD5 b29fac8bcb12eab337d5da644fb7eba3
BLAKE2b-256 7962845a85b1654cb46c6ea908356734545a25b2c013c981c9f2a79467f4b5b9

See more details on using hashes here.

File details

Details for the file kinto_http-10.2.0-py2-none-any.whl.

File metadata

  • Download URL: kinto_http-10.2.0-py2-none-any.whl
  • Upload date:
  • Size: 47.1 kB
  • Tags: Python 2
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.8.1 pkginfo/1.4.1 requests/2.18.4 setuptools/36.5.0 requests-toolbelt/0.7.0 clint/0.5.1 CPython/2.7.15+ Linux/4.18.0-12-generic

File hashes

Hashes for kinto_http-10.2.0-py2-none-any.whl
Algorithm Hash digest
SHA256 ccf086ec71e7a1b8b4216225c1e70821c208502b776982dcdd77b10f9d789c56
MD5 3ce726b1dd26a24ab3d63fb3981030e4
BLAKE2b-256 42ab2ef953609b097223e6298d560abc9b47eb450b5f8eabc2186e027cb64024

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page