This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

What is lifter?

Lifter is a generic query engine, inspired by Django ORM and SQLAlchemy.

Its goal is to provide a unique interface to query any type of data, regardless of the underlying query language or data backend (SQL, REST, Python objects…).

Warning: This package is still in alpha state and a lot of work is still needed to make queries faster and efficient. Contributions are welcome :)

Useful links:

Features

  • Queryset API similar to Django and SQLAlchemy
  • Lazy querysets
  • Composable queries
  • Lightweight: very little dependencies
  • Tested and working on Python 2.7 to Python 3.5

Example usage

Consider the following list of users, returned from a REST API endpoint:

users = [
    {
        "is_active": True,
        "age": 35,
        "eye_color": "brown",
        "name": "Bernard",
        "gender": "male",
        "email": "bernard@blackbooks.com",
    },
    {
        "is_active": True,
        "age": 34,
        "eye_color": "brown",
        "name": "Manny",
        "gender": "male",
        "email": "manny@blackbooks.com",
    },
    {
        "is_active": True,
        "age": 35,
        "eye_color": "brown",
        "name": "Fran",
        "gender": "female",
        "email": "fran@blackbooks.com",
    },
    # And so on ...
]

Now, imagine you have to extract data from this list. Let’s compare how you can do this using regular Python and lifter.

To use lifter in your project, you’ll only need the following instructions:

import lifter.models
from lifter.backends.python import IterableStore

class User(lifter.models.Model):
    pass

store = IterableStore(users)
manager = store.query(User)

Getting all active 26 years old users:

# vanilla Python
results = [
    user for user in users
    if user['age'] == 26 and user['is_active']
]

# lifter
results = manager.filter(User.age == 26, User.is_active == True)

Getting names and emails of inactive users under 56:

# vanilla Python
results = [
    (user['name'], user['email']) for user in users
    if not user['is_active'] and user['age'] < 56
]

# lifter
results = manager.filter(User.is_active == False, User.age < 56)\
                 .values_list('name', 'email')

Getting all active users except the one with brown eyes and sort them by age:

# vanilla Python
raw_results = [
    user for user in users
    if user['is_active'] and not user['eye_color'] == 'brown'
]
results = sorted(raw_results, key=lambda v: v['age'])

# lifter
results = manager.filter(User.is_active == True)\
                 .exclude(User.eye_color == 'brown')\
                 .order_by('age')

Getting minimum and average women age:

# vanilla Python
from statistics import mean # Only in Python >=3.4
women_ages = [
    user['age'] for user in users
    if user['gender'] == 'female'
]
women_average_age = mean(women_ages)
minimum_woman_age = min(women_ages)

# lifter
results = manager.filter(User.gender == 'female')\
                 .aggregate((User.age, mean), (User.age, min))

As you can see, lifter’s version is shorter and more readable than vanilla Python. It’s also less error prone, especially if you’re writing really complex queries, and quite familiar if you’ve already used an ORM.

Wanna know more? Have a look at the documentation!

History

0.4.1 (2016-8-2)

This release fix issue #42: some files were not included in lifter distribution, mainly the backends and contrib directories, causing various imports to fail.

0.4 (2016-7-20)

This release introduces the django contrib module to enable filtering with lifter’s python backend on a django queryset, effectively reducing number of queries sent to the database.

Work is also in progress regarding caching (see #39) but this is not over yet.

0.3 (2016-7-12)

This is a big release, that breaks backward-compatibility with previous ones.

This release implements a new flow to help implementing #33. The general idea is to make lifter generic and be able to query any data source with it.

The 0.3 release sets the foundation for that by moving all python-iterable related code to a dedicated backend, and by implementing the Store -> Adapter > Model layout to deal with queries and result parsing.

An additional, very simple, filesystem backend is provided to demonstrate how you can implement your own datasource in lifter.

The work, though, is still incomplete, because the filesystem store internally uses the IterableStore from the python backend.

A real store (such as REST or SQL) would be able to understand queries and pass them to a real backend (PostgreSQL).

Anyway, we’re in the good direction here :)

0.2.1 (2016-3-4)

This is a small release, with a few improvements on ordering API and on the overall documentation:

  • Can now order using multiple fields, fix #30
  • [Backward incompatible] Can now invert ordering in explicit engine using path and ~ operator. Passing a reverse argument to order_by is not possible anymore
  • Can now query for field existences, fix #26

0.2 (2016-2-23)

This is quite an important release:

  • A whole new API is now available to make queries, see #15 for more information [angru, eliotberriot]
  • Querysets are now lazy
  • A brand new documentation is now available at http://lifter.readthedocs.org
  • Splitted some huge files into submodules for more clarity

Considering the new query API, we basically switched from django’s ORM-style (keyword engine) to SQLAlchemy style (explicit engine).

The keyword engine is still available and will continue to work as before. It is neither under depreciation at the moment, but using the new engine is recommended.

0.1.1 (2016-2-21)

  • Can now pass arguments to underlying manager via lifter.load
  • Random order_by for queryset [Pentusha]
  • Improve code examples readability in readme
  • Removed duplicate method on queryset [Mec-iS]
  • Can now run some lookups within iterables (WIP) [Ogreman].
  • Lots of improvements and corrections (typos, examples, etc.) in README [johnfraney, youtux]
  • Can now return flat lists as results for aggregates [Ogreman]

0.1.0 (2016-2-17)

  • First release on PyPI.
Release History

Release History

0.4.1

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
lifter-0.4.1.tar.gz (62.1 kB) Copy SHA256 Checksum SHA256 Source Aug 2, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting