fliq 1.0.0

Fluent-based Lazily-evaluated Integrated Query for Python

Fliq

Fluent-syntaxed Lazily-evaluated Integrated Query.

Fliq is a lightweight Python library for high-performance processing of iterables, inspired by Django's ORM and LINQ. It provides a fluent syntax for lazily-evaluated operations on iterables, and it is tested to have on-par performance with the standard library.

• Documentation: https://oribarilan.github.io/fliq
• Source Code: https://github.com/oribarilan/fliq

Installation

pip install fliq

Fliq is

• Intuitive to use. Built for readability and usability.
• Lightweight wrapper for the standard library. No dependencies or bloat.
• Efficient as the standard library. Abstraction overhead is kept to a minimum.
• Lazy operations, evaluated only when needed and only as needed.
• Compatible with APIs consuming iterables.

Motivation

What is the output of the following code?

next(map(lambda x: x * 2, filter(lambda x: x % 2, [1, 2, 3, 4, 5]))) or -1

And what about this?

from fliq import q

(q([1, 2, 3, 4, 5])
    .where(lambda x: x % 2 == 0)
    .select(lambda x: x * 2)
    .first_or_default(-1))

And this is just a simple example.

Python's standard library provides a rich set of functions for processing iterables. However, it is not always easy to read and use.

This is especially true when chaining multiple operations together. This is where Fliq comes in. Fliq provides a fluent, easy to read syntax for processing iterables, while keeping performance on-par with the standard library.

Efficiency

Memory

Fliq is lazy. It does not materialize the iterable, unless the operation requires it (e.g., reverse). This allows Fliq to make minimal use of memory.

Abstracting the (lazy) evaluation, reduces cognitive load from the user, thus implicitly opting in for a more efficient computation, which in turn improves efficiency compared to eager list processing code.

Performance

Fliq is designed to be a lightweight wrapper for the standard library. It keeps abstraction overhead to a minimum, and it is tested to have on-par performance with the standard library.

There are two mechanisms for checking Fliq's performance: Performance tests and benchmarking.

Performance Tests

These tests are ran on every commit, and they compare Fliq's performance to the standard library. They allow performance difference to be 1% or smaller, and they are ran on every Python version, 3.9 and above.

You can find the performance tests in Performance tests.

Benchmarking

Fliq is being benchmarked against the standard library. The list of benchmarked scenarios expands over time.

Currently, there are 2 scenarios tested with varying dataset sizes (100, 10K, 1M).

• Scenario 1: zipping two iterables of Person objects, and taking the first 5 (by age asc) that are of different gender.

q(dataset).zip(shuffled).where(lambda ps: ps[0].gender != ps[1].gender).order(by=lambda ps: ps[0].age+ps[1].age).take(5).to_list()

• Scenario 2: filtering prepending and appending a list of Person objects

q(dataset).where(lambda p: 0 <= p.age < 100).prepend_many(first).append_many(last).select(lambda p: p.name).to_list()

In both scenarios, Fliq is on-par with the standard library:

You can find the full benchmarking code at Benchmark.