Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

FlpIt

🌐 View on GitHub | 📦 View on PyPI

Fluent LINQ for Python — LINQ semantics, Python-native execution

FlpIt brings a strongly typed, LINQ-style API to standard Python iterables. It combines the fluent query model and semantics of .NET LINQ with Python's native iterable and generator model.

Semantic goal: flp strives to faithfully reproduce .NET LINQ semantics.
Where Python's language or type system requires a different API contract, the contract is adapted accordingly. The implementation is actively developed, and semantic deviations may still exist.

Familiar LINQ, Now in Python

FlpIt is a Python implementation of the popular .NET Language Integrated Query (LINQ), adapted to Python.

If you know C# LINQ, you already know most of FlpIt. Existing LINQ examples, query patterns, and documentation can generally be translated directly. FlpIt aims to make its implemented LINQ operators behave like their corresponding .NET System.Linq.Enumerable counterparts.

This makes the extensive .NET LINQ ecosystem a valuable reference for developers and AI coding assistants alike.

⚡ Key Features

  • ⚡ Lazy Evaluation (FlpIt): Deferred, streaming execution wrapping any standard Python Iterable[T].
  • 📦 Materialized Container (FlpList): Eager collection implementing Python's Collection protocol (with indexing and slicing), retaining familiar collection access while providing .NET List<T>-style semantics and fluent LINQ query operations.
  • 🎯 LINQ Semantics: Familiar operators like .any(), .first(), .single(), .order_by(), and .then_by() with predictable, composable behavior.
  • 🔀 Independent Query Pipelines: Derived queries are independent and never mutate or retroactively affect separately created queries.
  • 🧠 Lazy Internal Materialization: Some operators, such as ordering and grouping, require internal materialization during deferred execution, while the query itself remains lazy.
  • 🔄 Source Enumeration Semantics — FlpIt preserves the enumeration characteristics of its source. A query over a re-enumerable source can be enumerated repeatedly; a one-shot source such as a Python generator remains one-shot. FlpIt does not clone or rewind arbitrary Python iterators.
  • 🔹 Static-First Typing: Designed for precise pyrefly inference without plugins (mypy and pyright might give varying results).
  • 🐍 Python-Native: Built on standard Python iterables, iterators, generators, and callable semantics.

Scope & Intent

FlpIt focuses exclusively on replicating the IEnumerable (System.Linq.Enumerable) model.

  • Direct Execution: It operates directly on standard Python callables (lambdas and functions) over Python iterables.
  • No Provider Abstraction: It does not model or emit Expression Trees (IQueryable) for translation by external providers. There is no query-provider layer that rewrites FlpIt operations into database queries or other backend-specific operations.

FlpIt therefore follows the execution model of in-memory LINQ: query operators build a pipeline, execution is deferred where the operator permits it, and each operator's actual enumeration and buffering behavior is part of the query semantics.

🤸 Flipping the Pipeline

Native Python functional operations tend to produce an inside-out, right-to-left reading pattern...

# 🔁 Native Built-ins
result = list(map(lambda x: x * 10, filter(lambda x: x % 2 == 0, range(1, 11))))

# 🤸 FlpIt Pipeline
result = (
    flp.it(range(1, 11))
    .where(lambda x: x % 2 == 0)
    .select(lambda x: x * 10)
    .to_list()
)

💡 Quick Start

from flpit import flp, FlpIt, FlpList

# Deferred iterable via shorthand
data: FlpIt[int] = flp.it(range(1, 11))

# Lazy, streaming pipeline
query: FlpIt[int] = (
    data
    .where(lambda x: x % 2 == 0)
    .select(lambda x: x * 10)
)

# Explicit materialization
result: FlpList[int] = query.to_list()
# [20, 40, 60, 80, 100]

# Or start directly with an eager container
eager_list: FlpList[int] = flp.lst([1, 2, 3, 4])

Typed Domain Objects

from dataclasses import dataclass
from flpit import flp


@dataclass
class Employee:
    name: str
    role: str
    salary: int


@dataclass
class Department:
    name: str
    employees: list[Employee]


company = [
    Department(
        "Engineering",
        [
            Employee("Alice", "Backend Engineer", 95_000),
            Employee("Bob", "Frontend Engineer", 90_000),
        ],
    ),
    Department(
        "Design",
        [
            Employee("Charlie", "UI Designer", 82_000),
        ],
    ),
]

# Extract the names of backend engineers earning at least 90,000.
backend_engineers = (
    flp.it(company)
    .select_many(lambda dept: dept.employees)
    .where(lambda emp: emp.role == "Backend Engineer")
    .where(lambda emp: emp.salary >= 90_000)
    .select(lambda emp: emp.name)
)

# The query is evaluated when it is enumerated.
print(list(backend_engineers))
# ['Alice']

JSON Serialization

FlpIt provides typed JSON serialization and deserialization through flpit.json.JsonConvert, backed by msgspec.

It supports dataclasses, nested models, generic containers, configurable property naming, and explicit field metadata.

from dataclasses import dataclass
from typing import Annotated

from flpit import FlpList
from flpit.json import JProp, JIgnore, JsonConvert

@dataclass
class Person:
    first_name: str
    internal_id: Annotated[str, JProp("id")]
    password: Annotated[str, JIgnore] = ""


person = Person("Ada", "usr-42", "secret")
raw = JsonConvert.serialize(person, Person)
# {"firstName":"Ada","id":"usr-42"}

people_json = """
[
    {"firstName": "Ada", "id": "usr-42"},
    {"firstName": "Grace", "id": "usr-43"}
]
"""

people: FlpList[Person] = FlpList.from_json(people_json, Person)

Mapping and conversion

  • Naming policies: Configure automatic property naming with JsonSerializerOptions. The default is camel case.
  • Explicit metadata: JProp assigns a specific JSON property name, JIgnore excludes a field, and JInclude explicitly includes otherwise excluded underscore-prefixed fields.
  • Serialization options: Control default-value omission and whether unknown JSON fields are accepted or rejected.
  • Typed collections: Pass the complete target type, such as list[Person], so nested and generic values can be decoded against the intended schema.

For large outputs, JsonConvert.serialize_to() writes a JSON array to a file-like stream without first materializing the entire serialized document.

By default, deserialization returns instances of the requested model type. Setting native=True opts into the generated msgspec.Struct representation instead, trading the original model type for the native representation and its performance characteristics.

🔬 LINQ Semantics, Python Runtime

flpit deliberately separates the two:

  • LINQ defines the query semantics.
  • Python provides the underlying iterable and execution model.

For example:

query = flp.range(1, 11).where(lambda x: x % 2 == 0)

query.any()       # LINQ-style terminal operation
query.first()     # LINQ-style terminal operation
query.single()    # LINQ-style terminal operation

The query can also be consumed through normal Python built-ins:

any(query)
list(query)

The two forms serve different purposes: LINQ-style operators provide the query API and its associated semantics, while Python built-ins continue to operate through the normal iterable protocol.

Source Repeatability

FlpIt does not imply that every source is repeatable. Re-enumeration depends on the underlying source:

source = [1, 2, 3]
query = flp.it(source)

list(query)  # [1, 2, 3]
list(query)  # [1, 2, 3]

A one-shot iterator remains one-shot:

source = (x for x in range(3))
query = flp.it(source)

list(query)  # [0, 1, 2]
list(query)  # []

Intentional Semantic Deviation: count()

FlpList is a LINQ-oriented collection. Its query methods follow LINQ semantics even when those semantics differ from Python's built-in collection methods.

In Python, list.count(value) counts occurrences of a specific value:

[1, 2, 2, 3].count(2)  # 2

In LINQ, Count() returns the total number of elements, or elements matching a predicate:

FlpList([1, 2, 2, 3]).count()                 # 4
FlpList([1, 2, 2, 3]).count(lambda x: x > 1)  # 3

FlpList.count() deliberately follows the LINQ meaning. Python-style value counting remains available through normal Python operations when needed:

list(FlpList([1, 2, 2, 3])).count(2)  # 2

This behavior is a core part of FlpList's LINQ-oriented contract, not an incomplete list implementation.

🔗 Independent Query Pipelines

Query pipelines can be composed independently without mutating the queries they were derived from:

For example:

@dataclass
class Item: 
    group: str 
    value: int

original = flp.it([Item("A", 2), Item("B", 1)])

ordered = original.order_by(lambda x: x.group)

extended = original.concat([Item("A", 1), Item("B", 2)])

ordered_by_group_and_value = ordered.then_by(lambda x: x.value)

The resulting query graph is:

original
   │
   ├── order_by ──→ ordered
   │                  │
   │                  └── then_by ──→ ordered_by_group_and_value
   │
   └── concat ─────→ extended

ordered_by_group_and_value is derived from ordered.
It does not modify original, and it is not affected by the separately created extended query.

This matches the independent-query behavior of .NET LINQ.

Joins

customers = [("Prakash", 1), ("Tim", 2), ("Robert", 3)]
orders    = [(10, 1), (11, 1), (12, 4)]          # (order_id, customer_id)

flp.it(customers).join(orders, lambda c: c[1], lambda o: o[1], lambda c, o: (c[0], o[0]))
# [("Prakash", 10), ("Prakash", 11)]                          inner join   (.NET Join)

flp.it(customers).left_join(orders, lambda c: c[1], lambda o: o[1], lambda c, o: (c[0], o and o[0]))
# [("Prakash", 10), ("Prakash", 11), ("Tim", None), ("Robert", None)]   (.NET 10 LeftJoin)

flp.it(customers).full_join(orders, lambda c: c[1], lambda o: o[1], lambda c, o: (c and c[0], o and o[0]))
# [("Prakash", 10), ("Prakash", 11), ("Tim", None), ("Robert", None), (None, 12)]

Without a result selector all three yield JoinTuple(outer, inner).

  • Elements whose key is None never match. left_join() / full_join() still emit them as unmatched.
  • An unmatched side is passed to the result selector as None (the Python equivalent of .NET default).
  • full_join() has no .NET LINQ equivalent. It yields the left_join() rows in outer order, followed by every inner element that was never matched (in inner order).
  • Argument validation is eager (InnerNoneError, OuterKeySelectorNoneError, InnerKeySelectorNoneError, ResultSelectorNoneError); everything else is deferred.
  • Custom equality comparers are not supported yet (keys are compared with == / hash, like group_by()).

Execution Characteristics

Lazy execution does not mean that every operator is streaming. Some operators must consume and buffer their input before they can produce their result. Sorting requires buffering: Any exact sort of an arbitrary input must inspect its elements before it can produce the ordered result. This is not specific to FlpIt; it is an inherent characteristic of sorting and applies equally to native Python sorting and .NET LINQ OrderBy().

Operator Execution deferred Processes incrementally Buffers input
where() / select() Yes Yes No
order_by() Yes No Yes
group_by() Yes No Yes
join() Yes Yes (outer) Yes (inner)
left_join() Yes Yes (outer) Yes (inner)
full_join() Yes Yes (outer) Yes (inner)

order_by() and group_by() therefore remain deferred at query construction time, but enumeration of either operator requires buffering its upstream input.

join(), left_join() and full_join() build a hash lookup of the inner sequence when enumeration starts and stream the outer sequence (O(N + M)).

group_by() produces a query of groupings rather than terminating the pipeline:

from typing import NamedTuple

class GroupResult(NamedTuple):
    remainder: int
    values: list[int]

result = (
    flp.it(range(1, 11))
    .group_by(lambda x: x % 2)
    .select(lambda group: GroupResult(
        remainder=group.key,
        values=list(group),
    ))
    .to_list()
)

Terminal operators instead evaluate the query and return a result rather than another query:

query.any()
query.first()
query.single()
query.count()
query.aggregate(...)
query.to_list()

For example, aggregate() processes its source incrementally but must consume the source completely to produce its final accumulated value.

🧠 Lazy Pipelines

Streaming operators do not materialize their upstream input:

query = (
    flp.it(source)
    .where(predicate)
    .select(selector)
    .order_by(key_selector)
    .take(100)
)

Conceptually:

source
  │
  ▼
where       ← streaming
  │
  ▼
select      ← streaming
  │
  ▼
order_by    ← lazy, internally buffers when enumerated
  │
  ▼
take        ← streaming
  │
  ▼
to_list     ← terminal materialization

Operations such as order_by() necessarily materialize their input internally to establish ordering.

⚙️ Task Runtime

FlpIt includes a lightweight, asyncio-free task runtime for scheduling and coordinating concurrent work, with synchronous callables executed through a bounded worker pool.

Key features:

  • Task scheduling: Task.run() schedules coroutines, awaitables, and synchronous callables.
  • Task coordination: Task.when_all() waits for all tasks while preserving input order; Task.when_any() returns the first completed task; Task.collect() yields tasks as they complete.
  • Cancellation: CancellationToken and CancellationTokenSource support cooperative cancellation, cancellation callbacks, timeouts, and linked cancellation tokens.
  • Resource management: Configure worker capacity and pending background-work limits, handle queue saturation explicitly, and shut down the process-wide runtime with an observable shutdown report.
  • Asyncio interoperability: FlpIt Tasks can be awaited directly inside asyncio coroutines when Task.asyncio_interop is enabled.
    • This is disabled by default.
    • Enabling it adds runtime-detection overhead to every pending await.
      • await in flpit runtime: 200ish nano sec (just loop detection)
      • await in asyncio loop:

The runtime starts lazily and manages its own lifecycle. Cancellation of running synchronous functions is cooperative.

See the Task Runtime examples for usage examples.

⚙️ Installation

uv add flpit

📜 License

Distributed under the MIT License. See LICENSE for more information.

Migrated .NET Unit Tests

Migrated from the official .NET Runtime LINQ Tests. Note: The test cases exercise both core types, FlpIt and FlpList, with transitions between lazy and eager execution occurring as part of the individual test cases.

================================================================================
| Test Module File                                               |  Test Cases |
--------------------------------------------------------------------------------
| tests/nettests/                                                |             |
| ├── linq/                                                      |             |
| │   ├── test_aggregate.py                                      |          42 |
| │   ├── test_all.py                                            |          80 |
| │   ├── test_any.py                                            |          90 |
| │   ├── test_average.py                                        |         572 |
| │   ├── test_chunk.py                                          |          22 |
| │   ├── test_concat.py                                         |          64 |
| │   ├── test_contains.py                                       |         108 |
| │   ├── test_count.py                                          |          37 |
| │   ├── test_distinct.py                                       |          87 |
| │   ├── test_element_at.py                                     |          43 |
| │   ├── test_first.py                                          |          29 |
| │   ├── test_first_or_default.py                               |          50 |
| │   ├── test_full_join.py                                      |          37 |
| │   ├── test_generic_list.py                                   |          95 |
| │   ├── test_group_by.py                                       |          23 |
| │   ├── test_join.py                                           |          47 |
| │   ├── test_last.py                                           |          29 |
| │   ├── test_last_or_default.py                                |          45 |
| │   ├── test_left_join.py                                      |          47 |
| │   ├── test_max.py                                            |        1100 |
| │   ├── test_min.py                                            |         968 |
| │   ├── test_order_by.py                                       |         100 |
| │   ├── test_order_descending_by.py                            |          45 |
| │   ├── test_reverse.py                                        |          35 |
| │   ├── test_select.py                                         |         140 |
| │   ├── test_select_many.py                                    |          85 |
| │   ├── test_single.py                                         |          34 |
| │   ├── test_single_or_default.py                              |          48 |
| │   ├── test_skip.py                                           |          75 |
| │   ├── test_sum.py                                            |         134 |
| │   ├── test_take.py                                           |          39 |
| │   ├── test_then_by.py                                        |          40 |
| │   ├── test_then_descending_by.py                             |          34 |
| │   ├── test_where.py                                          |          76 |
| │   └── test_zip.py                                            |         101 |
| └── task/                                                      |             |
|     ├── test_cancellation.py                                   |          69 |
|     ├── test_task_completion_source.py                         |          26 |
|     ├── test_task_factory.py                                   |           3 |
|     ├── test_task_rt.py                                        |          12 |
|     └── test_task_rt_core.py                                   |          22 |
--------------------------------------------------------------------------------
| GRAND TOTALS                                                   |        4733 |
================================================================================

Metadata

Release files for flpit 0.3.1.dev6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for flpit 0.3.1.dev6
File Size Uploaded Provenance Actions
flpit-0.3.1.dev6.tar.gz 46.5 kB Without provenance Details for flpit-0.3.1.dev6.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for flpit 0.3.1.dev6
File Interpreter ABI Platform Actions
flpit-0.3.1.dev6-py3-none-any.whl Python 3 none any Details for flpit-0.3.1.dev6-py3-none-any.whl

Total release size: 96.8 kB

Release files / flpit-0.3.1.dev6.tar.gz

Download URL flpit-0.3.1.dev6.tar.gz
Size 46.5 kB
Tags Source
SHA-256 checksum
How to use checksums
79b691f7e83ddd13daeceebb8403217d0feaabd2b12b94520e536d3d6e8f04c9
BLAKE2b-256 checksum
How to use checksums
e2f0c305e9d01e7e8aa35ea85fb2e02e88a34e243f9e02b91f2709130bd7fe21
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / flpit-0.3.1.dev6-py3-none-any.whl

Download URL flpit-0.3.1.dev6-py3-none-any.whl
Size 50.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8f9c100505183f96833c67f3cc71d8a6bcd1e6941e0a26866ead24e9a4ed93b5
BLAKE2b-256 checksum
How to use checksums
e4c3d02596a102a4879172b09ca01b48281ca9f3d7e1653994f35ac34361e055
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for flpit 0.3.1.dev6

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page