Skip to main content

More Streams! Chained function calls

Project description

More Streams!!

PyPI Latest Release Build Status Coverage Status Downloads

Python code is more elegant with method chaining!

Overview

There are two families of "streams" in this library, both are lazy:

  1. ByteStream - a traditional stream of bytes intended to pipe bytes through various byte transformers, like compression, encoding and encyrption.
  2. ObjectStream: An iterator/generator with a number of useful methods.

Example

In this case I am iterating through all files in a tar and parsing them:

results = (
    File("tests/so_queries/so_queries.tar.zst")
    .content()
    .content()
    .exists()
    .utf8()
    .to_str()
    .map(parse)
    .to_list()
)

Each of the steps constructs a generator, and no work is done until the last step

  • File().content() - will unzst and untar the file content to an ObjectStream of file-like objects. It is short form for stream(File().read_bytes()).from_zst().from_tar()
  • The second .content() is applied to each of the file-like objects, returning ByteStream of the content for each
  • .exists() - some of the files (aka directories) in the tar file do not have content, we only include content that exists.
  • .utf8 - convert to a StringStream
  • .to_str - convert to a Python str, we trust the content is not too large
  • .map(parse) - run the parser on each string
  • .to_list() - a "terminator", which executes the chain and returns a Python list with the results

Project Status

Alive and in use, but

  • basic functions missing
  • inefficient - written using generators
  • generators not properly closed

Optional Reading

The method chaining style has two distinct benefits

  • functions are in the order they are applied
  • intermediate values need no temporary variables

The detriments are the same that we find in any declarative language: Incorrect code can be difficult to debug because you can not step through it to isolate the problem. For this reason, the majority of the code in this library is dedicated to validating the links in the function chain before they are run.

Lessons

The function chaining style, called "streams" in Java or "linq" in C#, leans heavly on the strict typed nature of those langauges. This is missing in Python, but type annotations help support this style of programming.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mo-streams-1.542.24038.tar.gz (24.9 kB view details)

Uploaded Source

Built Distribution

mo_streams-1.542.24038-py3-none-any.whl (24.6 kB view details)

Uploaded Python 3

File details

Details for the file mo-streams-1.542.24038.tar.gz.

File metadata

  • Download URL: mo-streams-1.542.24038.tar.gz
  • Upload date:
  • Size: 24.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.6

File hashes

Hashes for mo-streams-1.542.24038.tar.gz
Algorithm Hash digest
SHA256 3f48ddd8228414765bbeb5325d4359146abd34651d8cbbd1cb356bde2705f4a7
MD5 b1752e3198240b7318bfee249058cc1b
BLAKE2b-256 e5d843e4e37774c07ef5d0f938b018c0e04a1b7b48f9ef8d407ca583da4f6293

See more details on using hashes here.

File details

Details for the file mo_streams-1.542.24038-py3-none-any.whl.

File metadata

File hashes

Hashes for mo_streams-1.542.24038-py3-none-any.whl
Algorithm Hash digest
SHA256 94420c92a5df84556de20b016a279e93ef49ed2abd4839a2651e3a7828f6dc98
MD5 4a27e11ce100c48e1ff39225674468a2
BLAKE2b-256 ba7448edecd1e43323d7fdf3e9bfbca6a2d41b8d1b5b4bb0ef611f8c9e529908

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page