More Streams!!
Python code is more elegant with method chaining!
Overview
There are two families of "streams" in this library, both are lazy:
ByteStream- a traditional stream of bytes intended to pipe bytes through various byte transformers, like compression, encoding and encyrption.ObjectStream: An iterator/generator with a number of useful methods.
Example
In this case I am iterating through all files in a tar and parsing them:
results = (
File("tests/so_queries/so_queries.tar.zst")
.content()
.content()
.exists()
.utf8()
.to_str()
.map(parse)
.to_list()
)
Each of the steps constructs a generator, and no work is done until the last step
File().content()- will unzst and untar the file content to anObjectStreamof file-like objects. It is short form forstream(File().read_bytes()).from_zst().from_tar()- The second
.content()is applied to each of the file-like objects, returningByteStreamof the content for each .exists()- some of the files (aka directories) in the tar file do not have content, we only include content that exists..utf8- convert to aStringStream.to_str- convert to a Pythonstr, we trust the content is not too large.map(parse)- run the parser on each string.to_list()- a "terminator", which executes the chain and returns a Pythonlistwith the results
Project Status
Alive and in use, but
- basic functions missing
- inefficient - written using generators
- generators not properly closed
Optional Reading
The method chaining style has two distinct benefits
- functions are in the order they are applied
- intermediate values need no temporary variables
The detriments are the same that we find in any declarative language: Incorrect code can be difficult to debug because you can not step through it to isolate the problem. For this reason, the majority of the code in this library is dedicated to validating the links in the function chain before they are run.
Lessons
The function chaining style, called "streams" in Java or "linq" in C#, leans heavly on the strict typed nature of those langauges. This is missing in Python, but type annotations help support this style of programming.
1 maintainer
Release files for mo-streams 1.484.24013
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| mo_streams-1.484.24013-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details |
Release files / mo_streams-1.484.24013-py2.py3-none-any.whl
| Download URL | mo_streams-1.484.24013-py2.py3-none-any.whl |
|---|---|
| Size | 24.6 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
c6aa244ec1e566f7c53f415358721d4bfcff33874130fa5f1fc7c025bfc9c369
|
|
BLAKE2b-256 checksum How to use checksums |
62eb95e3555b8f8e4573d79fe9b2bf7164ff45497caca7cf5f35edc80e2bf1ad
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.9.6
|
