Skip to main content

Build Status PyPI Release PyPI Status Python License

pyStream

Python implementation of stream library. Multiple protobuf messages can be written/read into/from a stream by using this library. It enables stream processing of protobuf messages and can be used for parsing all files encoded by stream library and writing protobuf instances into a file by the same encoding. Refer to the library github page for more information about formatting.

Installation

You can install pyStream using pip:

pip install pystream-protobuf

Examples

Here’s a sample code using the Stream class to read a file (so-called GAM file) containing a set of VG’s Alignment objects (defined here). It yields the protobuf objects stored in the file:

import stream
import vg_pb2

alns = [a for a in stream.parse('test.gam', vg_pb2.Alignment)]

Or use lower-level method open in order to have more control in opening the stream and reading data:

import stream
import vg_pb2

alns_list = []
with stream.open('test.gam', 'rb') as istream:
    for data in istream:
        aln = vg_pb2.Alignment()
        aln.ParseFromString(data)
        alns_list.append(aln)

This example also does the same without using with statement by calling close method explicitly after reading the data:

import stream
import vg_pb2

alns_list = []
istream = stream.open('test.gam', 'rb')
for data in istream:
    aln = vg_pb2.Alignment()
    aln.ParseFromString(data)
    alns_list.append(aln)
istream.close()

And here is another sample code for writing multiple protobuf objects into a file (here a GAM file):

import stream

stream.dump('test.gam', *objects_list, buffer_size=10)

Or using open method for lower-level control – here, appending for example:

import stream

with stream.open('test.gam', 'ab') as ostream:
    ostream.write(*objects_list)
    ostream.write(*another_objects_list)

Or without using with statement where the data is written after calling close method explicitly:

import stream

ostream = stream.open('test.gam', 'wb')
ostream.write(*objects_list)
ostream.write(*another_objects_list)
ostream.close()

More features

Buffered write

By default, all protobuf message objects provided on each call are written in a group of messages. The messages can be buffered and write to the stream in a group of fixed size whenever possible. The size of the buffer can be changed (from default value 0 — no buffer) by passing it through keyword argumnet buffer_size when Stream class is constructed or a stream is opened. This value is the number of objects which should be written in a group.

Grouping message

Messages can be grouped in varied size when writing to a stream by setting buffer size sufficiently large or infinity (-1) and calling flush method of Stream class whenever desired.

Group delimiter

Group of objects can be separated by a delimiter of the choice (or by default None) when reading from a stream. Sometimes, it can help to identify the end of a group which is hidden from the library user by default. This feature can be enable by setting group_delimiter True when constructing a Stream instance or openning a stream. The delimiter class can also be specified by delimiter_cls.

Metadata

Release files for pystream-protobuf 1.4.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pystream-protobuf 1.4.3
File Size Uploaded Provenance Actions
pystream-protobuf-1.4.3.tar.gz 8.4 kB Without provenance Details for pystream-protobuf-1.4.3.tar.gz

Release files / pystream-protobuf-1.4.3.tar.gz

Download URL pystream-protobuf-1.4.3.tar.gz
Size 8.4 kB
Tags Source
SHA-256 checksum
How to use checksums
883f7b4c33af67f41df4b4d43533d34b97fb5a7e64b4b48a3e109a01e8c57a14
BLAKE2b-256 checksum
How to use checksums
f80450fd6d30a926dc5cb192e81e8e89ffb2bc92eda1ad1076a2afae14062f1b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for pystream-protobuf 1.4.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page