Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Facilities to do with buffers, particularly CornuCopyBuffer, an automatically refilling buffer to support parsing of data streams.

Project description

Facilities to do with buffers, particularly CornuCopyBuffer, an automatically refilling buffer to support parsing of data streams.

Function chunky(bfr_func)

Decorator for a function accepting a leading CornuCopyBuffer parameter. Returns a function accepting a leading data chunks parameter (bytes instances) and optional offset and 'copy_offsets` keywords parameters.

Example::

@chunky
def func(bfr, ...):

Class CopyingIterator

Wrapper for an iterator that copies every item retrieved to a callable.

Class CornuCopyBuffer

An automatically refilling buffer intended to support parsing of data streams.

Attributes:

  • buf: a buffer of unparsed data from the input, available for direct inspection by parsers
  • offset: the logical offset of the buffer; this excludes unconsumed input data and .buf

The primary methods supporting parsing of data streams are extend() and take(). Calling .extend(min_size) arranges that .buf contains at least min_size bytes. Calling .take(size) fetches exactly size bytes from .buf and the input source if necessary and returns them, adjusting .buf.

len(CornuCopyBuffer) returns the length of .buf.

bool(CornuCopyBuffer) tests whether len() > 0.

Indexing a CornuCopyBuffer accesses .buf.

A CornuCopyBuffer is also iterable, yielding data in whatever sizes come from its input_data source, preceeded by the current .buf if not empty.

A CornuCopyBuffer also supports the file methods .read, .tell and .seek supporting drop in use of the buffer in many file contexts. Backward seeks are not supported. .seek will take advantage of the input_data's .seek method if it has one, otherwise it will use reads.

Class FDIterator

MRO: _Iterator
An iterator over the data of a file descriptor.

Note: the iterator works with an os.dup() of the file descriptor so that it can close it with impunity; this requires the caller to close their descriptor.

Class FileIterator

MRO: _Iterator, SeekableIteratorMixin
An iterator over the data of a file object.

Note: the iterator closes the file on del or if its .close method is called.

Class SeekableFDIterator

MRO: FDIterator, _Iterator, SeekableIteratorMixin
An iterator over the data of a seekable file descriptor.

Note: the iterator works with an os.dup() of the file descriptor so that it can close it with impunity; this requires the caller to close their descriptor.

Class SeekableFileIterator

MRO: FileIterator, _Iterator, SeekableIteratorMixin
An iterator over the data of a seekable file object.

Note: the iterator closes the file on del or if its .close method is called.

Class SeekableIteratorMixin

Mixin supplying a logical with a seek method.

Class SeekableMMapIterator

MRO: _Iterator, SeekableIteratorMixin
An iterator over the data of a mappable file descriptor.

Note: the iterator works with an mmap of an os.dup() of the file descriptor so that it can close it with impunity; this requires the caller to close their descriptor.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for cs.buffer, version 20181231
Filename, size File type Python version Upload date Hashes
Filename, size cs.buffer-20181231.tar.gz (11.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page