cs-buffer 20180805

Facilities to do with buffers, primarily CornuCopyBuffer, an automatically refilling buffer to support parsing of data streams.

Function chunky(bfr_func)

Decorator for a function accepting a leading CornuCopyBuffer parameter. Returns a function accepting a leading data chunks parameter (bytes instances) and optional offset and 'copy_offsets` keywords parameters.

Example::

@chunky def func(bfr, ...):

Class CopyingIterator

Wrapper for an iterator that copies every item retrieved to a callable.

Class CornuCopyBuffer

An automatically refilling buffer intended to support parsing of data streams.

Attributes:

• buf: a buffer of unparsed data from the input, available for direct inspection by parsers
• offset: the logical offset of the buffer; this excludes unconsumed input data and .buf

The primary methods supporting parsing of data streams are extend() and take(). Calling .extend(min_size) arranges that .buf contains at least min_size bytes. Calling .take(size) fetches exactly size bytes from .buf and the input source if necessary and returns them, adjusting .buf.

len(CornuCopyBuffer) returns the length of .buf.

bool(CornuCopyBuffer) tests whether len() > 0.

Indexing a CornuCopyBuffer accesses .buf.

A CornuCopyBuffer is also iterable, yielding data in whatever sizes come from its input_data source, preceeded by the current .buf if not empty.

A CornuCopyBuffer also supports the file methods .read, .tell and .seek supporting drop in use of the buffer in many file contexts. Backward seeks are not supported. .seek will take advantage of the input_data's .seek method if it has one, otherwise it will use reads.

Class SeekableFDIterator

An iterator over the data of a file descriptor.

Class SeekableFileIterator

An iterator over the data of a file object.