decode-acc 0.1.0

Incrementally decode bytes into strings and lines

# An incremental decoder of bytes into characters and lines

## The DecodeAccumulator class

The `DecodeAccumulator` class implements an incremental decoder: an object
that may be fed bytes (one or several at a time) as they are e.g. read
from a network stream or a subprocess's output, and that adds to a result
string as soon as enough bytes have been accumulated to produce a character
in the specified encoding.

Note that `DecodeAccumulator` objects are immutable value objects:
the `add()` method does not modify its invocant, but returns a new
`DecodeAccumulator` object instead.

Sample usage:

while True:
bb = subprocess.stdout.read(1024)
if len(bb) == 0:
break
acc = acc.add(bb)
assert(not acc.done)
if acc.splitter.lines:
# at least one full line was produced
(acc, lines) = acc.pop_lines()
print('\n'.join(lines)

if acc.buf:
print('Leftover bytes left in the buffer!', file=sys.stderr)

if acc.splitter.buf:
print('Incomplete line: ' + acc.splitter.buf)

final = acc.add(None)
assert(final.splitter.buf == '')
assert(final.splitter.done)
assert(final.done)
if acc.splitter.buf:
assert(len(final.splitter.lines) == len(acc.splitter.lines) + 1)

## The splitter classes: UniversalNewlines, FixedEOLSplitter, NullSplitter

The `decode_acc.newlines` module provides three classes that may be used to
split a text string into lines in different ways. The `UniversalNewlines`
class does its best to simulate the "universal newlines" behavior of `file`
objects. The `FixedEOLSplitter` class uses a specified string as a line
terminator to split on. The `NullSplitter` class does not do any splitting.

Sample usage:

spl = newlines.UniversalNewlines()
for char in input_string:
spl = spl.add(char)
spl.add(None)

for (idx, line) in enumerate(spl.lines):
print('line {idx}: {line}'.format(idx=idx, line=line))