stypes 0.1.44

Encode and Decode Textual Data into Rich Python Data Structures

Encode and Decode Textual Data into Rich Python Data Structures
===============================================================

stypes is a text parsing, conversion and formatting library written to
efficiently handle large fixed-width text record data files. Convert text
streams into dictionaries, lists, tuples, named tuples, ordered dictionaries
and more using text layout specifications. Nested data structures and
repeating elements are also supported.

One of the main design goals of the library was to handle legacy text-based
record data that is commonly found in COBOL system. Of course, the library
can be used to handle any fixed layout textual data.

A simple example of turning some text into a Named Tuple:

::
from decimal import Decimal
from stypes import NamedTuple, Integer, Numeric
spec = NamedTuple([
('name', 10),
('age', Integer(3)),
('weight', Numeric('999V99'))])
text = "Johnson 2109750"
rec = spec.unpack(text)
assert rec.name == 'Johnson'
assert rec.age == 21
assert rec.weight == Decimal("97.5")



And a more interesting example using nested data structures of a list of
records and actually updating a record.

::
from stypes import Array, Dict, Integer, Numeric
item = Dict([('line_no', Integer(2)),
('item_no', Integer(5)),
('total', Numeric("999.99"))])
invoice = Dict([
('invoice_no', Integer(4)),
('total', Numeric("999.99")),
('items', Array(3, item))])
text = "0001200.450100004002.000200006198.50"
rec = invoice.unpack(inv)
# rec is now
{'invoice_no': 1,
'items': [{'item_no': 4, 'line_no': 1, 'total': Decimal('2.00')},
{'item_no': 6, 'line_no': 2, 'total': Decimal('198.50')},
{'item_no': None, 'line_no': None, 'total': None}],
'total': Decimal('200.45')}

# Set the last invoice item
rec['items'][-1] = {
'line_no': 3,
'item_no': 10,
'total': Decimal("20")}

# Update the invoice total
rec['total'] = sum(i['total'] for i in rec['items'])
print rec.pack()
# '0001220.500100004002.000200006198.500300010020.00'




See the included tests.py file for more examples.

Errors in Data
------------------------------------
stypes takes the approach that errors in the textual data are not exceptions.
Data errors are to be expected and should be handled in the normal flow of the
program.

stypes includes the notion of an UnconvertedValue. When parsing text that cannot be
deserialized into the destination format, an UnconvertedValue instance is placed in
it's place. All container objects have a has_unconverted() method which allows
client code to easily detect if there was an error.

::
fmt = List([Numeric('99V9'), Integer(4)])
rec = fmt.unpack("44X001A")
print rec
[<UnconvertedValue string='44X' reason="Expected 1 digits. Found 'X'">,
<UnconvertedValue string='001A' reason='expecting all digits for integer'>]

assert rec.has_unconverted() == True

print rec[0].reason
"Expected 1 digits. Found 'X'"

print rec[1].reason
'expecting all digits for integer'



Installation
------------------------

You can install stypes using pip
::
pip install stypes



or download from PyPI at https://pypi.python.org/pypi/stypes/