Skip to main content

Encode and Decode Textual Data into Rich Python Data Structures

Project description

Encode and Decode Textual Data into Rich Python Data Structures

stypes is a text parsing, conversion and formatting library written to
efficiently handle large fixed-width text record data files. Convert text
streams into dictionaries, lists, tuples, named tuples, ordered dictionaries
and more using text layout specifications. Nested data structures and
repeating elements are also supported.

One of the main design goals of the library was to handle legacy text-based
record data that is commonly found in COBOL system. Of course, the library
can be used to handle any fixed layout textual data.

A simple example of turning some text into a Named Tuple:

from decimal import Decimal
from stypes import NamedTuple, Integer, Numeric
spec = NamedTuple([
('name', 10),
('age', Integer(3)),
('weight', Numeric('999V99'))])
text = "Johnson 2109750"
rec = spec.unpack(text)
assert == 'Johnson'
assert rec.age == 21
assert rec.weight == Decimal("97.5")

And a more interesting example using nested data structures of a list of
records and actually updating a record.

from stypes import Array, Dict, Integer, Numeric
item = Dict([('line_no', Integer(2)),
('item_no', Integer(5)),
('total', Numeric("999.99"))])
invoice = Dict([
('invoice_no', Integer(4)),
('total', Numeric("999.99")),
('items', Array(3, item))])
text = "0001200.450100004002.000200006198.50"
rec = invoice.unpack(inv)
# rec is now
{'invoice_no': 1,
'items': [{'item_no': 4, 'line_no': 1, 'total': Decimal('2.00')},
{'item_no': 6, 'line_no': 2, 'total': Decimal('198.50')},
{'item_no': None, 'line_no': None, 'total': None}],
'total': Decimal('200.45')}

# Set the last invoice item
rec['items'][-1] = {
'line_no': 3,
'item_no': 10,
'total': Decimal("20")}

# Update the invoice total
rec['total'] = sum(i['total'] for i in rec['items'])
print rec.pack()
# '0001220.500100004002.000200006198.500300010020.00'

See the included file for more examples.

Errors in Data
stypes takes the approach that errors in the textual data are not exceptions.
Data errors are to be expected and should be handled in the normal flow of the

stypes includes the notion of an UnconvertedValue. When parsing text that cannot be
deserialized into the destination format, an UnconvertedValue instance is placed in
it's place. All container objects have a has_unconverted() method which allows
client code to easily detect if there was an error.

fmt = List([Numeric('99V9'), Integer(4)])
rec = fmt.unpack("44X001A")
print rec
[<UnconvertedValue string='44X' reason="Expected 1 digits. Found 'X'">,
<UnconvertedValue string='001A' reason='expecting all digits for integer'>]

assert rec.has_unconverted() == True

print rec[0].reason
"Expected 1 digits. Found 'X'"

print rec[1].reason
'expecting all digits for integer'


You can install stypes using pip
pip install stypes

or download from PyPI at

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for stypes, version 0.1.44
Filename, size File type Python version Upload date Hashes
Filename, size stypes-0.1.44.tar.gz (20.0 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page