Skip to main content

Encode and Decode Textual Data into Rich Python Data Structures

Project description

Encode and Decode Textual Data into Rich Python Data Structures

stypes is a text parsing, conversion and formatting library written to efficiently handle large fixed-width text record data files. Convert text streams into dictionaries, lists, tuples, named tuples, ordered dictionaries and more using text layout specifications. Nested data structures and repeating elements are also supported.

One of the main design goals of the library was to handle legacy text-based record data that is commonly found in COBOL system. Of course, the library can be used to handle any fixed layout textual data.

A simple example of turning some text into a Named Tuple:

from decimal import Decimal
from stypes import NamedTuple, Integer, Numeric
spec = NamedTuple([
	('name', 10),
	('age', Integer(3)),
	('weight', Numeric('999V99'))])
text = "Johnson    2109750"
rec = spec.unpack(text)
assert rec.name == 'Johnson'
assert rec.age == 21
assert rec.weight == Decimal("97.5")

And a more interesting example using nested data structures of a list of records and actually updating a record.

from stypes import Array, Dict, Integer, Numeric
item = Dict([('line_no', Integer(2)),
	('item_no', Integer(5)),
	('total', Numeric("999.99"))])
invoice = Dict([
    ('invoice_no', Integer(4)),
    ('total', Numeric("999.99")),
    ('items', Array(3, item))])
text = "0001200.450100004002.000200006198.50"
rec = invoice.unpack(inv)
# rec is now
  {'invoice_no': 1,
   'items': [{'item_no': 4, 'line_no': 1, 'total': Decimal('2.00')},
	     {'item_no': 6, 'line_no': 2, 'total': Decimal('198.50')},
	     {'item_no': None, 'line_no': None, 'total': None}],
   'total': Decimal('200.45')}

# Set the last invoice item
rec['items'][-1] = {
    'line_no': 3,
    'item_no': 10,
    'total': Decimal("20")}

# Update the invoice total
rec['total'] = sum(i['total'] for i in rec['items'])
print rec.pack()
# '0001220.500100004002.000200006198.500300010020.00'

See the included tests.py file for more examples.

Errors in Data

stypes takes the approach that errors in the textual data are not exceptions. Data errors are to be expected and should be handled in the normal flow of the program.

stypes includes the notion of an UnconvertedValue. When parsing text that cannot be deserialized into the destination format, an UnconvertedValue instance is placed in it's place. All container objects have a has_unconverted() method which allows client code to easily detect if there was an error.

fmt = List([Numeric('99V9'), Integer(4)])
rec = fmt.unpack("44X001A")
print rec
[<UnconvertedValue string='44X' reason="Expected 1 digits. Found 'X'">,
 <UnconvertedValue string='001A' reason='expecting all digits for integer'>]

assert rec.has_unconverted() == True

print rec[0].reason
 "Expected 1 digits. Found 'X'"

print rec[1].reason
 'expecting all digits for integer'

Installation

You can install stypes using pip

pip install stypes

or download from PyPI at https://pypi.python.org/pypi/stypes/

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

stypes-0.23.2.tar.gz (21.2 kB view details)

Uploaded Source

Built Distribution

stypes-0.23.2-py3-none-any.whl (22.8 kB view details)

Uploaded Python 3

File details

Details for the file stypes-0.23.2.tar.gz.

File metadata

  • Download URL: stypes-0.23.2.tar.gz
  • Upload date:
  • Size: 21.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.8.10

File hashes

Hashes for stypes-0.23.2.tar.gz
Algorithm Hash digest
SHA256 74b09c90b115a45f6e1f415a46a145458df034ea354f2ea1d88675856dead86a
MD5 9e50f2015c53d23b3936f0c2bfdefd70
BLAKE2b-256 a27b8474b7ea8e9ba371053422cfdd622a2be65fd57476437768c6f2f48c11cd

See more details on using hashes here.

File details

Details for the file stypes-0.23.2-py3-none-any.whl.

File metadata

  • Download URL: stypes-0.23.2-py3-none-any.whl
  • Upload date:
  • Size: 22.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.8.10

File hashes

Hashes for stypes-0.23.2-py3-none-any.whl
Algorithm Hash digest
SHA256 224bf58a357bf9bf0500cf2912dee83864e284a69ed2bcb9cee31a7c9042d805
MD5 1e5a3d90d1172fa870b04c5c5079bf3a
BLAKE2b-256 b4284af7ce034d102f486f2205b1464ae71b5b4507ef29d2528623661d66c375

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page