Skip to main content

Fast parser for fixed-column line data files

Project description

lineparser

lineparser is a small library with one goal: parse fixed-width formatted files extremely quickly. In order to achieve this, lineparser uses Cython to obtain the speed offered by good old C code and the convenience of Python.

Documentation can be found here.

Installing

You should be able to install lineparser via pip if you are on a Windows or Linux 64 bit machine.

$ pip3 install lineparser

Example

demo.py:

import lineparser
import time

fields = [(lineparser.Float64, 12), (lineparser.Float64, 10), (lineparser.Float64, 12), 
        (lineparser.String, 6), (lineparser.String, 6), (lineparser.String, 14),
        (lineparser.String, 14), (lineparser.Float64, 6)]

try:
    start = time.time()

    # On a successfull parse, result will be a list of lists and numpy arrays (strings will be in 
    # lists, and numbers will be in numpy arrays). 
    result = lineparser.parse(fields, b'data/small_data.par')

    end = time.time()

    print(f"Took {end - start} seconds to parse")
except lineparser.LineParsingError as e:
    print(f"Encountered the following error while trying to parse:\n {str(e)}")

data/small_data.par

    31.43339 6.531E-28   31.442390     3     2       0.00048       0.00000   100
    41.89467 1.415E-26   62.878170     4     3       0.00065       0.00000   100
    41.89786 3.538E-27   62.876840     4     3       0.00064       0.00000   100
    ...

Running the example: python3 demo.py

Installing from Source

Installing from source is also easy. You must have GCC installed on your machine, and you must have Cython installed only if you want to modify the library. Then run this command:

$ python3 setup.py install

Building

lineparser is simple to build, and should only require one command:

$ python3 setup.py build_ext --inplace

You should then be able to import it in a python interpreter (in the build directory):

$ python3 setup.py build_ext --inplace
$ python3 
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lineparser
>>> ...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

lineparser-0.0.0.dev7-cp37-cp37m-win_amd64.whl (116.8 kB view details)

Uploaded CPython 3.7mWindows x86-64

lineparser-0.0.0.dev7-cp36-cp36m-win_amd64.whl (115.7 kB view details)

Uploaded CPython 3.6mWindows x86-64

File details

Details for the file lineparser-0.0.0.dev7-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: lineparser-0.0.0.dev7-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 116.8 kB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.7.4

File hashes

Hashes for lineparser-0.0.0.dev7-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 2671e11cdf7764dd5c78b993d359d22507ce3d960248d87c17e47504d161b5d3
MD5 8a87a5e21d7ee2e4bfa72bc225bcc025
BLAKE2b-256 52cda3854cf7d8a95892a0ee9478c71129fadcaa51933f2a317e85c240b865e7

See more details on using hashes here.

File details

Details for the file lineparser-0.0.0.dev7-cp36-cp36m-win_amd64.whl.

File metadata

  • Download URL: lineparser-0.0.0.dev7-cp36-cp36m-win_amd64.whl
  • Upload date:
  • Size: 115.7 kB
  • Tags: CPython 3.6m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.6.4

File hashes

Hashes for lineparser-0.0.0.dev7-cp36-cp36m-win_amd64.whl
Algorithm Hash digest
SHA256 a3bacf687adaf3d76ada444f365bcc7ad380dbd3a2406f728cff7a6b86b5082e
MD5 0aca2ac31b913c95fc1d579002dbc9c4
BLAKE2b-256 29306e567227a37c7f79be12f8c801738d2d107d6efac95712f3eb66c6a8f6c9

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page