This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

An assortment of lexcial and tokenisation functions useful for writing recursive descent parsers, of which I have several.

Generally the get_* functions accept a source string and an offset (often optional, default 0) and return a token and the new offset, raising ValueError on failed tokenisation.

  • as_lines(chunks, partials=None): parse text chunks, yield complete individual lines
  • get_chars(s, offset, gochars): collect adjacent characters from gochars
  • get_delimited(s, offset, delim): collect text up to the first ocurrence of the character delim.
  • get_envvar(s, offset=0, environ=None, default=None, specials=None): parse an environment variable reference such as $foo
  • get_identifier(s, offset=0, alpha=ascii_letters, number=digits, extras=’_’): parse an identifier
  • get_nonwhite(s, offset=0): collect nonwhitespace characters
  • get_other_chars(s, offset=0, stopchars=None): collect adjacent characters not from stopchars
  • get_qstr(s, offset=0, q=’”’, environ=None, default=None, env_specials=None): collect a quoted string, honouring slosh escapes and optionally expanding environment variable references
  • get_sloshed_text(s, delim, offset=0, slosh=’', mapper=slosh_mapper, specials=None): collect some slosh escaped text with optional special tokens (such as ‘$’ introducing ‘$foo’)
  • get_tokens(s, offset, getters): collect a sequence of tokens specified in getters
  • match_tokens(s, offset, getters): wrapper for get_tokens which catches ValueError and returns None instead
  • get_uc_identifier(s, offset=0, number=digits, extras=’_’): collect an UPPERCASE identifier
  • get_white(s, offset=0): collect whitespace characters
  • isUC_(s): test if a string looks like an upper case identifier
  • htmlify(s,nbsp=False): transcribe text in HTML safe form, using &lt; for “<”, etc
  • htmlquote(s): transcribe text as HTML quoted string suitable for HTML tag attribute values
  • jsquote(s): transcribe text as JSON quoted string; essentially like htmlquote without its htmlify step
  • parseUC_sAttr(attr): parse FOO or FOOs (or FOOes) and return (FOO, is_plural)
  • slosh_mapper(c, charmap=SLOSH_CHARMAP): return a string to replace c; the default charmap matches Python slosh escapes
  • texthexify(bs, shiftin=’[‘, shiftout=’]’, whitelist=None): a function like binascii.hexlify but also supporting embedded “printable text” subsequences for compactness and human readbility in the result; initial use case was for transcription of binary data with frequent text, specificly directory entry data
  • untexthexify(s, shiftin=’[‘, shiftout=’]’): the inverse of texthexify()
  • unctrl(s,tabsize=8): transcribe text removing control characters
  • unrfc2047(s): accept RFC2047 encoded text as found in mail message headers and decode
Release History

Release History

20160828

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

20150120

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

20150118

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
cs.lex-20160828.tar.gz (9.0 kB) Copy SHA256 Checksum SHA256 Source Aug 28, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting