Skip to main content
Help us improve Python packaging – donate today!

Replacement for shlex (that works with unicode) for Python 2.X.

Project Description

Inspired by ordereddict, this is a packaging of an improved shlex module for Python 2 that handles Unicode properly.

Shlex is “”“A lexical analyzer class for simple shell-like syntaxes.”“”

If you’ve found your way here, you probably already know that the standard shlex doesn’t handle Unicode prior to Python 3 (see bug 1170 for details). Since Python 2.7.3 however, it accepts unicode objects. Sadly, it still does not handle non-ascii chars:

>>> import sys, shlex
>>> sys.version
'2.7.5+ ...'
>>> shlex.split(u'Hello world')
['Hello', 'world']

>>> shlex.split(u'café')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/lib/python2.7/shlex.py", line 275, in split
    lex = shlex(s, posix=posix)
  File "/usr/lib/python2.7/shlex.py", line 25, in __init__
    instream = StringIO(instream)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in
                     position 3: ordinal not in range(128)

This module does handle unicode objects or byte strings:

>>> import ushlex as shlex
>>> shlex.split(u'café')
[u'caf\xe9']

>>> shlex.split(u'echo "☺ ☕ ♫"')
[u'echo', u'\u263a \u2615 \u266b']

>>> from ushlex import split as shplit
>>> shplit('echo "hello there"')
['echo', 'hello there']

I found these release notes inside:

# Module and documentation by Eric S. Raymond, 21 Dec 1998
# Input stacking and error message cleanup added by ESR, March 2000
# push_source() and pop_source() made explicit by ESR, January 2001.
# Posix compliance, split(), string arguments, and
# iterator interface by Gustavo Niemeyer, April 2003.
# Modified to support Unicode by Colin Walters, Dec 2007

Bugs

Packaging-only bugs may be submitted to bitbucket. Do not enter bugs for ushlex itself, as the packager is not the author.

Release history Release notifications

This version
History Node

0.99

History Node

0.98

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
ushlex-0.99.tar.gz (4.6 kB) Copy SHA256 hash SHA256 Source None Aug 2, 2014

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page