pyline 0.3.5

A grep-like, sed-like command-line tool for line-based processing in Python.

pyline

GitHub | PyPi | Warehouse | ReadTheDocs | Travis-CI

Pyline: a grep-like, sed-like command-line tool (Python package)

Features

• Compatibility with the original pyline recipe

• Python str.split by an optional delimiter str (-F, --input-delim)

• Python regex (-r, --regex, -R, --regex-options)

• Output as txt, csv, tsv, json, html (-O csv, --output-filetype=csv)

• Output as Markdown/ReStructuredText checkboxes (-O checkbox, --output-filetype=checkbox)

• Lazy sorting (-s, --sort-asc; -S, --sort-desc)

• Create path.py (or pathlib) objects from each line (-p, --path-tools)

• Functional namedtuples, iterators yield -ing generators

• optparse argument parsing (-h, --help)

• cookiecutter-pypackage project templating

Why

Somewhat unsurprisingly, I found the original pyline recipe while searching for “python grep sed” (see AUTHORS.rst and LICENSE.psf).

I added an option for setting p = Path(line) in the eval/compile command context and added it to my dotfiles ; where it grew tests and an optparse.OptionParser; and is now promoted to a GitHub project with ReadTheDocs documentation, tests with tox and Travis-CI, and a setup.py for PyPi.

What

Pyline is an ordered MapReduce tool:

Input Readers:

• stdin (default)

• file (codecs.open(file, 'r', encoding='utf-8'))

Map Functions:

• Python module imports (-m os)

• Python regex pattern (-r '\(.*\)')

• path library (p from --pathpy OR --pathlib)

• Python codeobj eval output transform:

  ls | pyline -m os 'line and os.path.abspath(line.strip())'
  ls | pyline -r '\(.*\)' 'rgx and (rgx.group(0), rgx.group(1)) or line'
  ls | pyline -p 'p and p.abspath() or ("# ".format(line))'
  
  # With an extra outer loop to bind variables in
  # (because (_p = p.abspath(); <codeobj>) does not work)
  find $PWD | pyline --pathpy -m os -m collections --input-delim='/' \
      'p and [collections.OrderedDict((
              ("p", p),
              ("_p", _p),
              ("_p.split()", str(_p).split(os.path.sep)),
              ("line.rstrip().split()", line.rstrip().split(os.path.sep)),
              ("l.split()", l.split(os.path.sep)),
              ("words", words),
              ("w", w)))
          for _p in [p.abspath()]][0]' \
         -O json

Partition Function:

None

Compare Function:

Result(collections.namedtuple).__cmp__

Reduce Functions:

bool(), sorted()

Output Writers:

ResultWriter classes

pyline -O csv
pyline -O tsv
pyline -O json

Installing

Install from PyPi:

pip install pyline

Install from GitHub as editable (add a pyline.pth in site-packages):

pip install -e git+https://github.com/westurner/pyline#egg=pyline

Usage

Print help:

pyline --help

Process:

# Print every line (null transform)
cat ~/.bashrc | pyline line
cat ~/.bashrc | pyline l

# Number every line
cat ~/.bashrc | pyline -n l

# Print every word (str.split(input-delim=None))
cat ~/.bashrc | pyline words
cat ~/.bashrc | pyline w

# Split into words and print (default: tab separated)
cat ~/.bashrc | pyline 'len(w) >= 2 and w[1] or "?"'

# Select the last word, dropping lines with no words
pyline -f ~/.bashrc 'w[-1:]'

# Regex matching with groups
cat ~/.bashrc | pyline -n -r '^#(.*)' 'rgx and rgx.group()'
cat ~/.bashrc | pyline -n -r '^#(.*)'

## Original Examples
# Print out the first 20 characters of every line
tail access_log | pyline "line[:20]"

# Print just the URLs in the access log (seventh "word" in the line)
tail access_log | pyline "words[6]"

Work with paths and files:

# List current directory files larger than 1 Kb
ls | pyline -m os \
  "os.path.isfile(line) and os.stat(line).st_size > 1024 and line"

# List current directory files larger than 1 Kb
#pip install path.py
ls | pyline -p 'p and p.size > 1024 and line'

Documentation

https://pyline.readthedocs.org/en/latest/

License

Python Software License

.

History

0.3.5 (2015-05-24)

• BUG,DOC: pyline.py: set default regex_options to ‘’, optparse helpstrings [fa9e9cb]

• ENH: pyline.py: add a codefunc() kwarg [be8dcc8]

• ENH,TST: pyline.py: add a codefunc() kwarg [91aa0a8]

• DOC: pyline.py: docstrings (calling a function, stdlib/vendoring) [ee22e2c]

0.3.4 (2015-04-25)

• DOC: HISTORY.txt

0.3.3 (2015-04-25)

• TST, BUG, CLN, DOC

0.3.2 (2014-11-30)

• DOC: pyline/pyline.py: docstrings, import path as pathpy

• BUG: pyline/__init__.py: Set pyline.pyline.__main__ correctly (so that python -m pyline.pyline --help works)

• DOC: usage.rst: add :shell: option to ‘pyline –help’ output

0.2.0 (2014-08-24)

• BUG: add NullHandler to logger (closes #6)

• ENH: Add checkbox output formatter (closes #5)

0.1.5 (2014-05-12)

• DOC: Updated HISTORY.rst

• DOC: setup.py keywords, classifiers

0.1.4 (2014-05-12)

• DOC: setup.py version, download_url, license

0.1.3 (2014-05-12)

• DOC: setup.py description spans newline

0.1.2 (2014-05-12)

• DOC: Setup.py long_description

0.1.1 (2014-05-12)

• CLN: factor functions out of main and pyline

• BUG: -p path.py option

0.1.0 (2014-05-12)

• First release on PyPI.

0.0.1 (Unreleased)

Source: http://code.activestate.com/recipes/437932-pyline-a-grep-like-sed-like-command-line-tool/

• Updated 2012.11.17, Wes Turner

• Updated 2005.07.21, thanks to Jacob Oscarson

• Updated 2006.03.30, thanks to Mark Eichin

Credits

• Graham Fawcett

• Jacob Oscarson

• Mark Eichin

• Wes Turner – https://github.com/westurner