A grep-like, sed-like command-line tool for line-based processing in Python.
Project description
pyline
GitHub | PyPi | Warehouse | ReadTheDocs | Travis-CI
Pyline: a grep-like, sed-like command-line tool (Python package)
Features
Compatibility with the original pyline recipe
Python str.split by an optional delimiter str (-F, --input-delim)
Python regex (-r, --regex, -R, --regex-options)
Output as txt, csv, tsv, json, html (-O csv, --output-filetype=csv)
Output as Markdown/ReStructuredText checkboxes (-O checkbox, --output-filetype=checkbox)
Lazy sorting (-s, --sort-asc; -S, --sort-desc)
Create path.py (or pathlib) objects from each line (-p, --path-tools)
Functional namedtuples, iterators yield -ing generators
optparse argument parsing (-h, --help)
cookiecutter-pypackage project templating
Why
Somewhat unsurprisingly, I found the original pyline recipe while searching for “python grep sed” (see AUTHORS.rst and LICENSE.psf).
I added an option for setting p = Path(line) in the eval/compile command context and added it to my dotfiles ; where it grew tests and an optparse.OptionParser; and is now promoted to a GitHub project with ReadTheDocs documentation, tests with tox and Travis-CI, and a setup.py for PyPi.
What
Pyline is an ordered MapReduce tool:
- Input Readers:
stdin (default)
file (codecs.open(file, 'r', encoding='utf-8'))
- Map Functions:
Python module imports (-m os)
Python regex pattern (-r '\(.*\)')
path library (p from --pathpy OR --pathlib)
Python codeobj eval output transform:
ls | pyline -m os 'line and os.path.abspath(line.strip())' ls | pyline -r '\(.*\)' 'rgx and (rgx.group(0), rgx.group(1)) or line' ls | pyline -p 'p and p.abspath() or ("# ".format(line))' # With an extra outer loop to bind variables in # (because (_p = p.abspath(); <codeobj>) does not work) find $PWD | pyline --pathpy -m os -m collections --input-delim='/' \ 'p and [collections.OrderedDict(( ("p", p), ("_p", _p), ("_p.split()", str(_p).split(os.path.sep)), ("line.rstrip().split()", line.rstrip().split(os.path.sep)), ("l.split()", l.split(os.path.sep)), ("words", words), ("w", w))) for _p in [p.abspath()]][0]' \ -O json
- Partition Function:
None
- Compare Function:
Result(collections.namedtuple).__cmp__
- Reduce Functions:
bool(), sorted()
- Output Writers:
ResultWriter classes
pyline -O csv pyline -O tsv pyline -O json
Installing
Install from PyPi:
pip install pyline
Install from GitHub as editable (add a pyline.pth in site-packages):
pip install -e git+https://github.com/westurner/pyline#egg=pyline
Usage
Print help:
pyline --help
Process:
# Print every line (null transform) cat ~/.bashrc | pyline line cat ~/.bashrc | pyline l # Number every line cat ~/.bashrc | pyline -n l # Print every word (str.split(input-delim=None)) cat ~/.bashrc | pyline words cat ~/.bashrc | pyline w # Split into words and print (default: tab separated) cat ~/.bashrc | pyline 'len(w) >= 2 and w[1] or "?"' # Select the last word, dropping lines with no words pyline -f ~/.bashrc 'w[-1:]' # Regex matching with groups cat ~/.bashrc | pyline -n -r '^#(.*)' 'rgx and rgx.group()' cat ~/.bashrc | pyline -n -r '^#(.*)' ## Original Examples # Print out the first 20 characters of every line tail access_log | pyline "line[:20]" # Print just the URLs in the access log (seventh "word" in the line) tail access_log | pyline "words[6]"
Work with paths and files:
# List current directory files larger than 1 Kb ls | pyline -m os \ "os.path.isfile(line) and os.stat(line).st_size > 1024 and line" # List current directory files larger than 1 Kb #pip install path.py ls | pyline -p 'p and p.size > 1024 and line'
Documentation
License
.
History
0.3.6 (2015-12-21)
BUG: pyline.py: #!/usr/bin/env python2 [9729816]
0.3.5 (2015-05-24)
BUG,DOC: pyline.py: set default regex_options to ‘’, optparse helpstrings [fa9e9cb]
ENH: pyline.py: add a codefunc() kwarg [be8dcc8]
ENH,TST: pyline.py: add a codefunc() kwarg [91aa0a8]
DOC: pyline.py: docstrings (calling a function, stdlib/vendoring) [ee22e2c]
0.3.4 (2015-04-25)
DOC: HISTORY.txt
0.3.3 (2015-04-25)
TST, BUG, CLN, DOC
0.3.2 (2014-11-30)
DOC: pyline/pyline.py: docstrings, import path as pathpy
BUG: pyline/__init__.py: Set pyline.pyline.__main__ correctly (so that python -m pyline.pyline --help works)
DOC: usage.rst: add :shell: option to ‘pyline –help’ output
0.2.0 (2014-08-24)
BUG: add NullHandler to logger (closes #6)
ENH: Add checkbox output formatter (closes #5)
0.1.5 (2014-05-12)
DOC: Updated HISTORY.rst
DOC: setup.py keywords, classifiers
0.1.4 (2014-05-12)
DOC: setup.py version, download_url, license
0.1.3 (2014-05-12)
DOC: setup.py description spans newline
0.1.2 (2014-05-12)
DOC: Setup.py long_description
0.1.1 (2014-05-12)
CLN: factor functions out of main and pyline
BUG: -p path.py option
0.1.0 (2014-05-12)
First release on PyPI.
0.0.1 (Unreleased)
Updated 2012.11.17, Wes Turner
Updated 2005.07.21, thanks to Jacob Oscarson
Updated 2006.03.30, thanks to Mark Eichin
Credits
Graham Fawcett
Jacob Oscarson
Mark Eichin
Wes Turner – https://github.com/westurner
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pyline-0.3.6-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1f9617ded59dde58c415a82d938cfd52e1c4b260970da83aef20827078d0f5dd |
|
MD5 | d83eebb1da1cbd8de64ad6e6276a2670 |
|
BLAKE2b-256 | 75d38c2e399aaeec41ee3f9deb7af71a9a39cda59bfb3dfc50c45e118909bc6b |