This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Aho-Corasick string replacement utility

Project Description

Copyright (c) 2015 Will Roberts <wildwilhelm@gmail.com>

Licensed under the MIT License (see file LICENSE.rst for details).

Search and replace on file(s), with matching on fixed strings.

fsed is a tool specially designed for situations where you have to do many string search-and-replace operations with fixed strings (that is, fsed doesn’t do regular expressions). By doing all the searching and replacing on all the patterns at the same time, fsed can be much faster than tools that do string rewriting one pattern at a time (like one-liners in sed or perl).

To do its searching, fsed uses the Aho-Corasick algorithm, which is a very clever way of matching multiple patterns at the same time, and was used to implement the original fgrep Unix utility (now accessed as grep -F). This algorithm is capable of finding matches which overlap each other, and in these cases, fsed must choose which matches to rewrite. The policy adopted by fsed is to be greedy, and always rewrite the shortest, leftmost match first.

For illustration, imagine a situation where we would like to rewrite a with b, aa with c, and aaa with d. What should we do when we see the input string aaa? Should we produce bbb, bc, cb, or d? fsed produces bbb in this case.

Install

fsed is written in Python; you can install it with pip:

pip install fsed

Usage

fsed [OPTIONS] PATTERN_FILE [INPUT_FILE [INPUT_FILE2 ...]]

If one or more INPUT_FILEs are specified, fsed reads and concatenates these as its input; otherwise, fsed reads the standard input.

Options:

--pattern-format=FMT
Set FMT to tsv or sed (default is sed) to specify the format of PATTERN_FILE.
-o/--output=OUTFILE
Specifies that the program output should be written to OUTFILE. If this option is not used, fsed writes to standard output.
-w/--words
Makes fsed match only on word boundaries; this flag instructs fsed to append \b to the beginning and end of every pattern in PATTERN_FILE.
--by-line/--across-lines
Sets whether fsed should process the input line by line or character by character; the default is --across-lines.
--slow
Indicates that fsed should try very hard to always find the longest matches on the input; this is very slow, and forces --by-line to be on.
-q
Quiet operation, do not emit warnings.
-v/--verbose
Turns on debugging output.

Note: fsed runs even faster using PyPy:

pypy -m fsed.fsed [OPTIONS] PATTERN_FILE [INPUT_FILE [INPUT_FILE2 ...]]

Pattern File

PATTERN_FILE contains a list of patterns to search and replace in the input; each pattern is listed on a separate line. fsed supports two formats for specifying patterns. The default, sed, specifies strings and their replacements the way the sed utility does:

s/SEARCH/REPLACE/

The character following the s character is the pattern delimiter, and can be any character (it does not have to be a forward slash).

The other format, tsv, specifies patterns using <TAB> characters as delimiters:

SEARCH<TAB>REPLACE

In this format, there must be only one <TAB> character per line.

Patterns can contain escape characters:

\\
Backslash (\)
\a
ASCII bell (BEL)
\b
Word boundary
\f
ASCII formfeed (FF)
\n
ASCII linefeed (LF)
\r
Carriage Return (CR)
\t
Horizontal Tab (TAB)
\v
ASCII vertical tab (VT)
Release History

Release History

This version
History Node

0.5.3

History Node

0.5.2

History Node

0.5.1

History Node

0.5.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
fsed-0.5.3-py2.py3-none-any.whl (19.3 kB) Copy SHA256 Checksum SHA256 py2.py3 Wheel May 31, 2017
fsed-0.5.3.tar.gz (503.2 kB) Copy SHA256 Checksum SHA256 Source May 31, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting