Treat lines of a file as elements of a set
Project description
Introduction
Lines is a simple program that allows you to manipulate lines of a file as if they were members of a set. It also provides a few other useful functions to analyse such files.
Set operations
Given two files
file1 containing
a b c d
and file2 with
c d e f
It’s possible to do things like
Unions
lines -u file1 file2
gives
a b c d e f
Intersections
lines -i file1 file2
gives
c d
Difference (All elements in file1 that are not in file2).
lines -d file1 file2
gives
a b
- Symmetric difference (All elements present in only one of the
sets).
lines -s file1 file2
gives
a b e f
Other Operations
These are a few other operations which I’ve found useful
Squeeze blanks
This operation squeezes out the blank lines in a file.
So, If you run lines –squeeze file1 where file1 looks like this
a b c
d
f
You’d get
a b c d f
Identify Patterns
This partitions the elements of the set into subsets all of whose members have an upper bound on the levenshtein distance from each other. This is useful to identify patterns in the input file.
So, if I have a file examples/f6 that looks like this
Archive.001-of-020.part Archive.002-of-020.part Archive.003-of-020.part Archive.004-of-020.part Archive.005-of-020.part Archive.006-of-020.part Archive.007-of-020.part .Archive.008-of-020.part.zbnrw Archive.009-of-020.part Archive.010-of-020.part Archive.011-of-020.part Archive.012-of-020.part Archive.013-of-020.part Archive.014-of-020.part Archive.015-of-020.part Archive.016-of-020.part Archive.017-of-020.part Archive.018-of-020.part Archive.019-of-020.part Archive.020-of-020.part
I can run python lines.py –patterns -l 5 examples/f6 and get
19 elements 1 elements - {‘.Archive.008-of-020.part.zbnrw’}
The -l 5 is to set the upper bound on the levenshtein distance to 5. The -p option allows us to specify an “outlier percentage”. If the number of elements in a subset is below this, it will print all the elements of the subset. This is useful to see the items that don’t match the general pattern in the file.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.