Skip to main content

Find (fuzzy) matches between two CSV files in the terminal.

Project description

CSV Match
=========

Find (fuzzy) matches between two CSV files in the terminal.

Tested on Python 2.7 and 3.5.


Installing
----------

pip install csvmatch


Usage
-----

Say you have one CSV file such as:

```
name
George Smiley
Percy Alleline
Roy Bland
Toby Esterhase
Peter Guillam
Bill Haydon
Oliver Lacon
Jim Prideaux
Connie Sachs
```

And another such as:

```
name
Maria Andreyevna Ostrakova
Otto Leípzig
George Smiley
Peter Guillam
Connie Sachs
Saul Enderby
Sam Collins
Toby Esterházy
Claus Kretzschmar
```

You can then find the matches:

```bash
$ csvmatch data1.csv data2.csv

name,name
Peter Guillam,Peter Guillam
George Smiley,George Smiley
Connie Sachs,Connie Sachs
```

By default, all columns are used to compare rows. Specific columns can be also be given to be compared -- these should be in the same order for both files. Column headers with a space should be enclosed in quotes.

```bash
$ csvmatch dataA.csv dataB.csv \
--fields1 name address \
--fields2 'Person Name' Address \
> results.csv
```

(This example also uses output redirection to save the results to a file.)

Either file can also be piped in using `-` as a placeholder:

```bash
$ cat data1.csv | csvmatch - data2.csv
```

CSV Match also supports fuzzy matching. By default this makes use of the [Dedupe library] (https://github.com/datamade/dedupe) built by Forest Gregg and Derek Eder based on the work of Mikhail Bilenko. This algorithm asks you to give a number of examples of records from each dataset that are the same -- this information is extrapolated to link the rest of the dataset.

```bash
$ csvmatch dataX.csv dataY.csv --fuzzy
```

The more examples you give it, the better the results will be. At minimum, you should try to provide 10 positive matches and 10 negative matches.

CSV Match also supports the [Double Metaphone] (https://en.wikipedia.org/wiki/Metaphone#Double_Metaphone) phonetic matching algorithm, which is based on how words are pronounced:

```bash
$ csvmatch data1.csv data2.csv --fuzzy metaphone
```

In the first example, an exact match missed that Toby Esterhase and Toby Esterházy are the same, despite being spelt differently. Metaphone will pick up such differences.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for csvmatch, version 1.3
Filename, size File type Python version Upload date Hashes
Filename, size csvmatch-1.3-py2.py3-none-any.whl (7.8 kB) File type Wheel Python version 2.7 Upload date Hashes View
Filename, size csvmatch-1.3.tar.gz (4.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page