csvmod
A stream-oriented CSV modification tool. Like a stripped-down “sed” command, but for tabular data.
TL;DR
Install:
$ pip install csvsed
Use:
# given a sample CSV
$ cat sample.csv
Employee ID,Age,Wage,Status
8783,47,"104,343,873.83","All good, but nowhere to go."
2003,32,"98,878,784.00",A-OK
# modify that data with a series of `csvsed` pipes
$ cat sample.csv \
| csvsed -c Wage s/,//g \ # remove commas from the Wage column
| csvsed -c Status 'y/A-Z/a-z/' \ # convert Status to all lowercase
| csvsed -c Status 's/.*(ok|good).*/\1/' \ # restrict to keywords 'ok' & 'good'
| csvsed -c Age 'e/xargs -I {} echo "{}*2" | bc/' # double the Age column
Employee ID,Age,Wage,Status
8783,94,104343873.83,good
2003,64,98878784.00,ok
Installation
$ pip install csvsed
Usage and Examples
Installation of the csvsed python package also installs the csvsed command-line tool. Use csvsed --help for all command line options, but here are some examples to get you going. Given the input file sample.csv:
Employee ID,Age,Wage,Status
8783,47,"104,343,873.83","All good, but nowhere to go."
2003,32,"98,878,784.00",A-OK
Removing thousands-separators from the “Wage” column using the “s” (substitute) modifier:
$ cat sample.csv | csvsed -c Wage s/,//g
Employee ID,Age,Wage,Status
8783,47,104343873.83,"All good, but nowhere to go."
2003,32,98878784.00,A-OK
Convert/extract some text using the “s” (substitute) and “y” (transliterate) modifiers:
$ cat sample.csv | csvsed -c Status 's/^All (.*),.*/\1/' \
| csvsed -c Status 's/^A-(.*)/\1/' \
| csvsed -c Status 'y/a-z/A-Z/'
Employee ID,Age,Wage,Status
8783,47,"104,343,873.83",GOOD
2003,32,"98,878,784.00",OK
Square the “Age” column using the “e” (execute) modifier:
$ cat sample.csv | csvsed -c Age 'e/xargs -I {} echo "{}^2" | bc/'
Employee ID,Age,Wage,Status
8783,2209,"104,343,873.83","All good, but nowhere to go."
2003,1024,"98,878,784.00",A-OK
That, however, called the external program for each column (quite inefficient with large data sets)… so let’s do that more efficiently, with a “continuous” mode program. Given the following id2name.py program which takes a CSV on STDIN with a single column (an employee ID) and writes a CSV to STDOUT with the IDs converted to names:
#!/usr/bin/env python
import sys, csvkit
table = {'8783': 'ElfenKyng', '2003': 'Stradivarius'}
# NOTE: *not* using csvkit's reader because it reads-ahead
# causing problems since this must be stream-oriented...
writer = csvkit.CSVKitWriter(sys.stdout)
while True:
item = sys.stdin.readline()
if not item: break
item = item.strip()
writer.writerow([table[item] if item in table else item])
sys.stdout.flush()
Then the following will efficiently convert the ‘Employee ID’ column to names:
$ cat sample.csv | csvsed -c 'Employee ID' 'e|./id2name.py|c'
Employee ID,Age,Wage,Status
ElfenKyng,47,"104,343,873.83","All good, but nowhere to go."
Stradivarius,32,"98,878,784.00",A-OK
Metadata
Release files for csvsed 0.2.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| csvsed-0.2.4.tar.gz | 21.0 kB | Without provenance | Details for csvsed-0.2.4.tar.gz |
Release files / csvsed-0.2.4.tar.gz
| Download URL | csvsed-0.2.4.tar.gz |
|---|---|
| Size | 21.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d5f68c9e2770990e3dba4af0dead31181c005db0cfe29e0efd2e1075a338c86a
|
|
BLAKE2b-256 checksum How to use checksums |
04f3875e57e6c2fe2809d6e6071a31f884822e26a4db0998152a4c0e3dc94a16
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for csvsed 0.2.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.