A simple command-line interface to work with CSV, excel and parquet files
Project description
___
/\_ \ __
___ ____ __ __ ___\//\ \ /\_\
/'___\ /',__\/\ \/\ \ _______ /'___\\ \ \ \/\ \
/\ \__//\__, `\ \ \_/ |/\______\/\ \__/ \_\ \_\ \ \
\ \____\/\____/\ \___/ \/______/\ \____\/\____\\ \_\
\/____/\/___/ \/__/ \/____/\/____/ \/_/
Description
A simple command-line interface to work with CSV, excel and parquet files. You can use it to:
-
Explore your data:
- navigate through the full contents of your tabular data fast and with a human-friendly format directly on the shell
- quickly see which columns, data-types are in the file and how many null values or unique values are per column
-
Filter and query:
- select subsets of the tabular data
- sort by a given column
- Need more? Ok then just run SQL queries on it!
- In all cases you can save the output to a new file
-
Change the format:
- You can convert from and to CSV, excel and parquet in any combination
- You can change the delimiter of your CSV file
Python version
Built on and tested on python 3.7. Installation on 3.7 tested and working.
Installation
Using Pip
$ pip install csvcli
Manual
$ git clone https://github.com/IgnacioMB/csvcli.git
$ cd csvcli
$ python setup.py install
Global parameters
Usage: csvcli [OPTIONS] FILEPATH COMMAND [ARGS]...
Arguments
FILEPATH
TEXT Path to the file i.e. myfiles/data.csv. You might need to wrap it in "" if your filename contains parentheses and such.
Options:
-
-d, --delimiter
TEXT (optional) Only for CSV files. If you want to override the automatic guess. Must be a 1-character string. -
--help
Show this message and exit.
Available commands
___
/\_ \
__ __ _ _____\//\ \ ___ _ __ __
/'__`\/\ \/'\/\ '__`\\ \ \ / __`\/\`'__\/'__`\
/\ __/\/> </\ \ \L\ \\_\ \_/\ \L\ \ \ \//\ __/
\ \____\/\_/\_\\ \ ,__//\____\ \____/\ \_\\ \____\
\/____/\//\/_/ \ \ \/ \/____/\/___/ \/_/ \/____/
\ \_\
\/_/
__ __ ___ __ __ _ __
/\ \/\ \ / __`\/\ \/\ \/\`'__\
\ \ \_\ \/\ \L\ \ \ \_\ \ \ \/
\/`____ \ \____/\ \____/\ \_\
`/___/> \/___/ \/___/ \/_/
/\___/
\/__/
__ __
/\ \ /\ \__
\_\ \ __ \ \ ,_\ __
/'_` \ /'__`\ \ \ \/ /'__`\
/\ \L\ \/\ \L\.\_\ \ \_/\ \L\.\_
\ \___,_\ \__/.\_\\ \__\ \__/.\_\
\/__,_ /\/__/\/_/ \/__/\/__/\/_/
These commands allow you to quickly get a sense of what the contents of the file look like.
-
show
: Displays the contents of the CSV, excel or parquet fileExample showing the contents of a CSV file.
csvcli myfiles/data.csv show
When working with CSV files, csvcli will try to guess the delimiter of your CSV files for you. If you are not happy with the guess, you can always specify the delimiter using the
-d
option:csvcli -d '|' myfiles/csv_with_pipes.csv show
-
head
: Displays only the first rows of the fileIf you do not indicate any number, it returns the first 5 rows of the file:
csvcli myfiles/data.csv head
You can also specify a custom number of rows to show:
csvcli myfiles/data.csv head 100
-
columns
: Displays the column names and data typescsvcli myfiles/data.csv columns
-
describe
: Displays a table with summary statistics of the numerical columnscsvcli myfiles/data.csv describe
-
null-counts
: Displays the counts of null values per columncsvcli myfiles/data.csv null-counts
-
value-counts
: Displays the unique values in a column with their respective counts. You must indicate a column using the-c
optionOptions:
-c, --column
TEXT Name of column to count the unique values for
Example
csvcli myfiles/data.csv value-counts -c "Region"
___ ___ __
/'___\ __ /\_ \ /\ \__
/\ \__//\_\\//\ \ \ \ ,_\ __ _ __
\ \ ,__\/\ \ \ \ \ \ \ \/ /'__`\/\`'__\
\ \ \_/\ \ \ \_\ \_\ \ \_/\ __/\ \ \/
\ \_\ \ \_\/\____\\ \__\ \____\\ \_\
\/_/ \/_/\/____/ \/__/\/____/ \/_/
____
/| _ \
|/\ |
\// __`\/\
/| \L> <_
| \_____/\/
\/____/\/
__ __ __ __ _ __ __ __
/'__`\/\ \/\ \ /'__`\/\`'__\/\ \/\ \
/\ \L\ \ \ \_\ \/\ __/\ \ \/ \ \ \_\ \
\ \___, \ \____/\ \____\\ \_\ \/`____ \
\/___/\ \/___/ \/____/ \/_/ `/___/> \
\ \_\ /\___/
\/_/ \/__/
-
select
: Allows you to display only a subset of columns. Also supports sorting by a given column.Options:
-c, --columns
TEXT Names of columns to show separated by commas-s, --sort-by
TEXT Name of column to sort by-asc, --ascending
BOOLEAN True for ascending and False for descending-save, --save-to
TEXT Path to the destination file i.e. 'myfiles/data.csv'. The file extension determines output format
Example selecting columns from a CSV file:
csvcli myfiles/data.csv select -c "url, clicks, impressions"
Example selecting columns and sorting by one using the
-s
option. You can addASC
for ascending order orDESC
for descending order. The default is ascending:csvcli myfiles/data.csv select -c "url, clicks, impressions" -s "clicks" DESC
Example saving a selection result into an output file using the option
-save
:csvcli myfiles/data.csv select -c "region, count" -save "subset.csv"
-
query
: If you need more advanced filters and functions, the query command allows you to query the CSV, excel or parquet file using SQL queries as you would any regular SQL table. You specify the query using the-q
option and use the keywordfile
to refer to your file as a source table.Options:
-q, --query
TEXT SQL query you want to run against the file i.e.SELECT * FROM file;
-save, --save-to
TEXT Path to the destination file i.e. 'myfiles/data.csv'. The file extension determines output format
Example running a query on a CSV file:
csvcli myfiles/data.csv query -q "SELECT Region,SUM(Units) FROM file GROUP BY Region;"
Example saving a query result into an output file using the option
-save
:csvcli myfiles/data.csv query -q "SELECT Region,Units FROM file;" -save "query.csv"
__
/\ \
___\ \ \___ __ ___ __ __
/'___\ \ _ `\ /'__`\ /' _ `\ /'_ `\ /'__`\
/\ \__/\ \ \ \ \/\ \L\.\_/\ \/\ \/\ \L\ \/\ __/
\ \____\\ \_\ \_\ \__/.\_\ \_\ \_\ \____ \ \____\
\/____/ \/_/\/_/\/__/\/_/\/_/\/_/\/___L\ \/____/
/\____/
\_/__/
__ __
/\ \__/\ \
\ \ ,_\ \ \___ __
\ \ \/\ \ _ `\ /'__`\
\ \ \_\ \ \ \ \/\ __/
\ \__\\ \_\ \_\ \____\
\/__/ \/_/\/_/\/____/
___ __
/'___\ /\ \__
/\ \__/ ___ _ __ ___ ___ __ \ \ ,_\
\ \ ,__\/ __`\/\`'__\/' __` __`\ /'__`\ \ \ \/
\ \ \_/\ \L\ \ \ \/ /\ \/\ \/\ \/\ \L\.\_\ \ \_
\ \_\\ \____/\ \_\ \ \_\ \_\ \_\ \__/.\_\\ \__\
\/_/ \/___/ \/_/ \/_/\/_/\/_/\/__/\/_/ \/__/
convert
: Allows you to convert from and to CSV, excel and parquet in any combination. Your original file will be overwritten.
Options:
-to, --format
TEXT Output format. Options: 'csv', 'excel' or 'parquet'-D, --delimiter
TEXT (optional) Only for CSV files. Delimiter if other than comma i.e. ';'. Must be a 1-character string.
Example converting a parquet file to CSV:
csvcli myfiles/data.parquet convert -to "csv"
Example converting an excel file to CSV with |
as delimiter. using the -D
option:
csvcli myfiles/data.xlsx convert -to "csv" -D "|"
-
change-delimiter
: Changes the delimiter of a CSV file. Your original file will be overwritten.Options:
-to, --new-delimiter
TEXT Output CSV delimiter i.e. ';'. Must be a 1-character string.
Example changing the delimiter of a CSV file originally separated by ';' to '|':
csvcli -d ";" data.csv change-delimiter -to "|"
Note about the author
Ignacio Marin is a Data Analyst based in Munich, Germany.
More info on https://www.linkedin.com/in/ignaciomarinb/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.