csvw
This package provides
- a Python API to read and write relational, tabular data according to the CSV on the Web specification and
- commandline tools for reading and validating CSVW data.
Links
- GitHub: https://github.com/cldf/csvw
- PyPI: https://pypi.org/project/csvw
- Issue Tracker: https://github.com/cldf/csvw/issues
Installation
This package runs under Python >=3.9, use pip to install:
$ pip install csvw
CLI
csvw2json
Converting CSVW data to JSON
$ csvw2json tests/fixtures/zipped-metadata.json
{
"tables": [
{
"url": "tests/fixtures/zipped.csv",
"row": [
{
"url": "tests/fixtures/zipped.csv#row=2",
"rownum": 1,
"describes": [
{
"ID": "abc",
"Value": "the value"
}
]
},
{
"url": "tests/fixtures/zipped.csv#row=3",
"rownum": 2,
"describes": [
{
"ID": "cde",
"Value": "another one"
}
]
}
]
}
]
}
csvwvalidate
Validating CSVW data
$ csvwvalidate tests/fixtures/zipped-metadata.json
OK
csvwdescribe
Describing tabular-data files with CSVW metadata
$ csvwdescribe --delimiter "|" tests/fixtures/frictionless-data.csv
{
"@context": "http://www.w3.org/ns/csvw",
"dc:conformsTo": "data-package",
"tables": [
{
"dialect": {
"delimiter": "|"
},
"tableSchema": {
"columns": [
{
"datatype": "string",
"name": "FK"
},
{
"datatype": "integer",
"name": "Year"
},
{
"datatype": "string",
"name": "Location name"
},
{
"datatype": "string",
"name": "Value"
},
{
"datatype": "string",
"name": "binary"
},
{
"datatype": "string",
"name": "anyURI"
},
{
"datatype": "string",
"name": "email"
},
{
"datatype": "string",
"name": "boolean"
},
{
"datatype": {
"dc:format": "application/json",
"base": "json"
},
"name": "array"
},
{
"datatype": {
"dc:format": "application/json",
"base": "json"
},
"name": "geojson"
}
]
},
"url": "tests/fixtures/frictionless-data.csv"
}
]
}
Python API
Find the Python API documentation at csvw.readthedocs.io.
A quick example for using csvw from Python code:
import json
from csvw import CSVW
data = CSVW('https://raw.githubusercontent.com/cldf/csvw/master/tests/fixtures/test.tsv')
print(json.dumps(data.to_json(minimal=True), indent=4))
[
{
"province": "Hello",
"territory": "world",
"precinct": "1"
}
]
Known limitations
- We read all data which is specified as UTF-8 encoded using the
utf-8-sigcodecs. Thus, if such data starts withU+FEFFthis will be interpreted as BOM and skipped. - Low level CSV parsing is delegated to the
csvmodule in Python's standard library. Thus, if acommentPrefixis specified in aDialectinstance, this will lead to skipping rows where the first value starts withcommentPrefix, even if the value was quoted. - Also, cell content containing
escapecharmay not be round-tripped as expected (when specifyingescapecharor acsvw.DialectwithquoteCharbutdoubleQuote==False), when minimal quoting is specified. This is due to inconsistentcsvbehaviour across Python versions (see https://bugs.python.org/issue44861).
CSVW conformance
While we use the CSVW specification as guideline, this package does not (and probably never will) implement the full extent of this spec.
- When CSV files with a header are read, columns are not matched in order with
column descriptions in the
tableSchema, but instead are matched based on the CSV column header and the column descriptions'nameandtitlesatributes. This allows for more flexibility, because columns in the CSV file may be re-ordered without invalidating the metadata. A stricter matching can be forced by specifying"header": falseand"skipRows": 1in the table's dialect description.
However, csvw.CSVW works correctly for
- 269 out of 270 JSON tests,
- 280 out of 282 validation tests,
- 10 out of 18 non-normative tests
from the CSVW Test suites.
Compatibility with Frictionless Data Specs
A CSVW-described dataset is basically equivalent to a Frictionless DataPackage where all
Data Resources are Tabular Data.
Thus, the csvw package provides some conversion functionality. To
"read CSVW data from a Data Package", there's the csvw.TableGroup.from_frictionless_datapackage method:
from csvw import TableGroup
tg = TableGroup.from_frictionless_datapackage('PATH/TO/datapackage.json')
To convert the metadata, the TableGroup can then be serialzed:
tg.to_file('csvw-metadata.json')
Note that the CSVW metadata file must be written to the Data Package's directory to make sure relative paths to data resources work.
This functionality - together with the schema inference capabilities
of frictionless describe - provides
a convenient way to bootstrap CSVW metadata for a set of "raw" CSV
files, implemented in the csvwdescribe command described above.
See also
- https://www.w3.org/2013/csvw/wiki/Main_Page
- https://csvw.org
- https://github.com/CLARIAH/COW
- https://github.com/CLARIAH/ruminator
- https://github.com/bloomberg/pycsvw
- https://specs.frictionlessdata.io/table-schema/
- https://github.com/theodi/csvlint.rb
- https://github.com/ruby-rdf/rdf-tabular
- https://github.com/rdf-ext/rdf-parser-csvw
- https://github.com/Robsteranium/csvwr
License
This package is distributed under the Apache 2.0 license.
Release files for csvw 4.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| csvw-4.1.1.tar.gz | 83.4 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| csvw-4.1.1-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details |
Total release size: 152.8 kB
Release files / csvw-4.1.1.tar.gz
| Download URL | csvw-4.1.1.tar.gz |
|---|---|
| Size | 83.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5714d43d744e4069e2b6d0e58a67829ca1bf8d0578aad92cfc74e87d69e6cb31
|
|
BLAKE2b-256 checksum How to use checksums |
3fab8c7b1c103a91656d023be24f824a6faab25d9ecc4a43310b134793200458
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.12.3
|
Release files / csvw-4.1.1-py2.py3-none-any.whl
| Download URL | csvw-4.1.1-py2.py3-none-any.whl |
|---|---|
| Size | 69.4 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
fce2c461899290771eaec28c770e2fd0f6bc1324edb81d688c89e09cf222fcdb
|
|
BLAKE2b-256 checksum How to use checksums |
fe9ea4e6f6aea5f0dedd53523c653c8f368eb6a2392bce09d36ac36de1db2c0f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.12.3
|
Security for csvw 4.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
