Skip to main content

No project description provided

Project description

csvw

Build Status codecov Requirements Status PyPI

CSV on the Web

Links

Installation

This package runs under Python >=3.6, use pip to install:

$ pip install csvw

Example

>>> import csvw
>>> tg = csvw.TableGroup.from_file('tests/csv.txt-metadata.json')

>>> tg.check_referential_integrity()
>>> assert len(tg.tables) == 1

>>> assert tg.tables[0] is tg.tabledict['csv.txt']
>>> tg.tables[0].check_primary_key()

>>> from collections import OrderedDict
>>> row = next(tg.tables[0].iterdicts())
>>> assert row == OrderedDict([('ID', 'first'), ('_col.2', 'line')])

>>> assert len(list(tg.tables[0].iterdicts())) == 2

Known limitations

  • We read all data which is specified as UTF-8 encoded using the utf-8-sig codecs. Thus, if such data starts with U+FEFF this will be interpreted as BOM and skipped.
  • Low level CSV parsing is delegated to the csv module in Python's standard library. Thus, if a commentPrefix is specified in a Dialect instance, this will lead to skipping rows where the first value starts with commentPrefix, even if the value was quoted.

Deviations from the CSVW specificaton

While we use the CSVW specification as guideline, this package does not (and probably never will) implement the full extent of this spec.

  • When CSV files with a header are read, columns are not matched in order with column descriptions in the tableSchema, but instead are matched based on the CSV column header and the column descriptions' name and titles atributes. This allows for more flexibility, because columns in the CSV file may be re-ordered without invalidating the metadata. A stricter matching can be forced by specifying "header": false and "skipRows": 1 in the table's dialect description.

Compatibility with Frictionless Data Specs

The CSVW-described dataset is basically equivalent to a [Frictionless DataPackage] where all Data Resources are Tabular Data. Thus, the csvw package provides some conversion functionality. To "read CSVW data from a Data Package", there's the csvw.TableGroup.from_frictionless_datapackage method:

from csvw import TableGroup
tg = TableGroup.from_frictionless_datapackage('PATH/TO/datapackage.json')

To convert the metadata, the TableGroup can then be serialzed:

tg.to_file('csvw-metadata.json')

Note that the CSVW metadata file must be written to the Data Package's directory to make sure relative paths to data resources work.

This functionality - together with the schema inference capabilities of frictionless describe - provides a convenient way to bootstrap CSVW metadata for a set of "raw" CSV files.

See also

License

This package is distributed under the Apache 2.0 license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for csvw, version 1.11.0
Filename, size File type Python version Upload date Hashes
Filename, size csvw-1.11.0-py2.py3-none-any.whl (35.2 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size csvw-1.11.0.tar.gz (34.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page