pyimporters-csv
Sherpa knowledge import plugin for CSV, Excel and plain text files.
It registers three parsers through the pyimporters.plugins entry point group:
| Plugin | Class | Extensions |
|---|---|---|
text |
pyimporters_csv.text:TXTKnowledgeParser |
txt, text, zip |
csv |
pyimporters_csv.csv_parser:CSVKnowledgeParser |
csv, tsv, zip |
excel |
pyimporters_csv.excel:ExcelKnowledgeParser |
xls, xlsx |
Each parser yields Term objects from pyimporters_plugins.base. Archives
(zip) are unwrapped transparently for the text and CSV parsers.
Requirements
- Python 3.12+
pyimporters_plugins(>=1.6.0,<1.7.0),pandas(>=2.0),structlog,openpyxl,xlrd
Installation
pip install pyimporters-csv
Usage
from pathlib import Path
from progress.bar import Bar
from pyimporters_csv.csv_parser import CSVKnowledgeParser, CSVOptionsModel
parser = CSVKnowledgeParser()
options = CSVOptionsModel(
encoding="utf-8",
identifier_col="ID",
preferredForm_col="prefLabel_en",
altForms_cols="altLabel_en",
multivalue_separator="|",
)
terms = list(parser.parse(Path("terms.csv"), options.model_dump(), Bar("Processing")))
Options
TXTOptions (text parser):
| Option | Default | Description |
|---|---|---|
encoding |
utf-8 |
Encoding of the file |
CSVOptions extends TXTOptions:
| Option | Default | Description |
|---|---|---|
separator |
, |
Field separator |
quotechar |
" |
Quote character |
multivalue_separator |
| |
Separator used to split multivalued columns |
header |
0 |
Row number (0-indexed) used as column names, blank if there is no header |
identifier_col |
identifier |
Name of the column used as the concept identifier |
preferredForm_col |
preferredForm |
Name of the column used as the preferred form |
altForms_cols |
altForms |
Name of the column used as alternative forms |
ExcelOptions is the same as CSVOptions without separator and quotechar.
The three *_col options take a column name, not an index: an unknown name
falls back to the default (column 0 for the identifier, the identifier column for
the preferred form, no alternative forms at all) and logs a warning. Values in
altForms_cols are split on multivalue_separator, so a single column can hold
several alternative forms.
Development
The build is driven by Task and uv,
with the shared stages coming from the python-archetype submodule:
git submodule update --init
task stages # print the pipeline stages, in order
task # run the pipeline up to (but excluding) py:publish
task -- --skip-tests # same, without the test stage
task up-to -- lint # run the pipeline up to and including one stage
task jenkins # run every stage, exactly what Jenkins runs
Individual stages:
| Task | Description |
|---|---|
task py:sync |
Install the workspace and its dependencies (uv sync) |
task py:lint |
ruff check and ruff format --check |
task py:format |
Reformat the code with ruff |
task py:test |
Run the test suite |
task py:test-marker -- <m> |
Run the tests carrying one pytest marker |
task py:sbom |
Generate a CycloneDX SBOM of the resolved environment |
task py:check-vulnerabilities |
Check for known CVEs |
task py:check-updates |
Check for dependency updates |
task py:build |
Build the wheel and sdist (uv build) |
task py:publish |
Publish the distributions (uv publish) |
uv.lock is not versioned here (the Jenkinsfile removes it before building), so
py:sync always resolves dependencies from scratch.
SBOM & vulnerability check
task py:sbom and task py:check-vulnerabilities wrap the underlying tools; to
run them by hand:
uv sync --extra sbom
uv run cyclonedx-py environment -o sbom.cdx.json --output-format json
uv run pip-audit --format json --output audit-report.json
uv run pip-audit --strict # fail on any known vulnerability
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pyimporters_csv-1.6.44.tar.gz.
File metadata
- Download URL: pyimporters_csv-1.6.44.tar.gz
- Upload date:
- Size: 16.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
21a8fdc25b520b2d95baf274cdafaf5b8d6f9a0adcafef65aebea48f341c758d
|
|
| MD5 |
32b4c2de4399792dfa2c97abac9f954a
|
|
| BLAKE2b-256 |
aa209e49fd8e3854ce7e973e4a6b3c6083bbe76a582ffdbff70c6ce75fb2de4d
|
File details
Details for the file pyimporters_csv-1.6.44-py3-none-any.whl.
File metadata
- Download URL: pyimporters_csv-1.6.44-py3-none-any.whl
- Upload date:
- Size: 7.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7d9780e9ec8e58286d20757bb4f223e144070a10a38939ba7afccd1c0f1ff9a2
|
|
| MD5 |
a4f994d2954d47a7fae2b496e2ebd42d
|
|
| BLAKE2b-256 |
6e1073f5b9d85141dc64976b72aacd4dbf6af9dce4c934ad745f8aef25556584
|
