pyimporters-csv
Sherpa knowledge import plugin for CSV, Excel and plain text files.
It registers three parsers through the pyimporters.plugins entry point group:
| Plugin | Class | Extensions |
|---|---|---|
text |
pyimporters_csv.text:TXTKnowledgeParser |
txt, text, zip |
csv |
pyimporters_csv.csv_parser:CSVKnowledgeParser |
csv, tsv, zip |
excel |
pyimporters_csv.excel:ExcelKnowledgeParser |
xls, xlsx |
Each parser yields Term objects from pyimporters_plugins.base. Archives
(zip) are unwrapped transparently for the text and CSV parsers.
Requirements
- Python 3.12+
pyimporters_plugins(>=1.6.0,<1.7.0),pandas(>=2.0),structlog,openpyxl,xlrd
Installation
pip install pyimporters-csv
Usage
from pathlib import Path
from progress.bar import Bar
from pyimporters_csv.csv_parser import CSVKnowledgeParser, CSVOptionsModel
parser = CSVKnowledgeParser()
options = CSVOptionsModel(
encoding="utf-8",
identifier_col="ID",
preferredForm_col="prefLabel_en",
altForms_cols="altLabel_en",
multivalue_separator="|",
)
terms = list(parser.parse(Path("terms.csv"), options.model_dump(), Bar("Processing")))
Options
TXTOptions (text parser):
| Option | Default | Description |
|---|---|---|
encoding |
utf-8 |
Encoding of the file |
CSVOptions extends TXTOptions:
| Option | Default | Description |
|---|---|---|
separator |
, |
Field separator |
quotechar |
" |
Quote character |
multivalue_separator |
| |
Separator used to split multivalued columns |
header |
0 |
Row number (0-indexed) used as column names, blank if there is no header |
identifier_col |
identifier |
Name of the column used as the concept identifier |
preferredForm_col |
preferredForm |
Name of the column used as the preferred form |
altForms_cols |
altForms |
Name of the column used as alternative forms |
ExcelOptions is the same as CSVOptions without separator and quotechar.
The three *_col options take a column name, not an index: an unknown name
falls back to the default (column 0 for the identifier, the identifier column for
the preferred form, no alternative forms at all) and logs a warning. Values in
altForms_cols are split on multivalue_separator, so a single column can hold
several alternative forms.
Development
The build is driven by Task and uv,
with the shared stages coming from the python-archetype submodule:
git submodule update --init
task stages # print the pipeline stages, in order
task # run the pipeline up to (but excluding) py:publish
task -- --skip-tests # same, without the test stage
task up-to -- lint # run the pipeline up to and including one stage
task jenkins # run every stage, exactly what Jenkins runs
Individual stages:
| Task | Description |
|---|---|
task py:sync |
Install the workspace and its dependencies (uv sync) |
task py:lint |
ruff check and ruff format --check |
task py:format |
Reformat the code with ruff |
task py:test |
Run the test suite |
task py:test-marker -- <m> |
Run the tests carrying one pytest marker |
task py:sbom |
Generate a CycloneDX SBOM of the resolved environment |
task py:check-vulnerabilities |
Check for known CVEs |
task py:check-updates |
Check for dependency updates |
task py:build |
Build the wheel and sdist (uv build) |
task py:publish |
Publish the distributions (uv publish) |
uv.lock is not versioned here (the Jenkinsfile removes it before building), so
py:sync always resolves dependencies from scratch.
SBOM & vulnerability check
task py:sbom and task py:check-vulnerabilities wrap the underlying tools; to
run them by hand:
uv sync --extra sbom
uv run cyclonedx-py environment -o sbom.cdx.json --output-format json
uv run pip-audit --format json --output audit-report.json
uv run pip-audit --strict # fail on any known vulnerability
Release files for pyimporters-csv 1.6.474
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pyimporters_csv-1.6.474.tar.gz | 34.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pyimporters_csv-1.6.474-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 42.0 kB
Release files / pyimporters_csv-1.6.474.tar.gz
| Download URL | pyimporters_csv-1.6.474.tar.gz |
|---|---|
| Size | 34.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
514a6b126ed05cd7f8b24f453925ded8fd42347bf13b6df9511af33e60feeda5
|
|
BLAKE2b-256 checksum How to use checksums |
338f53abddce99716dc8906e79ebf7f045969b23680fe31a5286115096a0ea6a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / pyimporters_csv-1.6.474-py3-none-any.whl
| Download URL | pyimporters_csv-1.6.474-py3-none-any.whl |
|---|---|
| Size | 7.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8f7672aca117b019e7de1458ff4e8fd66a0784050bd8a6c534e21ed6d48a6089
|
|
BLAKE2b-256 checksum How to use checksums |
5d157f9b6f9c8834cd2e1e38879c787c9353677d0c77ed043086a072a9dc7255
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"22.04","id":"jammy","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
