Convert FLEx data to CLDF-ready CSV.
Project description
cldflex
Convert FLEx data to CLDF-ready CSV.
Many descriptive linguists have annotated language data in a FLEx (SIL's Fieldworks Lexical Explorer) database, which provides perhaps the most popular and accessible assisted segmentation and annotation workflow.
However, a reasonably complete data export is only available in XML, which is not human-friendly, and is not readily converted to other data.
A data format growing in popularity is the CLDF standard, a table-based approach with human-readable datasets, designed to be used in CLLD apps and easily processable by any software that can read CSV files, including R, pandas or spreadsheet applications.
The goal of cldflex
is to convert lexicon and corpus data stored in FLEx to CSV tables, primarily for use in CLDF datasets.
Installation
cldflex
is available on PyPI:
pip install cldflex
Command line usage
At the moment, there are three commands: cldflex corpus
for .flextext
files; cldflex dictionary
and cldflex wordlist
for .lift
files.
All commands create a number of CSV files.
One can either use cldfbench to create one's own CLDF datasets from these files, or add the --cldf
argument to create a simple CLDF dataset.
Project-specific configuration can be passed by --conf your/config.yaml
, or creating a file cldflex.yaml
corpus
Basic usage:
cldflex corpus texts.flextext
Connect the corpus with the lexicon:
cldflex corpus texts.flextext --lexicon lexicon.lift
Create a CLDF dataset:
cldflex corpus texts.flextext --lexicon lexicon.lift --cldf
dictionary
Extract morphemes, morphs, and entries from lexicon.lift
:
cldflex dictionary lexicon.lift
Create a CLDF dataset with a Dictionary
module:
cldflex dictionary lexicon.lift --cldf
wordlist
Create a CLDF dataset with a Wordlist
module:
cldflex wordlist lexicon.lift --cldf
API usage
The functions corresponding to the commands above are cldflex.corpus.convert()
and cldflex.lift2csv.convert()
.
Configuration
There is no default configuration.
Rather, cldflex
will guess values for most of the parameters below and tell you what it's doing.
It is suggested to start out configuration-free until something goes wrong or you want to change something.
Create a YAML file for CLI usage, pass a dict to the convert
methods.
obj_lg
: the object languagegloss_lg
: the language used for glossing / translationmsa_lg
: the language used for storing POS informationlang_id
: the value to be used in the created tablesglottocode
: used to look up language metadata from glottologcsv_cell_separator
: if there are multiple values in a cell (allomorphs, polysemy...), they are by default separated by"; "
form_slices
: set tofalse
if you don't want form slices connecting morphs and word formsmappings
: a dictionary specifying name changes of columns in the created CSV files
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.