Skip to main content

Convert FLEx data to CLDF-ready CSV.

Project description

cldflex

Convert FLEx data to CLDF-ready CSV.

Versions PyPI License

Many descriptive linguists have annotated language data in a FLEx (SIL's Fieldworks Lexical Explorer) database, which provides perhaps the most popular and accessible assisted segmentation and annotation workflow. However, a reasonably complete data export is only available in XML, which is not human-friendly, and is not readily converted to other data. A data format growing in popularity is the CLDF standard, a table-based approach with human-readable datasets, designed to be used in CLLD apps and easily processable by any software that can read CSV files, including R, pandas or spreadsheet applications. The goal of cldflex is to convert lexicon and corpus data stored in FLEx to CSV tables, primarily for use in CLDF datasets.

Installation

cldflex is available on PyPI:

pip install cldflex

Command line usage

At the moment, there are three commands: cldflex corpus for .flextext files; cldflex dictionary and cldflex wordlist for .lift files. All commands create a number of CSV files. One can either use cldfbench to create one's own CLDF datasets from these files, or add the --cldf argument to create a simple CLDF dataset. Project-specific configuration can be passed by --conf your/config.yaml, or creating a file cldflex.yaml

corpus

Basic usage:

cldflex corpus texts.flextext

Connect the corpus with the lexicon:

cldflex corpus texts.flextext --lexicon lexicon.lift

Create a CLDF dataset:

cldflex corpus texts.flextext --lexicon lexicon.lift --cldf

dictionary

Extract morphemes, morphs, and entries from lexicon.lift:

cldflex dictionary lexicon.lift

Create a CLDF dataset with a Dictionary module:

cldflex dictionary lexicon.lift --cldf

wordlist

Create a CLDF dataset with a Wordlist module:

cldflex wordlist lexicon.lift --cldf

API usage

The functions corresponding to the commands above are cldflex.corpus.convert() and cldflex.lift2csv.convert().

Configuration

There is no default configuration. Rather, cldflex will guess values for most of the parameters below and tell you what it's doing. It is suggested to start out configuration-free until something goes wrong or you want to change something. Create a YAML file for CLI usage, pass a dict to the convert methods.

  • obj_lg: the object language
  • gloss_lg: the language used for glossing / translation
  • msa_lg: the language used for storing POS information
  • lang_id: the value to be used in the created tables
  • glottocode: used to look up language metadata from glottolog
  • csv_cell_separator: if there are multiple values in a cell (allomorphs, polysemy...), they are by default separated by "; "
  • form_slices: set to false if you don't want form slices connecting morphs and word forms
  • mappings: a dictionary specifying name changes of columns in the created CSV files

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cldflex-0.1.1.tar.gz (21.6 kB view details)

Uploaded Source

Built Distribution

cldflex-0.1.1-py3-none-any.whl (22.8 kB view details)

Uploaded Python 3

File details

Details for the file cldflex-0.1.1.tar.gz.

File metadata

  • Download URL: cldflex-0.1.1.tar.gz
  • Upload date:
  • Size: 21.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.0 CPython/3.8.10 Linux/5.15.0-73-generic

File hashes

Hashes for cldflex-0.1.1.tar.gz
Algorithm Hash digest
SHA256 6381ebf7b0b2647d0d0c6ddd8bf48acdba5c2dbfc00e3effc407e02a319a3606
MD5 e707ea125c3aa5d10c68e2d207c34c99
BLAKE2b-256 fb71ff444f58d4d9491396817ae180ad68cb3ee9bc95984859fa110bd7be1402

See more details on using hashes here.

File details

Details for the file cldflex-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: cldflex-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 22.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.0 CPython/3.8.10 Linux/5.15.0-73-generic

File hashes

Hashes for cldflex-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 87cbcb2eb15ae1cf41becb8793af1f70d057dd6b5ea67dbfca0110ffd8add4ec
MD5 d2023200cbdb512ea7ef70cd6c8495f0
BLAKE2b-256 4489e9bd891849896a60286a473a4e941ffee7ccde195b4c57af1768ee677276

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page